• White Hat SEO

  Robots.txt file advice

  Posted by GotYourTell1 on October 12, 2023 at 3:29 am

  I recently ran a Site Audit (using Microsoft Power Suite Site Audit tool) and one of the warning I recieved was:

  “A robots.txt file is not available on your website. Create a valid robots.txt file with instructions for the search engine bots.”

  I have no idea what this means, or if its necessary. Any advice would be appreciated!

  FWIW, I am a travel blogger who was crushed by the September HCU. Everything is completely white hat, just photos and guides to all of the places I go to, so I am doing whatever I can to improve the site’s technical integrity.

  GotYourTell1 replied 1 year, 8 months ago 1 Member · 0 Replies
• 0 Replies

• 

  Anonymous

  Guest

  October 12, 2023 at 3:37 am

  In theory a blank robots.txt file is fine. Sometimes I draw ascii art. If you really want something out of the search index, handle it in head tags.

Log In to Reply