-
Google is indexing a development site
Our development site is indexed by google. How big a problem is it? It is indexed from 28th August. Is it advised to use a new domain?
Also, what is the correct way of preventing google from indexing a website?
Is it with robots.txt, inserting a meta tag, or with htaccess?
-
6 Replies
-
robots.txt
-
A no index tag would do it
-
Remove the `disallow` rules from your `robots.txt` file, and make sure every page has `noindex` meta tags on it.
Excluding it from crawling with `robots.txt` is something that seems like a good idea, but if it’s already entered Google’s or Bing’s index, as it seems to be the case, it’ll actually ensure that it’ll never leave their index by blocking their crawlers from discovering that the pages have `noindex` tags.
-
As everybody said, is easy to block. But nobody gave you the answer to how big of a problem is it.
If it’s a small page or it doesn’t depend too much on organic traffic from search engines it’s ok. If you depend on google you will probably be penalized because of duplicated content.
I have seen both sides, sites disappearing forever or only for a couple hours / days
-
Fuck is my website going to be penalized? I never thought of this. I have an unaltered theme on my site and it is live, so there is duplicated content.
-
For my dev site on WordPress I also use a maintenance plugin so it’s just shows a maintenance page. I access the site to work on it logged into the WordPress dashboard.
Log in to reply.