Have you ever essential to avoid Google from indexing a particular URL on your world-wide-web internet site and exhibiting it in their research engine final results pages (SERPs)? If you manage world wide web websites extensive adequate, a day will very likely come when you require to know how to do this.
The three techniques most generally utilised to stop the indexing of a URL by Google are as follows:
Utilizing the rel=”nofollow” attribute on all anchor factors utilised to hyperlink to the web page to avoid the inbound links from getting adopted by the crawler.
Applying a disallow directive in the site’s robots.txt file to reduce the page from becoming crawled and indexed.
Using the meta robots tag with the content=”noindex” attribute to prevent the site from remaining indexed.
While the dissimilarities in the three ways surface to be subtle at initial look, the efficiency can range drastically dependent on which approach you select.
Making use of rel=”nofollow” to stop Google indexing
Lots of inexperienced website owners endeavor to avert Google from indexing a individual URL by working with the rel=”nofollow” attribute on HTML anchor features. They incorporate the attribute to each and every anchor factor on their internet site applied to connection to that URL.
Which includes a rel=”nofollow” attribute on a website link stops Google’s crawler from pursuing the link which, in change, helps prevent them from getting, crawling, and indexing the concentrate on site. Whilst this process could possibly function as a small-time period option, it is not a viable extensive-phrase remedy.
The flaw with this tactic is that it assumes all inbound one-way links to the URL will include things like a rel=”nofollow” attribute. The webmaster, even so, has no way to reduce other internet web sites from linking to the URL with a followed connection. So the chances that the URL will sooner or later get crawled and indexed utilizing this technique is pretty large.
Applying robots.txt to stop Google indexing
Yet another widespread system utilized to avert the indexing of a URL by Google is to use the robots.txt file. A disallow directive can be added to the robots.txt file for the URL in query. Google’s crawler will honor the directive which will stop the site from being crawled and indexed. In some cases, on the other hand, the URL can nevertheless appear in the SERPs.
Occasionally Google will display screen a URL in their SERPs even though they have never indexed the contents of that web page. If sufficient net sites hyperlink to the URL then Google can often infer the subject matter of the site from the hyperlink text of all those inbound back links. As a outcome they will clearly show the URL in the SERPs for linked queries. When making scraping google of a disallow directive in the robots.txt file will stop Google from crawling and indexing a URL, it does not ensure that the URL will in no way show up in the SERPs.
Working with the meta robots tag to protect against Google indexing
If you require to protect against Google from indexing a URL whilst also avoiding that URL from staying exhibited in the SERPs then the most powerful tactic is to use a meta robots tag with a written content=”noindex” attribute inside the head factor of the internet site. Of training course, for Google to in fact see this meta robots tag they will need to very first be capable to learn and crawl the site, so do not block the URL with robots.txt. When Google crawls the web page and discovers the meta robots noindex tag, they will flag the URL so that it will never be revealed in the SERPs. This is the most powerful way to avert Google from indexing a URL and displaying it in their look for benefits.