Why? - Meta Robots Tag used

By: Devendra Jaiswal | Posted: 25th June 2006

Why? - Meta Robots Tag

Meta Robots Tag gives you the ability that a particular page should NOT be indexed by a search engine. To keep spiders out, simply add this text between your head tags on each page you don't want indexed.

However, there is no need for using Meta Robots tag if one is already using detailed robots.txt file to block any specific indexing.

The various commands used under Meta Robots Tag are:

Index: allows the spider to index that page.
Noindex: instructs the spider not to index the page.
Follow: instructs the spider to follow the links from that page and index them.
Nofollow: instructs the spider not to follow links from that page for indexing.

The format is shown below:

Open Head Tag
Open Tag Meta name="robots" content="index, follow" Closed Tag
Open Tag Meta name="robots" content="noindex, follow" Closed Tag
Open Tag Meta name="robots" content="index, nofollow" Closed Tag
Open Tag Meta name="robots" content="noindex, nofollow" Closed Tag
Clesed Head tag

Note: Use only one of the above given commands.

Most major search engines support the Meta robots tag. However, the robots.txt convention of blocking indexing is more efficient, as you don't need to add tags to each and every page. If you use do a robots.txt file to block indexing, there is no need to also use Meta robots tags.

If you have not specified any Meta Robots Tag on a page, by default, the spiders understand that the page and all the links appearing on that page are open for indexing. Therefore, it makes more sense to use this Meta Tag in case you don't want certain parts of your web page indexed.

Visit us:

http://www.halfvalue.com/top-articles/seo-resources.html

The author is a regular contributor to halfvalue.com where more information about SEO Resources and other more accessories is available. This article is free for republishing
Printed From: http://www.articlealley.com/article_66723_62.html
Back to the original article

Tags: meta tag, search engine, major search engines, spiders, contributor