<META name robots>Description:Generally you should not need to worry about which pages within your site are indexed by search engines. Most search engines will attempt to spider every page you have on your site. Occasionally there are valid reasons why you do not want this to occur. Perhaps you have a slideshow presentation which needs to be viewed from start to finish. It would be awkward for someone to be dumped in the middle of the presentation. In this case, you can use the ROBOTS Metatag to control exactly which pages are indexed by search engines. Unfortunately, this tag is not widely recognized as of yet. Lycos does use the tag, and the Atomz site-specific search engine also honors the tag. If you need to ensure precise control over what is and is not indexed on your site, use the more widely recognized robots.txt file. Attributes:
Examples:
|