| |
<META name googlebot>
Generally you should not need to worry about which pages within your
site are indexed by the
Google search engine (which uses the
Googlebot
spider). By default,
Google will attempt to spider every page you have on your site.
Occasionally there are valid reasons why you do not want this to occur.
Perhaps you have a slideshow presentation which needs to be viewed from
start to finish. It would be awkward for someone to be dumped in the
middle of the presentation.
In this case, you can use the ROBOTS
metatag to control the behavior of all spiders, and the GOOGLEBOT
Metatag to control exactly which pages are indexed by Google.
| <meta
name="Googlebot" content="text"> |
|
text is one of the following four combinations.
| all |
Same as index,follow. |
| index,follow |
The default, meaning index the page
and follow all links from the page. |
| noindex, follow |
don't index the page but do follow
all links from the page. |
| index,nofollow |
index the page, but do not proceed to
the links from the page. |
| noindex,nofollow |
do not index the page and do not
proceed to links from the page. |
| none |
same as noindex,nofollow. |
| nosnippet |
From Google help: "A snippet is a
text excerpt from the returned result page that has all query
terms bolded. The excerpt allows users to see the context in which
search terms appear on a web page, before clicking on the result.
Users are more likely to click on a search result if it has a
corresponding snippet." This value of NOSNIPPET removes the text
snippet. |
| noarchive |
From Google help: "Google keeps the
text of the many documents it crawls available in a cache. This
allows an archived, or "cached", version of a web page to be
retrieved for your end users if the original page is ever
unavailable (due to temporary failure of the page's web server).
The cached page appears to users exactly as it looked when Google
last crawled it. The cached page also includes a message (at the
top of the page) to indicate that it's a cached version of the
page."
"If you want to prevent all robots from archiving content on
your site, use the NOARCHIVE meta tag." |
|
<meta name="GOOGLEBOT" content="noindex,nofollow">
|
Don't index the current page, and do not continue
following links from the page.
  
|