Meta Tag Googlebot - SEO Checklist
Last updated on June 1st, 2023 by RGB Web Tech
Googlebot is the general name for two different types of crawlers: a desktop crawler that simulates a user on desktop, and a mobile crawler that simulates a user on a mobile device. Your website will probably be crawled by both Googlebot Desktop and Googlebot Smartphone
Googlebot Meta Tag
Meta Name = "Googlebot" Since June 2017, Google announced that they are no longer supporting using the description and title from DMOZ, the Open Directory Project.
Sometimes an old description tag remained in the index of Google. Despite the fact that you changed relevant information on your website and submitted this to Google, the old description stayed in the Google search results.
This problem originated from results in the Open Directory Project DMOZ . Your website, including the old meta description, also had a description and a link in the ODP listings. Googlebot used this description while spidering your website.
Using the Googlebot meta tag you could tell the Googlebot NOT to duplicate the ODP description but the description which was located on your website.
Key Takeaway
Now the Googlebot spiders your desktop and mobile website (smartphone).
You can still keep the Googlebot away from some pages of your website if needed.
You define those pages in the robots.txt file. But remember, if you put a page online it will be found eventually! Don;t want a page to be spidered? Then don’t put it online on your website.
HTML Code Google Bot Tag Example
Why Did The Googlebot Tag Exist?
Sometimes an old meta tag, the description tag, remained in the index of Google. Despite the fact that you change relevant information on your website and submit this to Google, the old description remains in the Google search results. Yahoo and Live will index the new information, Google won’t.
This problem originated from the results in the Open Directory Project DMOZ (also check the relations between the search engines). Your website, including the old meta description, also has a description and a link in the ODP listings. The Googlebot has used this description while spidering your website.
5 Variations
There used to be 5 variations
1. Googlebot: noarchive – does not allow Google to display cached content
2. Googlebot: nosnippet – does not allow Google to display excerpted or cached content
3. Googlebot: noindex – similar to the robots meta element
4. Googlebot: nofollow – does not allow Google to pass any PageRank or link popularity to the link served.
5. Googlebot: no archive - Google is not allowed to place this page in the cache memory
If you have more website list other than our list, Please sent us an email we will enclude those websites into our website list.
Written by RGB WEB TECH
Design, Development & Marketing Company
We are a startup and innovative IT Company. We provide complete Web & APP Design, Development and Marketing solutions.