Comments on: Different Types of Google Bots And How They Work

By: Houses For Rent

Houses For Rent — Mon, 29 Aug 2011 22:01:43 +0000

Thanks for explaining the different Google Bots, do you have any information about creating robots.txt files for these bots?

By: antheay

antheay — Sat, 28 May 2011 04:51:30 +0000

Its a good article..If you add more details about how google count fresh backlinks through algoritham then it will be more useful..thank you

By: Giorgio

Giorgio — Tue, 09 Feb 2010 09:03:31 +0000

Very interesting article!! Well done! ;)

Has ever happened to anyone to see a different response (status HTTP) from different Google bots that crawled the same resource at the same time? The right status code is 200 because the page is online and there are no redirections.

In other words:

66.249.71.153 – - [07/Feb/2010:02:44:11 -0600] “GET /mypage.htm HTTP/1.1″ 301 465 “-” “Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)”

66.249.71.42 – - [07/Feb/2010:02:44:11 -0600] “GET /mypage.htm HTTP/1.1″ 200 13942 “-” “Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)”

By: Nimit Kashyap

Nimit Kashyap — Fri, 05 Feb 2010 22:20:50 +0000

useful post, and i didn’t know about the fresh bot before.