Comments on: Slow Googleblot Crawling: Stop Crashing Shared Hosting Servers

By: James beuthling

James beuthling — Mon, 26 Oct 2009 15:24:29 +0000

I to experienced the full force of goog – 5000 + page request a day wtf. Sometimes 10 request a secound. And mysite uses http and databases. Ate my limit up pretty quick. And to beat that goog has asign me a special crawl rate that I cannot adjust unless i personally request it.
@webmaster – you can sometimes set crawl rate thu the google webmastertools link. The main google settings page, with your gmail and adwords and stuff.

By: www.eng-rom.com

www.eng-rom.com — Sat, 31 Jan 2009 07:49:37 +0000

Google crawl only 3 pages per day… WTF? I have optimized the site… I have a sitemap.. I wast my time with this stupid googlebot

By: Best of February 2008

Best of February 2008 — Wed, 24 Dec 2008 09:14:28 +0000

[...] Slow Googleblot – Stop crashing shared hosting servers. [...]

By: Experiences with Downtime, Dreamhost, Googlebot and SSH

Experiences with Downtime, Dreamhost, Googlebot and SSH — Mon, 26 May 2008 08:13:40 +0000

[...] This is a temporary measure as you finally want Googlebot to index your site. You can also slow Googlebot crawl and help your shared hosting. Now we could restore our websites back. Go to the top [...]

By: ?????? ??? ????

?????? ??? ???? — Wed, 14 May 2008 07:59:26 +0000

great info

By: Indexed by Google in 2 Minutes!

Indexed by Google in 2 Minutes! — Tue, 11 Mar 2008 18:23:37 +0000

[...] Google Webmaster tools, Googlebot is set at normal crawling speed since the last month (after I had slowed Googlebot down) and is not set on the faster crawling mode (will that mean instant indexing by [...]

By: webmaster

webmaster — Tue, 19 Feb 2008 11:26:39 +0000

Thanks for the info.
am already using Google Analytics for my site but
how can set crawl rate by my own?