Google (XML) Sitemap Generator Plugin is a popular wordpress plugin which can generate a search engine compatible sitemap of your blog in seconds … but it was leading to Dreamhost to terminating our wordpress scripts and resultant internal server errors.
Internal Server Error Causes
Following investigation into cause of our persistent internal server errors, Dreamhost support kept informing us repeatedly to identify the wordpress scripts which were tripping the server. They said it was due to scripts being stopped by the Process Watcher, as the memory limits were being crossed on the shared server. They advised we should find causes of heavy usage, especially 3rd party plugins which have poor memory management (better to delete than deactivate). Of course they promoted Dreamhost PS with higher memory limits to meet our site requirements.
Sitemap Generator WordPress Plugin and Memory Usage
So how much memory was Sitemap Generator consuming everytime it generated a sitemap of over 3000 urls? Around 25MB!
And to top that it was eating up that much memory every time we edited, deleted or created new articles. That was a setting which is usually enabled to have the most current sitemap.
And when I often tried to rebuild the sitemap manually, I got this error
Basically the script was taking too much memory (from the entire server which also hosts other sites) everytime it generated a sitemap with so many urls and the Dreamhost Process Watcher used to stop the scripts automatically due to your sites going over Memory limits, which basically stopped all WordPress processed till some tech support guy manually released the process watcher and activated the site. It was only thanks to caching that the site traffic sustained despite these errors.
Though this happened several times over the last few weeks, I was able to identify the plugin causing the memory usage spikes only recently. Thereafter I deactivated the plugin. It is obvious that if we need to run this plugin with all that memory usage, it is not possible on shared hosting and you need to upgrade to Dreamhost Private Server where the memory limit is as high as you set it.
Advanced Sitemap Settings
But the sitemaps are essential to let search engines know about your latest urls. The plugin author has created a wonderful FAQ which explains all these issues. Some which relates to us like minit number of posts, Increase server memory limit as well as increase maximum execution time.
Since it is already consuming too much memory, raising the memory or script execution time might not help. So we can continue to use the plugin after limiting the number of posts, which seems like the most practical solution as of now.
We also upgraded from PHP4 to PHP5 and both these fixes should take care of our internal server errors. Keeping our fingers crossed …
NOTE: The Google (XML) Sitemap Generator plugin is a very useful and highly recommended plugin. This article is for informational purpose to inform that large wordpress blogs will require larger memory to generate sitemaps and sometimes it may be beyond allowed memory limits on shared hosting servers. So either you upgrade to VPS hosting or tweak the sitemap plugins to work on a limited basis.