Regular QOT visitors would notice the site was down over last few days off and on with 500 Internal server errors. Still the site topped over 20,000 pageviews traffic everyday. The amazing secret is page caching!
When I tested most pages on the site, they would display 500 Internal server errors over the last few days and for a few days last week also. As I continued to send support tickets to Dreamhost, we expected a steep decline in site traffic, but on the contrary, if you see out usual traffic stats, the site traffic stayed not much lower than usual (Tuesday is still 4 hours to go).
Here is how the hourly traffic looks like today even though the site was down with internal server errors since morning!
This is all thanks to page caching by the amazing WP Supercache wordpress plugin which converts our wordpress php pages to html, and then caches and served them to visitors without any database requests, enabling this popular site to run over 700,000 pageviews per month on a shared hosting server on our usually amazing hosting by Dreamhost.
When the database errors persisted, even this plugin went down, but these cached html pages did not expire (since the plugin was not working!) and kept on serving html pages when visited. Moreover, since comments did not get posted due to internal server errors, and the pages were not refreshed by new comments and continued to be served.
What is great about caching is that the most visited pages by referred links and google traffic were already cached. The lesser visited pages were not cached and showed internal server errors, but that did not matter much in terms of traffic. Strangely none of the cached urls worked for me in Firefox, but were accessible in Internet explorer and Google Chrome, maybe someone can explain why (The cache was clear).
Since html pages were working, we changed the index.php page of our front page to a simple custom index.html page which had no php queries whatsoever. So the front page started working and if visitors wanted to subscribe, the email and RSS links were already working. The most popular pages were already cached, so most visitors with a high bounce rate and who did not interact further with the site in terms of comments, searching or link browsing would never notice the site was down!
So this is my story about maintaining site traffic with internal server errors, and I thank and highly recommend WP-Supercache for the same, without which we could not tide these errors. The site has just started working and some internal server errors are still on, and I am just sending this article through to let you know… till they start again.
Update: For those who inquired, traffic and ad revenue was 20-30% lower during these days. Most users never noticed the site was down and even tech support had to be convinced by pointing out to the wp-admin login url that the site was indeed showing errors and was down.