20K Pageviews Traffic Despite Internal Server Errors!

By Posted 2009 Updated   BloggingDomain HostingSite NewsTutorialsWordPress

Regular QOT visitors would notice the site was down over last few days off and on with 500 Internal server errors. Still the site topped over 20,000 pageviews traffic everyday. The amazing secret is page caching!

When I tested most pages on the site, they would display 500 Internal server errors over the last few days and for a few days last week also. As I continued to send support tickets to Dreamhost, we expected a steep decline in site traffic, but on the contrary, if you see out usual traffic stats, the site traffic stayed not much lower than usual (Tuesday is still 4 hours to go).

server down traffic

Here is how the hourly traffic looks like today even though the site was down with internal server errors since morning!

server down hourly

This is all thanks to page caching by the amazing WP Supercache wordpress plugin which converts our wordpress php pages to html, and then caches and served them to visitors without any database requests, enabling this popular site to run over 700,000 pageviews per month on a shared hosting server on our usually amazing hosting by Dreamhost.

When the database errors persisted, even this plugin went down, but these cached html pages did not expire (since the plugin was not working!) and kept on serving html pages when visited. Moreover, since comments did not get posted due to internal server errors, and the pages were not refreshed by new comments and continued to be served.

What is great about caching is that the most visited pages by referred links and google traffic were already cached. The lesser visited pages were not cached and showed internal server errors, but that did not matter much in terms of traffic. Strangely none of the cached urls worked for me in Firefox, but were accessible in Internet explorer and Google Chrome, maybe someone can explain why (The cache was clear).

Since html pages were working, we changed the index.php page of our front page to a simple custom index.html page which had no php queries whatsoever. So the front page started working and if visitors wanted to subscribe, the email and RSS links were already working. The most popular pages were already cached, so most visitors with a high bounce rate and who did not interact further with the site in terms of comments, searching or link browsing would never notice the site was down!

So this is my story about maintaining site traffic with internal server errors, and I thank and highly recommend WP-Supercache for the same, without which we could not tide these errors. The site has just started working and some internal server errors are still on, and I am just sending this article through to let you know… till they start again.

Update: For those who inquired, traffic and ad revenue was 20-30% lower during these days. Most users never noticed the site was down and even tech support had to be convinced by pointing out to the wp-admin login url that the site was indeed showing errors and was down.

4 comments on “20K Pageviews Traffic Despite Internal Server Errors!

  1. Sumesh says:

    I think admins don’t see cached pages, which could explain why you saw server errors only in Firefox (assuming you were logged in only in it).

    • QuickOnlineTips says:

      @Sumesh – admins don’t see cached pages >> that would explain it for Firefox.

  2. David says:

    So what ended causing your internal server error? I had the same thing happen.. i think mine was caused by popular posts plugin.. but i’m not sure.. I removed super-cache just to be safe.

  3. ThePicky says:

    @QOT – Dreamhost is overgrown, you might think of switching host. You can try mediatemple, eleven2 hosting or something reliable. I don’t want to recommend hosting it’s up to you, but I tried mediatemple grid server, our sites(3 combined ) has more than 20K page views daily, because of GPU usage it didn’t work. We are with eleven2.

Leave a Reply

Your email address will not be published. Required fields are marked *

Next Article »