Monday, December 2, 2002
We're a Little Too Clever For Our Own Good
Posted by Jason Dunn in "UPDATE" @ 01:50 PM
Most of you will have noticed that this site has been up and down today like a yo-yo - we've had three complete melt downs where processes have gotten completely out of control, resulting in a server too slow to process anything. Here's why...
Web-site functionality is a double-edged sword. On one hand, our integration of the front page with phpBB is very functional and useful. It allows us to mine the database in different ways, exposing useful elements like the last ten posts, the top stories, number of responses to each post, a survey, and more. The problem is that for each one of those clever integrations, it requires a query to our database. Each query requires RAM and CPU resources to process - I believe our opening page has in excess of 25 SQL queries, and when 1000's of you are visiting every day, the net result is an overloaded server. Dynamic sites may be "cool", but they're brutal on a server if not designed to scale with large amounts of traffic. Every time you visit the opening page, if you're cookied, you get logged into the phpBB forum. Clever, yes, but resource consuming like you wouldn't believe!
We're working on several ways to combat this problem, including the potential removal of some of functionality of the opening page. We're going to try and implement a caching solution that will reduce the load, but if that doesn't work something has to change - we can't continue to offer rich functionality if it kills the site on a regular basis. Thanks for your patience in this while we work through the issues, and I'll keep you informed of any changes that we need to make that will impact your use of the site.
Web-site functionality is a double-edged sword. On one hand, our integration of the front page with phpBB is very functional and useful. It allows us to mine the database in different ways, exposing useful elements like the last ten posts, the top stories, number of responses to each post, a survey, and more. The problem is that for each one of those clever integrations, it requires a query to our database. Each query requires RAM and CPU resources to process - I believe our opening page has in excess of 25 SQL queries, and when 1000's of you are visiting every day, the net result is an overloaded server. Dynamic sites may be "cool", but they're brutal on a server if not designed to scale with large amounts of traffic. Every time you visit the opening page, if you're cookied, you get logged into the phpBB forum. Clever, yes, but resource consuming like you wouldn't believe!
We're working on several ways to combat this problem, including the potential removal of some of functionality of the opening page. We're going to try and implement a caching solution that will reduce the load, but if that doesn't work something has to change - we can't continue to offer rich functionality if it kills the site on a regular basis. Thanks for your patience in this while we work through the issues, and I'll keep you informed of any changes that we need to make that will impact your use of the site.
