Lunarpages is kicking me off their servers for using too many resources. I don’t know when I’ll have time to move the site. Since it’s using too many resources I only have 2 options, either re-write it so it uses lots less or use a dedicated server which will cost way too much money. Plus I have to find a new ISP that can support my traffic without charging me an arm and a leg.
So, Greggman.com will be offline for at least a little while. Sorry, I also am turning comments off since I don’t know when they will shut me down at which point I will not be able to backup the site.
You must have some buggy code… scripts going into infinite loops or something, cuz how can you be that popular? How many visitors do you get a day?
Anyways, you don’t have to go dedicated to get higher-resource hosting. You need what’s known as “E-Commerce” hosting or “Virtual Private Server”.
Well, I get about 0.7 gig of traffic a day. About 7000 pages served a day. Not that many but it works out. My pages are pretty heavy so at 100K a page * 7000 pages = 0.7 gig of traffic.
The thing is the pages are processor heavy as well. All the side lists are generated on the fly. For example the Category list, News List, Feature List, etc. The base data is sitting around already in a table in the correct order for each thing, newest to oldest or in the category case as you see it, but for example to display the right column feature list the entire feature list is loaded, the top 3 newest items are extracted and then a random 12 more.
That is done for each list on the right.
All images are checked to see if they have correspondiing larger file (ie, click to enlarge), and all images are checked to see if a border should be added (the drop shadows you see on some images).
The comments are looked up and displayed which includes re-formating them and filtering the HTML, etc.
Users are tracked, if you are logged in then there will be “edit” links on your comments.
When I moved to Lunarpages they complained the script was using too many resources. I changed it so it cached the front page since that one does the most work. Being built on the fly it also had to get the lists of features, news, blog etc, then for each article, lookup that articles pieces, extract a blurb, icon, # of comments, etc. Anyway, I cached that do now it’s only remade when it needs to be.
That was fine for about 6 months at Lunarpages but then 3 weeks ago they kicked me off, no warning, no chance to fix it, just “your account is being cancelled”
So, I’ve moved to this new server, and changed it so it caches almost all pages and I split the scripts up so that the parts that generate the page don’t have to get loaded if it’s just going to send the cached version. I also had to remove the “lastest comments” from everything but the front page (otherwise any new comment would invalidate all pages [*])
I had to remove the custom edit links. Now all messages have an edit link but you should be refused editing if you are not logged in and the message doesn’t belong to you. That way there only needs to be one cached page.
Note that caching is separated by language, browser and format so that any one page could have about 20 different versions in the cache. The page served to a cell phone is different than Netscape 4.7- which is different than webTV which is different then everything else plus there’s the print version of each page, etc.
Anyway, I thought that would be enough but regardless it’s running REALLY SLOW on this new server and I’m pretty sure it’s a configuration problem.
[*] I suppose I could put the side lists in separete files and do server side includes for them, that would get rid of some more processing but as they are cached now it’s no big deal I think.