It’s been a long weekend modifying update feed php scripts as well as removal scripts on the KickRSS server. Last two weeks while traffic was going up I noticed speed and performance was going down. By Friday night mySQL was crashing from far too many processes running.

Turns out our feed updates (some starting every five minutes) were becoming a major hog on server resources. I spent most of Saturday playing with the remove old feeds functions (which kills content older than a selected date) and I think I’ve got it running smoothly for now.

We were killing old feeds every 30 days but I was only running the cron one time per day. That made sense since it would kill all content older than the selected time. However, it may have to kill 156,059 rss feeds on a single run, since the database has been growing daily.

Instead of killing feeds in one big chop at one time per day I modified settings to bite off little pieces of old feeds every other hour. I was not thinking worldwaide time zones and did not realize at first that data could be deleted in intervals.

I also spread out the updates of feed content into 2000 feeds at time blocks. Maybe not the best way to do it but things appear to be running better than they were on Friday and content appears to be getting updated in a reasonable amount of time.

One of the next projects on my list is to split up the RSS Users Page so that all active user accounts don’t load up all at once. I’ll either break that page into groups or create multiple user categories with better organized feed data.

If anyone is interested in doing some php/mysql work to clean up the appearance and load times for the user pages get in touch with me. I’m looking for a good php, mySQL person anyway.

Cheers!