It’s not perfect yet but after 30 days it appears that the new setup for running cron on updating feeds for KickRSS has taken some of the load off of the server.

What was happening is I was running updates on feeds (over 12,000) every few minutes. As a result cron jobs were taking longer to load and complete and at some points the input_feeds option would just be choking the server.

What I did was split feed updates into groups of 1000 (input_feeds 1000 2000) and ran those are different times 2x daily. It’s resulted in less up to date feeds (up to day in most cases rather than up to the minute) and less load on the server.

I’m still noticing some pressure on server load at certain times and some feed updates are timing out. Hopefully in 30 days from now I’ll have even a better solution.

Here are some examples of feed input reports…

Feeds 7000-8000
Total time: 422 seconds
Total SQL queries: 10641
Total RSS feeds fetched: 925
Total RSS feeds not updated: 0
Failed RSS feed updates: 4
New articles added: 205
Average speed per RSS feed 0.456216216216

Feeds 2000-3000
Total time: 921 seconds
Total SQL queries: 11515
Total RSS feeds fetched: 878
Total RSS feeds not updated: 0
Failed RSS feed updates: 6
New articles added: 471
Average speed per RSS feed 1.04897494305

Feeds 7000-8000
Total time: 1024 seconds
Total SQL queries: 10455
Total RSS feeds fetched: 923
Total RSS feeds not updated: 0
Failed RSS feed updates: 7
New articles added: 82
Average speed per RSS feed 1.10942578548

These are all from yesterday. I’d like to see the times improve. Will post on updates later!