Apache Drupal XML with statistics - High load on events
Today we had another event with Ski Classics. And as usual we find ourself having performance issues.
We have XML files of statistics showing stats of the race at the same time we are streaming the event. The streaming is done by a third part and we just embed a jw player for flash and html 5 playback for iOS support.
This causes the server/apache to use a lot of memory even on small number of users.
We are dealing with this problem currently by downloading the part of the page evert 5th second and then store it as clean html. This makes sure that the users only view fixed html version of the page and decrease the server load a lot (have occasionally seen 8 processors been reduced from 99% usage to 10 %).
I do this my using the watch command (watch -n 5 ./script.sh) on a script.
The script first sets the file and then reloads it with wget.
If anyone have any other solutions please let me know.