:: 11-Apr-2004 14:51 GMT (Sunday) ::
The new statsbox is built out and I’ve got the backup of the statsdb loaded
and audited. Apache and PHP are up and running and I’ve got our admin
scripts loaded and going.
I’ve done some preliminary tuning of PostgreSQL and I’m not seeing any
disk-based sort_memory used during the normal stats processing process.
Some timing comparisons between blower and the new box:
Sample run from blower’s logs (45 minutes total):
00:49 (statsbox-iii/r72) Beginning daily processing routines
01:10 (statsbox-iii/r72) Daily processing for 20040121 has completed
01:10 (statsbox-iii/ogr) Beginning daily processing routines
01:34 (statsbox-iii/ogr) Daily processing for 20040121 has completed
On the new box (9 minutes total):
19:40 (statsbox-iv/r72) Beginning daily processing routines
19:46 (statsbox-iv/r72) Daily processing for 20040205 has completed
19:46 (statsbox-iv/ogr) Beginning daily processing routines
19:49 (statsbox-iv/ogr) Daily processing for 20040205 has completed
I’m seeing raw hourly log import times in the 20-30 second range. Overall
it’s taking the new box ~32 minutes to import an entire day’s worth of logs
for both projects combined.
This is giving us a total daily processing time of 41 minutes with log
saturation from the keymaster. At that rate, stats should be current
before too long.
Thanks again for your patience, everyone. We’re plugging away at getting
this box up and available again.