:: 07-Dec-1999 05:49 (Tuesday) ::
“Uhoh.”
– Number one thing you don’t want to hear a stats person say while
testing new statsrun code
For all you stats-junkies that noticed, stats was down for several
hours. I had changed the queries that insert new participants into
the main participants table in the interest of cleaner and faster
code.
Unfortunately, what should have been a relatively simple change didn’t
work. The net result was that some 13,000 emails were added to
STATS_participant even thought they were already in the table. This is
a BadThing(tm).
To make matters worse, while deleting the duplicated records, I ‘went a
bit too far’, and deleted new rc5-64 participants from yesterday (57
people all told). I was able to add these records back in, but any changes
made to these records during the day yesterday (such as joining a team,
setting up your friends, adding a motto, etc.) were lost.
Anyway, everything is pretty much back in one piece again. Sorry for all
the cowfusion!
On a different note, as Nugget mentioned, we found out what was slowing
statsbox to a crawl before. It was a change made right before the slowdown
that really shouldn’t have had the impact that it did. The only thing we
can figure is that the change was just enough to cause sybase to start
turning over it’s internal cache at a very rapid rate. Another BadThing(tm).
Bruce, Nugget, and I came up with a plan for fixing this on a more long-term
basis, which we should be implementing sometime this week. Once we do
implement that change, we should be able to make some other changes that
should speed up the statsrun.
RC5 stats are now running, and I’ve got apache disabled to speed them up as
much as possible. Stats should be back up in an hour or two.
Thanks as always for your patience.