:: 21-Dec-1999 05:39 (Tuesday) ::
During tonight’s stats run things got extremely unhealthy during the
creation of the rc5-64 overall email rankings table. Sybase consumed
all it’s locks and several processes became deadlocked and refused to
die gracefully. (Heck, they wouldn’t even die screaming).
We spent a few hours poking and prodding, trying to convince the blocking
process to go away to no avail. Unfortunately, the only recourse at that
point was to down the whole server in a rather indelicate manner. (not
quite a kill-9, but pretty close).
Right now we’re sifting through the data now that the server is back
online. We’re trying to determine the scope of the damage (if any)
and get a feel for the situation.
More info as the situation progresses…