:: 09-Dec-1999 03:53 (Thursday) ::

For those of you who are either hyper-observant, or addicted to
stats, you will notice that although the CSC main page says we
did 1.7% of the keyspace yesterday (which we did), and reports
that we’ve completed 26.9% of the keyspace overall (which we
have), that this math doesn’t make sense since yesterday it
reported that we had completed 26.4% of the keyspace overall
(which we hadn’t). This was because I forgot to delete a
record when the Dec 5 statsrun got botched.

This only affects the main CSC stats page and the CSC graphs.
No participant info was harmed in the making of this ‘whoops’.

All is now well again on that front.

Unfortunately, rc5-64/psearch.php3 is now broken. To make a long
story short, nugget had backed out a change made to this page
on the live site manually. I then fixed the file and put the
fixed version into CVS, and ran the normal command to update
the main site from CVS. CVS saw that the version of psearch on
the main site had been modified, so it created an ‘annotated
checkout’ of the file, with some nice commentary to let us know
what changed. Of course, PHP has not clue what all the extra
stuff in this file is supposed to do, so trying to pull up that
page results in a ‘The document contains no data’ error. Since
I don’t have root on the box, I can’t fix the garbled file either.
So, it’ll be broken until I can find someone rootly to fix it,
which hopefully won’t be long.

Sorry for the cowfusion and thanks for your patience!

:: 08-Dec-1999 10:09 (Wednesday) ::

I’ve just made version 1.1 of the LOGVIS (Log Visualizer) utility
available for download. It adds a number of improvements in user
interface, including drag-and-drop file acceptance, command line
filename opening, Reload menu option, Status bar text updates, and
several other minor things. You can find the download link on our
addons page: http://www.distributed.net/download/addon.html

:: 07-Dec-1999 14:56 (Tuesday) ::

I guess now that I have a .plan, I really ought to have a plan.

By way of introduction, I’m the guy who eats, drinks and
breathes SQL. I also have a Real Job, a wife and a baby on the
way. Whatever time is left after all of the above usually
goes to Starcraft. If you want to know more, check out my home
page at http://www.geocities.com/bruce-wilson

At the moment, I’m trying to help Nugget and Decibel stay out
of trouble, and to avoid opening any big cans of worms
with stats until (a) the changes Nugget and Decibel and I have
discussed are in place and working, (b) I have a good feel for
the flow of the data, and (c) my wife delivers the baby (any
day now).

Moo!

:: 07-Dec-1999 05:49 (Tuesday) ::

“Uhoh.”

– Number one thing you don’t want to hear a stats person say while
testing new statsrun code

For all you stats-junkies that noticed, stats was down for several
hours. I had changed the queries that insert new participants into
the main participants table in the interest of cleaner and faster
code.

Unfortunately, what should have been a relatively simple change didn’t
work. The net result was that some 13,000 emails were added to
STATS_participant even thought they were already in the table. This is
a BadThing(tm).

To make matters worse, while deleting the duplicated records, I ‘went a
bit too far’, and deleted new rc5-64 participants from yesterday (57
people all told). I was able to add these records back in, but any changes
made to these records during the day yesterday (such as joining a team,
setting up your friends, adding a motto, etc.) were lost.

Anyway, everything is pretty much back in one piece again. Sorry for all
the cowfusion!

On a different note, as Nugget mentioned, we found out what was slowing
statsbox to a crawl before. It was a change made right before the slowdown
that really shouldn’t have had the impact that it did. The only thing we
can figure is that the change was just enough to cause sybase to start
turning over it’s internal cache at a very rapid rate. Another BadThing(tm).

Bruce, Nugget, and I came up with a plan for fixing this on a more long-term
basis, which we should be implementing sometime this week. Once we do
implement that change, we should be able to make some other changes that
should speed up the statsrun.

RC5 stats are now running, and I’ve got apache disabled to speed them up as
much as possible. Stats should be back up in an hour or two.

Thanks as always for your patience.

:: 06-Dec-1999 22:47 (Monday) ::

The glibc2.0, nomt linux-ppc client is on the FTP sites.

A glibc2.1 mt client will hopefully follow shortly, but I need to
upgrade my build machine first. I won’t be incorporating the AltiVec
cores until binutils supports AltiVec.

:: 06-Dec-1999 05:58 (Monday) ::

By popular request, a new add-on utility is being provided to people
who have not been able to live with the new Win32 client because it
couldn’t “moo” like the old Win32GUI client could. The MooSounds
utility was developed by Scott Ludwig and myself, and a download link
has been provided on http://www.distributed.net/download/addon.html

:: 04-Dec-1999 23:24 (Saturday) ::

Congratulations (and thanks, to those of you that have helped!), we’ve
reached the 20% complete point in the CSC challenge. The odds are
getting better and better that we find the key in the next day.

We’ve reached the point where we’re about to start recycling keyspaces. This
means that we’ve assigned almost all the keys in the CSC keyspace and need
to begin reassigning keys that have yet tobe returned. Optimally, we would
like to reassign keys in the order that we originally assigned them (so that
minimal work is wasted), but due to our “keyspace jumping” method of
assigning keys, this isn’t realistically possible. Regardless, we will be doing
our best to reassign in the rough order that we started with.

This should serve as a significant reminder to most people that it’s not a good
idea to buffer an amount of blocks that your computer(s) couldn’t complete in
four or five days. Accordingly, now is an optimal time to review your
configuration for optimal performance, and to flush any outstanding
CSC blocks.

Keep on cracking. Good luck!

-dbaker

:: 03-Dec-1999 04:45 (Friday) ::

Well, just when it looked like things were in good shape.. :) Just can’t
win today, I guess.

As you can see, stats are nearly back to full operations. It’s been a busy
36 or so hours catching up the rc5-64 data, but we’re back on track there.
The only little hitch from today is that the csc stats for 2-Dec are
incomplete. For some reason the 23:00 through 23:59 logfile never made
it across from the keymaster.

Normally I’d just re-run the day’s stats, but rc5-64 stats are running
now and I’m reluctant to try running both at the same time. So…
the numbers for yesterday are ~1/24th too low, and I’ll re-process with
the missing logfile in the morning.

Moo.

:: 03-Dec-1999 00:32 (Friday) ::

I’d like to encourage all the people sending me Mac port bug reports
to use bugzilla instead. It lives to serve you. Feed it.

http://www.distributed.net/bugs/

In port news, AltiVec RC5 support is definitely in the next beta
release. The coders are trying to get one out tonight, but tomorrow’s
more likely.

:: 01-Dec-1999 21:24 (Wednesday) ::

Well, this is most excellent news. We’ve isolated the slowdown to
a single php page on the stats site, and a single change made to
the code on that php page.

We had recently made a modification to the /rc5-64/psearch.php queries
that pulled the search data from a different table than it normally
pulls from. For some reason, this is running much, much slower than
it should and is causing the incredible slowdown while stats are
up and available.

When we remove access to this one page (rc5-64/psearch) stats seems
to run just fine and dandy. So, at this point, the worst-case scenario
we’re looking at is not being able to do a participant search while the
daily processing is running. At least until we can figure out why
the query is running so slowly.

I’ve already rebuilt the relevant indexes and that hasn’t helped any,
so there’s still more debugging we need to do. The good news is that
since we know what has to happen to repair the performance, I can
go ahead and start catching up the rc5-64 data to bring it current.

More details as the situation progresses.