:: 20-Nov-1998 19:10 (Friday) ::

STATSBOX-II MIGRATION
nugget@distributed.net

Anyone even remotely interested in distributed.net has probably been
frustrated at one time or another in having to deal with our quite
deficient statsbox. While statsbox is certainly the most visible
component of the distributed.net network, we’ve had to be conservative.
As we’ve been able to add hardware, we’ve opted to spend our dollars on
the actual keymaster and keyserver network. While a slow statsbox may
be frustrating, a slow keymaster could damage the integrity of the
entire project.

I’m pleased to be able to report that we’re making great progress on
the implementation of a replacement statsbox which will be much better
able to handle the extreme load required to serve up distributed.net’s
statistics. However, none of this is recent news. There is, however,
one detail that has been up in the air. This is also the primary cause
of the delay in implementation.

It’s been well over a year now since we introduced the original
statsbox and the database market has certainly been through some
incredible changes in that time. Most specifically, the recently-
announced Sybase ASE for Linux which brings a real, full-featured,
freely deployable SQL server to the unix world. While there are
attractive offerings from other database vendors, the Sybase license
alone is unique in that it is free for deployment as well as
development work. This is understandably an attractive feature for
distributed.net and its limited financial resources. The fact that the
existing code is all Transact-SQL has made moving to Sybase relatively
painless as well.

While I’ve been pleased with the performance of our existing NT+MSSQL
solution, it has been severely lacking in several areas not the least
of which being ease of remote management. It’s a real burden having to
remotely administrate an NT box that’s 3,000 miles away. This is
probably the most compelling factor influencing the migration to Linux.
Simply put, Unix is a better platform for us. With the now-available
Sybase product, we are no longer tied to NT and MSSQL.

Although there are still technical wrinkles to work out, I’m finally
comfortable enough that the Linux+Sybase solution is a viable
alternative for us, and as a result I now feel safe in announcing our
intentions. In other words, this is why it’s taking so long. :)

In measuring the performance of the Sybase/Linux installation, I’ve
found it to be close enough to the performance of NT+MSSQL to make
the migration quite painless from a capacity standpoint. So,
we take no performance hit and we gain the stability, ease of remote
management, and far greater flexibility provided by a unix solution.

More details will follow as things progress. Thanks again, everyone,
for your incredible patience and understanding. I know how frustrating
it’s been with the slow statsbox. Expect rapid progress.

:: 20-Nov-1998 17:11 (Friday) ::

New news added to http://www.distributed.net/pressroom/press-dnet.html —
Another Facet of the Internet Revealed, Article in the News Record
of the University of Cincinnati written by T. Nathan Mundhenk.

If you have any other sightings of press related to distributed.net,
please mail alde@distributed.net so it can be included in our pressroom.

–Pete

:: 20-Nov-1998 04:51 (Friday) ::

Another very productive day in statsbox-ii land. I spend most of the
day fiddling with the em_RANKing code from the old statsbox. Now that
I’ve built a golden participants table, and imported a huge chunk of
rc5-64 activity (start through mid-october) I actually have enough
data to start doing things. Remember, since we’ve moved to a
participant IDs system, *all* the code needs to change in one way
or another, so this is a pretty big undertaking. ID #’s were assigned
chronologically, starting with the rc5-56 logs.

Anyway, I’ve got the code that does the overall and yesterday
participant rankings wrapped up. It’s not optimized, but it’s
functional. Time to run dropped from 1:27 on the current stats
hardware to just under 22 minutes on the new hardware, and I’m
thinking that with additional tweaks I can get that under 15 minutes
if not cut it in half.

I’ve also been able to add stuff as well, for instance you will now
have the ability to cause your email to not display on the stats
site. Instead of “user@domain.com” you can have yourself listed
as “Participant #[x]” instead. While I really hope that this does
not become a commonly-used feature, I do recognize that there are
people with privacy concerns for who this is a necessary feature.
I’d certainly prefer to see people using this feature instead of
reverting to @hotmail.com-style shields.

And last, and possibly most importantly, the email retire_to feature
is fully implemented as well. This will allow people who are changing
email addresses to carry over their old blocks to their new email
address. Without a doubt, this has been the singlemost requested
feature for the stats site. Unfortunately, until I moved to a
participant ID structure, it was not technically feasible.

More big news soon, so keep your eyes open. It’s just starting to
get exciting!

:: 20-Nov-1998 00:44 (Friday) ::

Rewrote -until, -h and hour handling today. All of them are ‘:’ now aware.
Whoever wrote that stuff originally was one heck of a lazy SOB. -until works
again (obviously).

I rewrote Do…/UndoCheckpoint() and some buffer code as well. The client
only uses one checkpoint file (for all contests together) now. I think I
forgot to mention (last week?) that the checkpoint time interval option
is now obsolete. The interval is now computed dynamically and is the greater
of one percent average change and one minute.

Fetches and flushes invoked from the command line are now correctly logged
to file/mail now. I’m sure that those odd-ball users who don’t trust stats
will appreciate this.

Time stamps are now tagged as being “UTC” rather than “GMT”. Besides being
technically correct, I expect that this will greatly expand the vocabulary
of the people who thought GMT stood for “General Mountain Time” (yes, that
is a real quotation).

I haven’t heard from dbaker regarding the potentially-new euro proxy yet.
From what I hear, he’s very busy counting finger requests.

Davehart has kindly accepted to redesign the buffer file format. Although
we won’t (technically speaking) need it for implementing POW, its an ideal
opportunity I’d rather not pass up.

:: 18-Nov-1998 22:03 (Wednesday) ::

distributed.net .plans are now easily browsable in archive form through
the ‘Archive’ link off the main finger page. An example:

http://nodezero.distributed.net/cgi/planarc.cgi?user=vetere

This should prove more useful as our plans tip over the 128-line limit,
which they’re approaching…

:: 18-Nov-1998 15:19 (Wednesday) ::

The new beta testing signup page is online and operational. However, I’m
keeping the URL quiet for now. :)

Next up on list: distributed.net .plan archive display from a central
source and after that, stats emailing…

:: 18-Nov-1998 06:05 (Wednesday) ::

Not much happening for me. Just getting back into the swing of
things. Projects to come. More work from me to come.

:: 18-Nov-1998 05:18 (Wednesday) ::

The connectivity problems at the master’s site have been resolved
and the proxy network is now operating normally.

:: 18-Nov-1998 00:59 (Wednesday) ::

The keymaster is currently having some connectivity problems and has
been temporarily shutdown until routing is again stable. No blocks
will be lost and stats will continue to be processed as usual.

Keep in mind that the purpose of all the key server proxies is to be
able to run distributed.net without incident if the keymaster is
turned off or unavailable at any given point of time.

:: 17-Nov-1998 14:30 (Tuesday) ::

Updated public sources are available at http://www.distributed.net/cores
Here’s the list of fixes :
– Win32 now builds ok with MS VC++. You will need TASM or Cygwin to
compile the assembly cores, though.
– The client now prints the core used.
– Updated the configure script for POWER / Power-PC targets.
– Added BeOS support code from the official client. Note that the BeOS
x86 clients has been made with a customized GCC compiler, so you may
not be able to reproduce it. Meanwhile Be switched to EGCS and ELF, so
the various *.xcof files may not be very useful to you.
– MMX s-boxes has been converted to NASM format
– Various other fixes from the official 420+ client, look in the $Log
in each source file.