:: 27-Sep-1999 15:51 (Monday) ::

Ok… I’ve been pretty busy lately, so I haven’t had much of a chance
to work on stats. Right now I am helping GregH out with some OGR
testing. So far, OGR is looking pretty nice. Here is a teaser from
my test full proxy:
ogr r=4998/5000, d=0/0, 0.0 Mkeys/sec, tot=89034

Yes we know that the rate screwed up.

I don’t know when the release is actually planned though, so don’t ask.

:: 24-Sep-1999 14:52 (Friday) ::

There is something quite unhealthy on statsbox at the moment which is
causing daily updates to take an extremely long time to complete. Last
night we disabled the phistory queries to see if that would alleviate the
problem, and it didn’t seem to help. I’m hoping to have time tomorrow to
really take a close look at the issue. In the meantime, we’re going to
temporarily start restricting access to stats during portions of the
update.

I’ll post more as I know what’s going on. It might turn out to be
something simple… no way to know yet.

:: 24-Sep-1999 01:40 (Friday) ::

Well, there’s good news and there’s bad news. The good news is that
I think we found out what was causing the major slowdown on
stats-box (yesterday’s stats-run took about 20 hours to complete,
obviously not a good thing }:8( ).

Working off of a NuggetHunch, I’ve disabled the participant history
functions. It’s too soon to tell for sure, but this seems to have
brought the currently running update back up to normal speed. Of
course, this means that you can’t check your history right now.

The good news is that if this is the problem, we can probably
cure it by adding memory to the box (256 meg ECC DIMMS, for those
of you in a generous mood };8P ). The current theory is that the
history query is killing our caching, since it has to sift through
all ~20 million records in the main rc5 table.

We should be able to tell in a few hours if this actually solved
the problem or not. Thanks for your patience!

:: 20-Sep-1999 05:57 (Monday) ::

Finally, some OGR news! An OGR test master is now operational. For the
latest news, see:

http://www.distributed.net/~gregh/ogr-todo.html

:: 19-Sep-1999 13:23 (Sunday) ::

As some of you have noticed, the classic “win32gui” clients have
recently been removed from the clients.html and select.cgi link pages.
The separate “GUI” client was originally deemed necessary because the
only alternative was a console-based version that had a number of
significant limitations:

– required significant user involvement for installation.

– no user interface for client configuration.

– could not be used in environments needing an unobtrusive, hidden
mode of operation that did not interrupt normal logouts

– inability to minimize to the system tray

– no extras, such as “moo sounds” or “log graphing” or pretty
percentage bars.

HOW IT WAS
———-
We have maintained separate releases of both the GUI and CLI Win32
clients for quite awhile now, with our client development focused
around the common CLI-based source code, and the GUI client involving
a fair amount of clever wrappers around it. Although this made
development of the separate versions easier than they would have been
otherwise, there was always a significant delay in the release of the
Win32GUI relative to the equivalent Win32CLI. Additionally, the GUI
client has always suffered from numerous obscure operational bugs
ranging from benign to severe issues.

Meanwhile, each newer release of the common code-base tried to make
incremental code infrastructure changes. This new infrastructure made
construction of other “GUI wrappers” with less effort than what was
necessary to produce the level of integration that the Win32GUI had.
However, because of these major adjustments in the organization of the
common code-base, the Win32GUI began to lag even further behind and
grew kludgier since much of the back-bending efforts that were
previously necessary no longer were. The Win32GUI portion of the code
base really just needed a complete rewrite.

Additionally, the Win32CLI releases had slowly been getting enhanced
with each successive release. The Win32CLI was no longer a console
app at all, and had actually become a GUI-based application itself. A
reasonably friendly textual configuration menu-set was now available
for all clients derived from the common code-base. The Win32CLI was
also able to cleanly start and stop in hidden environments, run as an
NT service, or minimize itself to the system tray. More importantly,
since the new “GUI-ish CLI” was directly compiled from the common
code-base, and the Win32CLI was traditionally one of our primary
development environments, the release delays and obscure GUI-specific
bugs were minimized.

WHAT IS HAPPENING NOW
———————
A new “self-installer” version of the Win32CLI will now be available
for download to make installations simplified. This installer is
nearly identical to the installer that the Win32GUI came packaged in.
Significant effort has been put into making “upgrading” from a prior
Win32GUI installation easy.

The Win32CLI will also now ship with a WinHelp HLP file to make
introductions easier for first time users. Additionally, the
distributed.net Log Visualizer is a new utility that is based heavily
upon code derived from the log grapher portion of the Win32GUI. Full
source code for the Log Visualizer will be available for download on
our source code page.

Although the last released version of the Win32GUI will continue to
operate for the foreseeable future, there will be no newer versions of it
released, and support questions regarding it will generally recommend
switching to the CLI. The Win32GUI will continue to be available on
our FTP distribution sites, although it will not be in the
“current-clients” directory.

We will continue to list and support GUI clients for the other
operating systems that we currently provide separate GUI clients for
(MacOS and RiscOS), however they too will similarly become unnecessary
once our plans (described below) are fully completed.

THE FUTURE
———-
It had always been part of our plans to try to slowly move away from
producing a separate GUI and CLI versions. This included the Win32GUI
and Win32CLI, the MacOS GUI and FBA clients, OS/2 GUI and CLI clients,
RiscOS GUI and CLI, and possible X11/Gnome/etc-based clients.
Instead, we planned to create “detached” front-end GUI applications
that could be run alongside the traditional CLI clients. This would
prevent the complexity of various OS-specific GUI programming
requirements from affect the design of the common client source pool.
Additionally, this detached mechanism might allow for remote
configuration and monitoring to occur over a local network, which has
always been a popularly requested feature for our clients.

So with all of this said, we will now be diverting our efforts from
attempting to maintain separate GUI-specific versions of our clients
and toward achieving our separated GUI frontend and CLI back-end
model. We are beginning development of an extensible communications
interface that will allow multi-platform GUI configuration utilities
to be constructed and arbitrarily configure and monitor remote or
local clients. The projected availablilty of these new clients cannot
be easily estimated at this time, though we will definitely make
future announcements when further information is known.

:: 17-Sep-1999 21:19 (Friday) ::

Since Silby’s temporary departure to concentrate on school,
I am helping out with help@distributed.net support. I had
been helping Silby over the past 6 months or so in this role,
but now that role has gotten larger. I am also helping out
with documentation.

—–BEGIN PGP SIGNATURE—–
Version: PGPfreeware 6.5.1 for non-commercial use

iQA/AwUBN+KuhtllZllpF5qpEQJPRQCdFsDKQytKaFXheokLSkotJd5Km/oAoLhm
2yr/i7RVW1A4TNfDTqpHC/Sg
=p5pb
—–END PGP SIGNATURE—–

:: 17-Sep-1999 05:29 (Friday) ::

They say hindsight is 20:20, but they lie. I have no clue what just
happened. The part of stats where the day’s activity is moved from the
“daytable” work tables into the big rc5_64_master table (all 20 million
rows of glory) simply failed to happen.

It silently didn’t append the 16-Sep data to the big table.

Then, of course, it went on to re-rank everything with 15-Sep data.
Needless to say, it didn’t take long for everyone to catch on that
something had gone wrong.

I pulled apache offline and tried the append query manually. worked fine.
I scratched my head, and started all the rankings queries again. They’re
nearing completion right now.

Everything is looking just fine, but there’s still the small matter of not
knowing what went wrong in the first place. In true programmer fashion,
I’m not truly concerned if we can’t repeat the error, so we’ll see what
happens tomorrow. Worst case, we’ll just blame the OS. :)

:: 16-Sep-1999 22:26 (Thursday) ::

See below for more details. I’ve got the “pretty” version of phistory
all polished up and ready for display.

http://stats.distributed.net/rc5-64/phistory.php3?id=1

There’s also a convienent link from everyone’s psummary pages, so the
participants who don’t read the .plans will find it, too.

As daa said, the graphing is mostly functional, although it’s still
slaving from nodezero which is a pessimal solution. I’m also going
to try to come up with a solution that doesn’t involve the mess of
gnuplot that we were using. I should be able to do the graphs in
php which is a much more efficient solution.

then, of course, team histories…

:: 16-Sep-1999 21:21 (Thursday) ::

Well , now that I got Nugget to get phistory_raw.php3 to work.

http://www.distributed.net/statistics/em_plot.cgi?id=1

is back , but will be moving to stats.distributed.net

:: 16-Sep-1999 20:43 (Thursday) ::

Just a quick update, but a pretty momentous one. The first (and most
crucial) step to getting history graphs back online has been passed.

As I’ve explained earlier, the implementation of retiring emails had
significantly increased the complexity (and cost) of running history
queries. This is why we haven’t really rushed to get those queries
re-implemented.

It finally dawned on me today how to accomplish the historical query
at a reasonable speed and still incorporate retired emails into the
equation. This also marks the return of the fabled _raw queries from
the old statsbox.

http://stats.distributed.net/rc5-64/phistory_raw.php3?id=1

As you can see, it’s really not much to look at yet, but your scripts
should love it. Next step is to make a pretty, human-readable version
and then we do graphs. The hard part is behind us at this point.

Time to throw on the headphones, crank up some crunchy guitar and
start coding furiously. :)