A Commentary on the History of NetNews/Usenet and my current life

Michael and Rhonda Hauben and others mentioned me as a pioneer netizen in the book about Netizens. At that time I did mention that I was involved in the instigation and creation of Netnews.  It was later reported that Tom Truscott and Jim Ellis (who won an award for the creation of Usenet) dismissed my comments and indicated that they would have done it anyway.

My point is: neither Tom or Jim were the ones that selected which "news" program from the Conference Tape was installed on the Duke system.  I was working on my Dad's PDP-11/20, which was in the same room as Computer Science's PDP-11/45 which was running UNIX, and was reading through the information pages that described the packages that were on the tape.  I was intrigued by the various replacements for the "motd" and primitive "news" that were offered. Reading the three or so descriptions, I picked one that seemed most flexible and most organized. I asked Jim Ellis, one of the CS administrators, to roll that particular news program onto the system while he was loading another package from the tape.  He did so, and I spent a little time at the console meshing the "checknews" program into the login sequence and creating an initial introduction to the news facility and putting a few reasonable directory hierarchies in place.

That was all I did. I didn't have any clue of what would come after that. I was not the one who interacted with the folks at UNC. I was not the one who noticed that they were using the same news program, nor was I the one to notice that there were similar topics in their news hierarchy.  That was the genius of Tom and Jim et alia.  The first implementation of the multi-system news synchronization between Duke and UNC was written as a set of shell scripts that used UUCP and crontabs to do the transfers after midnight.

I do recall one crisis point well. One evening Dr. Bierman, chairman of the Computer Science Department at the time, was anxiously waiting for and monitoring the progress of a numerical simulation program on the Duke UNIX machine.  Along comes midnight, and the news synch script started doing its preparations for communicating with UNC. Dr. Bierman's simulation progress slowed way down and he got mad about it.  One of the admins, I forget exactly who, quickly killed off the offending processes.  Dr. Bierman almost made them take the remote news synch off the Duke machine.  Fortunately, he didn't insist, but recommended that a more efficient method be developed.

Understand a few things about the situation in 1977.  The modem was a 110 bps telephone dial-up link, and calls to Chapel Hill from Durham were "long-distance" because Durham was a GTE territory and Chapel Hill was an ATT territory.  Midnight to 6AM was the lowest cost rate time, so the synchronization was slotted in that period.  Also, the PDP-11/45 was a 16-bit system, and did not have either the speed nor the memory capacity that a modern smartphone has. The shell script based transfer system had to scan the news directory tree (which required a lot of slow disk access) and make a list of the day's topic postings.  That list was then transferred to UNC, and UNC sent a similar list back. it might take an hour for the transfers of the lists to be done.  After the lists were exchanged, the lists were compared to the existing tree (more slow disk access and file name analysis) to figure out what files from UNC would be needed, and UNC did the same thing with the list from Duke.  Finally UUCP requests were generated to get the files from UNC and UNC set up a bunch of UUCP requests to fetch files from Duke. Then things waited on UNC to dial the Duke site, run all it's requests for files from Duke, which, even with compression, could take a few hours. And then Duke got to fetch files from UNC, which would take another couple of hours.  Clearly, some better method was needed to limit the amount of work done at any one time.

Truscott, Ellis, and Steve Bellovin from UNC called for a meeting to discuss the problem. A fair number of Computer Science folk from Duke and UNC went to the pizza party at Mario's on the recently completed Durham-Chapel Hill Boulevard. I attended that meeting, ate pizza and listened more than talked.  Out of this meeting arose a more formalized message format (make it look like an email) and a queuing protocol that scheduled a transfer when a message was posted, and a hierarchical naming system that placed the broadest category first, followed by a more specific category. This system was Netnews, and the nascent network was called Usenet.

Much of the development work was done via a serial connection between the Duke CS (duke) machine and the Department of Physiology machine (phs) that was just a 4-wire connection between two buildings on the Duke campus.  UNC was doing much the same thing, and the dial-up transfers got accelerated by using 1200 and 2400 baud modems. This system was released to the wider UNIX community at another Usenix Conference, and promoted as "The Poor Man's ARPANet"

Then, like Topsy, "it just growed."  Bell Laboratories at Murry Hill, the home of UNIX, joined when Ken Thompson and Dennis Ritchie had their machine (research) call Duke to pickup and send articles. Then research would connect to a site at the University of California at Berkeley (berkeley) and other sites at Bell Labs would call research and some other sites. Since Bell Labs was AT&T at the time, they had "free" calling to anywhere, without Bell Labs, Usenet would have been stillborn.

As the network grew, and traffic increased, it became clear that the original Netnews/Usenet/Bnews protocols needed a re-design. It had not anticipated the number of site that joined, and the huge increase of traffic that created. Another round of design and programming created the Cnews programs. This iteration allowed a practically unlimited depth to the topics hierarchy (although a convention called for being parsimonious with the topic tree) and allowed for loops in the distribution net. Since Berkeley and Bell Labs had connections to the ARPAnet, traffic between ARPAnet and Uesnet became possible.  Topic trees containing posting from ARPAnet mail lists were placed in the "fa" (from ARPA) hierarchy, and a looser hierarchy called "net" was for everything else. For example, the SF-Lovers ARPAnet mail list was gated to "fa.sf-lovers" and general UNIX discussions took place in the "net.comp.os.unix.*" hierarchy.

By this time, my Dad's PDP-11/20 had been moved to another lab, and replaced by a PDP-11/34. The Duke Comp Sci machine had been upgraded to a PDP-11/70, and soon to a VAX-750.I was administrator for the 11/34 (duke34) and connected to the "duke" site via a null-modem serial line stretched across the floor between the two machines. The distribution tree of Usenet was semi-controlled by the Administrators of the "backbone" machines and new sites were encouraged to link via geographically nearby sites. The creation of new groups in the net hierarchy required a password, and at one point I called Bell Labs to get the password, since no one was around at DukeCS at the time.  The Usenet protocol was based on UUCP, which used a routing name convention of "machine!machine!*!user" for email. From this it became convenient to refer to someone by their "bang path" from a well known backbone site. [I was duke!duke34!ggw]

As more sites joined, the "backbone" admins were getting less and less willing to take one new connections, and pushed new sites to the periphery unless they were willing to become a hub for their area. This reluctance led to the myth of the "Backbone Cabal" (tinc) who secretly controlled all of Usenet. The only time the Cabal acted in a directed manner was to nurture and direct the Great Renaming of the Usenet namespace. This moved from fa.* and net.* to the "Big 7" hierarchy of {comp, misc, news, rec, sci, soc, talk}  later the humanities group was added. Around the time of the Great Renaming, a semi-formal convention was developed to limit the creation of new groups. This procedure and formality did not rest well with some users, and the wilderness of the alt.* hierarchy sprang up. Following the GR, the Cabal dissolved, and the Big-8 Management Board was created. (Now that is a real Cabal!) Wikipedia has a decent description of the modern Usenet.

[Fast forward to 1984, while I got married and worked as a consultant in the NY/NJ metro area for a couple of years.] Back at Duke, I was now System Programmer (Administrator) for the Center for Demographic Studies (dukecds) as we moved from a mainframe computing paradigm to a PC and Macintosh based network. I also got a PC at home, and ran UNIX on it, along with a BBS. This home site was "The Wolves Den BBS" and "duke!wolves!ggw" was my non-duke email id. There were lots of hassles along the way, including a possible tie-in to Operation Sun Devil FBI raids, accusations of "selling UNIX" (I considered out loud the possibility of selling dial-in access to the shell command line in addition to the BBS) and sexual innuendos. The Wolves Den BBS actually provided access to Usenet groups by request, and carried a fair amount for technical and social groups. I considered getting into FIDOnet and WWIV, but those connections did not happen on wolves. [Wolves Den had started on a Commodore 64 and later a C=128. Lots of fun with Commodore while it lasted.] At one point I had six computers on a home network.

In 2006, I applied for, and was eventually granted, disability for a number of ailments.  In 2008, I lost my house in the housing bubble collapse and spent 14 months homeless. My wife had a quadruple by-pass heart graft, and eventually had several strokes, was institutionalized and died in 2011. Now I live with a family of choice, and by blood relatives (two older brother) live nearby as well. I have ADSL 11 Mbit internet service and a wired and wireless network of eight computers (2 linux, 4 windows 7, 1 xbox one, and an iMac)  I fix PCs and laptops, do administration for a number of folks who don't have any idea of how their computers really work, and consult with local Karaoke Hosts who sometimes need help with integrating their computers with other devices for display and special effects. And I do volunteer work on several FOSS projects (testing and debugging linux.) The only difference between "work" and my current life is that I can take a nap or break anytime I need.

So, this is my commentary and recollection on the history of Usenet and my life.

Just saw some bees in the 'hood

I was out for a walk in the neighborhood today. While passing the power line greensward I looked at some black raspberry canebreaks to see how this years crop was going. Lo and behold, there were 4 or 5 honeybee-like insects feeding on the blooms.

It has been a few years since I have seen honeybee types in this area. We have the occasional Carpenter bees looking for some real wood to drill nests into, but no smaller bees. To see that many bees was a heartening sight.  I am pretty sure they are honeybees, though I did not attempt to catch one or get too close. As a child I lived with an apple orchard and am familiar with honeybees and hives.

So, despite the fact that bees in this area have been sorely affected by Colony Collapse, there do seem to be some wild bees left.  They pretty much have to be wild since there aren't any nearby locations that would call for commercial hive placement.

All is not lost, nature goes on despite the worst efforts of humankind.