Archive Post brief GD history for clarification

From Archive.org:

I wanted to make a few comments about the recent changes to how the Grateful Dead recordings are hosted on the Internet Archive to clarify some misconceptions I have seen posted. These comments reflect only my view as a participant and instigator is implementing the Grateful Dead collection on the Archive. Around 2000, a group of people including myself came together to collect and convert a variety of Grateful Dead recordings into “lossless compressed” format (aka shn) for ftp exchange over the internet. At that time, there were few servers that hosted live music performances. A key event was the creation of the “db.etree.org” database and web site by Tom Anderson. This established a mechanism for tracking different sources using md5 signatures. Tom’s contributions were fundamental. Between 2000 and 2003, our initial effort of digitizing music of the Grateful Dead expanded into collecting the complete “circulating” opus, defined by the shns-in-circ pages for the Grateful Dead at db.etree.org, compatible with the band’s trading policy, onto a single server. We estimated that it would require about 3 Terabytes of disk space and take about 1 year to complete. The server was tol.etree.org and was located in Sweden. Many people contributed their time and resources to this project. Hard disk drives were purchased and loaded with shows and mailed to Sweden. Individual source ftp uploads were coordinated concurrently. We tried to standardize the way the directories were named so that one could find the recordings one desired. As much as possible we tried to include Grateful Dead internet trading groups by having public logins to tol.etree.org. These groups included “Dead Net Central”, BUDD, deadlists, Grateful Dead forums on “The Well”, etree.org and others. Efforts to collect rare and famous audience recordings and transfer them to the digital domain also occurred during this time interval. The recordings on the Internet Archive were not supplied by the Grateful Dead to the Archive. The soundboard and audience recordings on the Archive are those that were being traded, in effect, by the aggregate of collectors that we are aware of. Each source has its own story, some colorful, some pure fiction (but always entertaining). The impact of the music is reflected in the fact that the various sources have been kept by individual people for so many years that present day audio technology can make it widely available. Around fall 2003, I contacted Brewster Kahle, the vision behind the Internet Archive, if the Internet Archive would be interested in backing up the contents on tol.etree.org. Enough time and money had been expended by so many people that we thought it best to try to implement a contingency plan in the event tol experienced a catastrophic hardware failure or was no longer available. The Internet Archive’s vision of “building a digital library of Internet sites and other cultural artifacts in digital form … [to] provide free access to researchers, historians, scholars, and the general public” was an ideal match considering the pivotal role of the Grateful Dead in live concert recordings. Brewster and I met and he agreed to archive tol’s gd collection and to host it on the Internet Archive as an “etree server”, consistent with our understanding of the band’s trading policies regarding live recordings. I was personally interested in finding a permanent steward for the audience recordings knowing that by and large the soundboards are professionally cared for in the Grateful Dead’s Vault. Around spring 2004 public access to the Archive’s gd collection began as we rolled out the collection in years. The infrastructure of the Internet Archive enabled the recordings to be freely accessed in a variety of formats (shn/flac, streaming, mp3/ogg). Thanks should go to Jon Aizen and Parker Thompson of the Internet Archive (and others I am not aware of) who coded and help fix all of the nagging issues to get the sources on line with correct metadata. From 2004 to the present one can read threads in different gd forums on this site and others that wondered if having everything in one place for anyone to access was the right thing to do. I haven’t collected every opinion that was articulated but one can safely assume that any conceivable opinion has been voiced by someone, somewhere. What those involved in the effort (and there have been several hundred) have strived to do was implement our idea of a single online collection with an attention to quality and completeness and at all times consistent with our understanding of the band’s trading policy. Fans of the Grateful Dead are known for their obsessive compulsive enthusiasm to sharing the music of the Grateful Dead and we are guilty of this charge. What you see on the archive is “Phase 1” of the GDIAP (Grateful Dead Internet Archive Project). Given the uniqueness of the Grateful Dead’s role in the history of live music performances, we had planned to develop software that would enable researchers to explore the audio collection in new ways to follow how music evolved. We wanted to create supplementary databases (though the help of the public) that would transcribe the between song banter so it could be searched with links to the audio source material, who played on every song, song timings, links to themed jam development and others. We contemplated creating software that would use set list statistics based on frequency and song order to create hypothetical setlists and then “stream” them to you as an amusing diversion to “shows that could have been”. Maybe you want to hear every “Yellow Dog” story back to back? Developing the software to enable you to do that transparently was on our “to do “list. We don’t claim to know all future applications of the GD audio collection on the Archive. Given the inventiveness of the GD fan base throughout the last 40 years, clever, useful or fun concepts are bound to arise we can’t envision at this time. Several months ago, Brewster, Parker and I met with representatives of Grateful Dead Productions (GDP) and presented our visions of where the GDIAP project was going. These meetings were very productive. I felt very much that GDP understood the materials we had presented to them. The Grateful Dead’s music has an infectious enthusiasm that we all share including the hardworking and dedicated people at GDP. I also learned something about their roles and functions, some of the problems they face and the constraints they operate under. The position of the Internet Archive has always been to support the bands in hosting their music on the Archive. I think it is very important for me to personally say that there has always been an unspoken generosity about sharing the music from the band members, the support personnel and the fans. These were the only meetings that I ever had with the any one officially connected with the band and I felt that spirit was there. The people who have contributed to the GDIAP project want the Internet Archive to exist “forever” so things like the audio recordings of the Grateful Dead are around long after we have gone our separate ways. Matt Vernon