By the Censorware Project
February 17, 1998
In our report From Ada to Yoyo: Blacklisted by Cyber Patrol, issued in late December, we disclosed that Cyber Patrol blocks the premier Usenet archive and search engine, Deja News. As a result of that report and the attendant publicity, Microsystems Inc., publishers of Cyber Patrol, unblocked many of the sites we listed. However, the company stood its ground and refused to unblock Deja News.
In a January 17 email message to Jamie McCarthy of The Censorware Project, Susan Getgood, the director of marketing for Cyber Patrol, said:
"Deja News provides access to a number of newsgroups that meet the CyberNOT criteria. We will certainly review the categories that we use for Deja News, however, the site will remain on the NOT list."
Deja News is a huge archive of messages posted to Usenet news groups; it may be searched by topic, sender, recipient or keyword. In a recent interview, Deja News CEO Guy Hoffman said the archive stores messages from 50,000 discussion groups:
"We have more than 300 gigabytes of data, representing 250 million messages dating back to March 1995. Over 4 million people access our site, viewing over 90 million page views each month."
A small portion of this text has sexual content Cyber Patrol wants to block (Deja News does not archive or give access to the graphics uploaded to Usenet). In order to ban the relatively small amount of undesirable text, Cyber Patrol blacklists the entire site--effectively all of Usenet.
Thus, Deja News is stigmatized as a "porn" site, off limits to patrons of the Austin Public Library and anyone else who is subject to Cyber Patrol's caprice. However, Deja News is in reality a serious research resource used by many professionals for a variety of purposes having nothing to do with prurient Usenet content.
Mark Pruner of WebCounsel LLC, a White Plains, N.Y. web marketing company, said, "I have been a consulting expert in several suits and regularly use Deja News to see what if anything the other side has said. I also use it in my primary business of website based marketing to check out what kind of on-line presence my clients have, whether said by them or about them."
Russ Smith, who operates Consumer.Net, a consumer information resource, uses Deja News to track "activities of junk e-mailers or Internet scams that people report to me. Often, a search on an IP address or a phrase will lead me to specific information."
Steven Cherry says: "As editor for two professional/practitioner magazines, I used Deja News routinely. Most often it was because someone had been recommended to me as a writer, editor, or source, and I wanted to see what if anything he had written to which newsgroups."
Jonathan Wallace, a Censorware Project member, is co-author of Sex, Laws and Cyberspace (New York: Henry Holt, 1996), a book on Internet censorship. Wallace said, "We used Deja News extensively to see what people in various newsgroups were saying about the topics we covered. In many cases, postings we found through Deja News led us to primary sources whom we quoted in the book."
Abby Beifeld, a programmer for a software company based in the Washington, D.C. area, said that she uses Deja News to obtain answers to Windows programming questions. "Whenever I'm working on a tough project, Deja News becomes invaluable to me. So yes, I use it and I've become pretty dependent on it as well."
J.D. Abolins, who works in a government computing environment, uses Deja News to check information about "computers and networks". He noted another benefit of Deja News: "The state government's Internet connection does not readily provide for Usenet. Usenet postings can be looked up via the http connections we can use."
A. Michael Froomkin, Associate Professor of Law at the University of Miami, uses Deja News to research technical issues. "I frequently use it to get information about computer hardware (and to a lesser degree software). Altavista is my #1 tool, but it takes several weeks for stuff to show up there. If I want full coverage of more up-to-date stuff (latest motherboard, chip, driver release and attendant conflicts), then Deja News is the place to go as some Usenet groups cover these issues quite well."
Another user in search of answers to technical questions is Gabriel Wachob, Chief Operating Officer of Findlaw Inc., a gateway to legal information on the Internet. "I use Deja News all the time for looking for answers to technical questions in the newsgroups.... A good example is comp.mail.sendmail. I only have to deal with sendmail configuration once in a long while, but its nice having a central place to look for answers -- and Deja News is a great way to search through these various 'spaces'. ...As a part-time sysadmin, I deal with a very wide variety of technologies and I like the ability to tap into the mindspace of topic-specific newsgroups without having to actually subscribe."
Some of the people we spoke to commented that they frequently find it easier to get answers to technical questions by using Deja News than by accessing a vendor's support page. One network administrator, who finds Deja News "essential", said, "Got a problem with Microsoft software? Their knowledgebase (if it is working today) has never heard of it? Put the error message into Deja News. Odds are that you'll find bunches of people hitting the same problem, and many times a solution too."
Another Deja News user who operates his own server agreed. "I find Deja News a much better source of tech support info than the Microsoft site. Often I can search on an error message or file that is causing trouble and quickly find the answer. Often (usually) similar searches at the MS site yield no results."
Even Censorware defender David Burt, a librarian who fervently supports the installation of CyberPatrol in public libraries, uses Deja News to research the activities of free speech advocates. In an email message to a librarians' mailing list dated October 28, 1997, Burt wrote, "The ACLU may finally be ready to strike. It looks like Kern County, California is the place where they are going to sue. This post retrieved from Deja News...."
Cyber Patrol is installed in a number of public libraries, including those in Austin, Texas and Boston, Massachusetts. People use libraries to get answers to questions about censorship and regulation, business competitors, software bugs, junk mailers and scams--exactly the types of information the users quoted above were seeking. Library patrons using terminals with Cyber Patrol installed are blocked from obtaining important information via Deja News. Libraries are in the business of providing such knowledge, not denying it to their patrons. Blocking library patrons from using Deja News because some of the newsgroups have sexual content is the equivalent of refusing to carry the Encyclopedia Britannica because some of the articles cover sexual topics.
The point is not that lawyers, P.R. professionals or software developers will necessarily need library computers to access Deja News. It is that these uses of Deja News establish it as a serious resource which deserves to be made available in libraries, even for people who cannot afford to pay for internet access or who don't have access through their job. Only a small percent of the U.S. population has home or work access - but everyone has a public library.
In the "Blacklisted by Cyber Patrol" report, we disclosed the blocking of two huge collections of user pages maintained by ISP's, the West Hollywood pages of Geocities and members.tripod.com. Cyber Patrol blocked large amounts of innocent content (fifty thousand Web pages in West Hollywood, and more than a million at Tripod) due to a few explicit pages maintained by users. In other words, the company couldn't be bothered to filter through these areas of the Web when it was easier to block an entire domain or directory.
After "Blacklisted by Cyber Patrol" was published and Microsystems earned some unfavorable publicity, the company unblocked both areas. But the underlying problem remains: by some estimates, there are two hundred million documents available today on the Internet, and the number is growing exponentially. With these numbers, any company claiming to filter the Net might as well be claiming to filter the Pacific Ocean. Microsystems, like other censorware companies, is secretive about the number and skills of the people it employs to scan the Web. But it is impossible for any reasonably-sized staff to keep up with the Web, whatever their credentials. While Microsystems has effectively acknowledged that it must scan every page of West Hollywood and Tripod before blocking them, it is unwilling to undertake the same forbidding task for Deja News, where the content is even more dynamic and fast-moving.
Cyber Patrol's wholesale blacklist of Deja News illustrates the profound flaws of censorware, and the impossibility of the task it sets out to accomplish. In addition, by denying library users an important research tool, Cyber Patrol teaches us another significant lesson: censorware does not belong in public libraries.
The Censorware Project is a group of Internet activists opposed to blocking software and ratings systems for the Web on the grounds that both approaches promote government censorship of the Net. For more information, please contact Jamie McCarthy at jamie@mccarthy.org.