Enhancing the usability of text through computer delivery and formative evaluation: the superbook pr

collections, and recently the dissemination of some materials on CD-ROM. While there has been a great deal of publicity and excitement, especially among promoters of these techniques, there have been serious gaps in their enduser utility, and especially in their actual usability for reading text on-line. They require arcane and inconsistent access routes and user dialogues and almost always employ Boolean query languages that are too difficult for most end users. Consequently, ‘intermediary’, information specialists are typically needed to intercede between the actual user of the information and the database. Not only is this expensive and time-consuming, but it adds a difficult and error-prone translation process to an already difficult and error-prone search process. Although some professionals, such as lawyers, use these systems heavily, they are also often quite dissatisfied with them. An apparently consensus view is that of the US federal judge who recently told one of us, "those computer search systems are sometimes helpful, but you just can’t rely on them to do the whole job. " Empirical studies (Blair and Maron, 1985) have shown that in some circumstances, even with diligent intermediary-based search, recall rates (the proportion of actually relevant documents found) for legal searches may be less than 20%. Unfortunately, we have no comparable numbers for the traditional paper-library-clerk techniques but it is clear that the legal profession is not ready to abandon paper and read the documents off a screen. Another aspect of the problem is that a very large portion of the use of online systems is merely for identification of the document, with the actual reading being done either by local printing of the document or by delivery in paper from a depository. There are probably economies to be realised by this delivery technique, since excess copy production and distribution may be avoided. But it is still obviously expensive compared to the possibilities for fully electronic distribution and use. Another barrier to full electronic document delivery has been the unavailability of large collections of text in suitable forms. Several factors are involved. Firstly, although most publishers have been producing text by electronic means – word processors or automated printing – for many years now, most of them have been discarding the machine-readable form. Even the Association for Computing Machinery, which is a large publisher in its own right, has not retained its electronic files. Secondly, there are literally hundreds of different text tagging and formatting systems and conventions that greatly complicate any attempt to provide a collection of text from more than one source. For the most part, text that has been distributed on CD-ROM has come bundled with proprietary indexing and search software that makes it virtually impossible to integrate smoothly with other sources. Thirdly, until recently, the cost of electronic storage, especially in the more flexible and desirable magnetic media, of local distribution networks, and of high-quality terminal display equipment, has been too high to be attractive for general use with full text. Finally, delivery tools to make electronic text at least as usable as the paper counterpart have only recently been developed. Currently, potential solutions to all of these problems exist for some kinds of document collections and some applications. Thus new experiments with electronic delivery of full text on a much larger scale than mere prototypes, but still much smaller than whole libraries, have become feasible. The CORE project is one such experiment (for additional proposals see Kahn and Cerf, 1988; Information Networking Institute, 1990). Its goal is to deliver a large majority of the journal literature needed by one academic speciality in electronic form to workstations in a library and terminals at the desks of professors and students. The project is a collaboration among five institutions. The Cornell University Albert Mann Library houses and administers the experiment. The American Chemical Society is providing ASCII and microfilm versions of the last 10 years of 20 journals. Chemical Abstracts Service provides electronic versions of their hierarchical indexing scheme tagged to all of these articles. The Online Computer Library Center (OCLC) is contributing expertise in large database storage, access and search techniques. Bellcore is contributing expertise on text and graphic conversion, and transmission as well as developing experimental prototype user interfaces. Bellcore is also responsible for the formative design evaluation studies for the delivery vehicles. It is these studies and their bearing on general problems in the design of electronic text delivery systems that we will discuss here.