159

99.1

Conclusions andrecommendations
Summary of results of the project

The team set out to produce an online catalogue which would:
allow access to the bibliographic files of a small to medium libraryand operate on a low-priced LAN, andbe readily usable without training or experience, without sacrificing
effectiveness or being tedious for experienced users.
The first goal has been achieved. The prototype OPAC, Okapi, could
serve as the principal means of access for small to medium collections(probably up to about half a million bibliographic records, and more withcomparatively minor software enhancements).
It is difficult to make an objective assessment on the second goal (seeSection 9.4.6 for a discussion of some of the problems of the evaluation of
OPACs) but the evaluation results given in Chapter 8 suggest that it has
been fairly successful: it is undoubtedly easy to use, and appears to berelatively effective.
It must be emphasised that the prototype Okapi is not intended to be afinished system. There are a number of ways in which it is simply
unfinished: there are, for instance, some access points which have notbeen made available to users, and the subject search function could
readily be improved. The search programs contain mistakes andinconsistencies. There is no provision for the input or editing of records.
Apart from this, all OPACs covering loan collections should be linked tocirculation control. The importance of Okapi lies in the fact that it has
demonstrated that an OPAC can be both easy to use and effective, andthat it is in many respects a good test system for further research anddevelopment.

9. Conclusions and recommendations

160
9.2

OPACs — advantages and disadvantages

The paragraphs below, labelled (A)-(E) and (l)-(3), list some of theadvantages and disadvantages of OPACs. The cross-references are tosections later in this chapter in which there is further discussion of someof the ways by which OPACs might be improved, together with somefairly specific recommendations for further research and development.
Potentially, OPACs provide the opportunity for libraries and their users
to achieve the following objectives:(A)

Powerful and flexible subject access (Section 9.4.1).

(B)

Specific item access which is as good as that in conventional
catalogues for precisely specified items, and better for inaccurately
specified ones (Sections 9.4.2 and 9.4.3).

(C)

Access to contents, summaries, or even full text ( Sections 9.4.1 and9.4.5).

(D)

Location and availability information.

(E)

Access to other collections.

As yet, none of these objectives, with the exception of (D), has beenachieved in a live, general purpose OP AC. Further, OPACs (or rather
their users) suffer the following disadvantages in comparison with mostconventional catalogues:(1)

They are often more difficult to use and to learn than conventional
catalogues and less tolerant of mistakes and misapprehensions. To
use even the most elementary facilities (well-specified known item
search) requires a higher degree of involvement (Sections 9.4.2 and9.4.4).

(2)

It seems that many users do not regard OPACs as being catalogues
in the traditional sense (Chapter 8); presumably it is the interactive
nature of OPACs which gives rise to this attitude. Thus anineffective OPAC may raise users' expectations initially, followedby disillusionment. More seriously, users are too ready to besatisfied with an incorrect or inadequate OPAC response to their
queries — the OPAC seems to give an impression of infallibility
which traditional catalogues do not have (Sections 9.4.4 and 9.4.6).

9. Conclusions and recommendations

161

(3)

The poor quality of the display and the small amount of information
which can be shown at one time leads to browsing facilities which
are in some respects inferior to those provided by microform
catalogues (Section 9.4.3).

9.3

LANs and library automation

Library automation functions differ widely in their importance forlibrary users and for library staff and management, and in the demands
they make on software and hardware resources. For users, access to thecatalogue is undoubtedly the most important, closely followed bycirculation control. In lending libraries the catalogue access andcirculation functions must be combined. Clearly, catalogue access needs
cataloguing facilities, so the provision of these is also a primary function.
Functions of secondary importance include acquisitions, serials control,
management information and other office automation procedures.
Circulation control makes the heaviest demands on the hardware because
it requires instant updating of copy and user files, a guaranteed quick
response time and high reliability. The public access catalogue needs far
more terminals and good response time, and is unquestionably the mostdifficult to design and to program, but files need only periodic, batch,
updating. Catalogum^ is not a heavy load on resources unless it is requiredthat new records be instantly accessible to users of the public accesscatalogue.
This project has demonstrated that a LAN is an eminently suitable hostfor the catalogue of a smallish library, and that other activities can takeplace simultaneously with the use of the catalogue. The hardware andsystem software used would be suitable for the full automation
requirements of most libraries of up to a million or so bibliographic
records. For installations towards the top of this range it would benecessary to have more than one file server and disc drive. This gives thefurther advantage that the system can still work if one disc drive fails(Section 3.5, footnote); this is particularly important for circulation
control. Large libraries considering the use of a LAN should study theGeac installation at the Bobst Library of New York University [1],A set of LANs, each with one or more file servers, possibly linked by fast
telephone lines, would work well as the host for an OPAC for a multi-site
library, but circulation control, which needs real-time updating on all162

9. Conclusions and recommendations

sites simultaneously, might not be so satisfactory as on a lightly loaded
multi-user mini installation. For some multi-site libraries the OPAC
might most practically be updated across sites by reloading the catalogue
files at each site using a cassette or other tape (at 1000 characters per
second on a fast telephone line, an indexed file of a quarter of a million
catalogue records would take one or two days to transfer).
With present technology and current library software, the difference
between the use of a LAN and a multi-user single processor system forlibrary automation is minimal and ill-defined: it is difficult to think ofthings which can be done with one and not with the other. There may be acost difference. In the longer term it will be important for the design ofhighly responsive interactive systems (Section 9.4.4) that each user
station should have its own processor(s) and a large amount of local
memory (core certainly, backup possibly), and stations have to be linked(e.g. for circulation). This does call for a LAN or other multiprocessor
system, and fast, accurate data transmission.
9.4

Some conclusions and recommendations for research anddevelopment

9.4.1

On the enhancement of subject searching in OPACs

This is too large a topic to be covered exhaustively here; interested
readers are recommended also to read the recommendations of Markey[2], of Kaske and Sanders in [3] and of Walker in [4].
There are really two approaches to this problem. The first is to index
everything in the (preferably enhanced) MARC record which could
possibly have subject content, then to make this indexing more fuzzy by,for example, the application of word stemming rules, and then to do acompletely post-coordinate search on what the user enters. This might becalled the sledgehammer approach. It is highly likely to result in the
retrieval of something relevant, when there is anything, but at the cost ofa large proportion of "false drops", particularly in a large file.
The second approach is to make the best possible use of the controlled
subject information in the records by helping searchers to find and to
make use of assigned subject headings and/or classification codes. When
this is the only method of subject access in an OPAC, as it is in some of theearlier systems, many searches fail because the user cannot find an initialentry point to the subject indexing. In any case, facilities for browsing9. Conclusions and recommendations

163

subject headings or class marks must be provided. The burden is placedon the user rather than on the system. In a notable study of the use of an
LCSH (card) catalogue in an academic library, Bates [5] found a
strikingly low success rate; catalogue familiarity had a significant
beneficial effect but, alarmingly, subject knowledge was of no help, being
if anything slightly detrimental to users' success in finding an entry.
It is fairly obvious that both "free" and "controlled" approaches must be
utilised for any collection other than the smallest (Section 2.5).
METHODS OF IMPLEMENTING ENHANCED SUBJECT ACCESS

Hildreth [6] says "users' difficulties with online subject searching areprimarily of two kinds: (1) matching the terms which they use to express
their topic to the terms indexed in the online catalog; and (2) finding
terms related to their search terms, including broader terms andnarrower terms". While we do not disagree that these are the problems,
observation and logging of catalogue use show that it is not realistic to
expect the user to play the active part in these tasks. Many users haveneither the persistence nor the skills required. It follows that the OPAC
itself must play a larger part, and make the minimum demands upon theuser.
Hildreth's first point is best satisfied by ensuring that there are as many
entries to the catalogue as possible, that is by providing the sledgehammer
approach.
There has been a considerable amount of research into providing
additional access points beyond what is derivable from the content of
MARC records. The "BOOKS" project of Atherton and others [7]investigated the utility of adding terms from contents pages and back-ofthe-book indexes, with encouraging results. Although the average extra
cataloguing time per item was only about ten minutes, it seems unlikelythat the larger cooperatives, or even many individual libraries, will beable or prepared to make even this rather small investment.
Currently, Markey and others are doing an elaborate study of theadditional entry points (and browsing facilities) which can be provided
by making available online extracts from the Dewey schedules andrelative index, linked to the MARC records by means of the class number[8,9 — also several interim reports available from OCLC]; this canundoubtedly provide more subject entry terms, and, since the Dewey

164

9. Conclusions and recommendations

files are available in machine-readable form, it does not require any
additional cataloguing effort. However, it does appear that the languageof the Dewey schedules, being designed to help cataloguers rather thanusers, is rather far from the language of many subject search queries.
On Hildreth's second point, the problem is one of finding related records
rather than related terms. The Okapi logs do indeed quite often show
revision of queries following an unsatisfactory search (a typical example
is ''London Broadcasting Company" followed by the broader query"independent radio"), but it is not possible for most OP AC searchers
to learn the kinds of skills which are used by experienced intermediaries
in conventional online searching.
RELEVANCE FEEDBACK AND RANKED OUTPUT

Much research in IR has been directed towards investigating ways of
automatically finding records which have a high probability of being
similar in subject content to those which match the terminology of thequery. Such methods include clustering techniques (terms are divided into
groups on the basis of their probability of co-occurrence, and the system
will retrieve records indexed by as many as possible of the terms which
regularly occur in conjunction with index terms matching the query),and the use of relevance feedback: the query is modified by, for example,
adding terms from records judged by the user to be relevant to the query,and reprocessing the search. Harper's thesis [10] contains a rather
comprehensive account of relevance feedback techniques. Combinationsof clustering and relevance feedback techniques can be used. Most ofthem are difficult to reconcile with Boolean searching, and some,particularly clustering techniques, can impose very heavy computationalloads. One feature which most of them have in common is that theresulting display of records will not be in the order in which they occur inthe source file, but in order of decreasing similarity to the query — themeaning of "similarity" being dependent on the method used; it isdesirable that this has some correlation with probability of relevance.
For subject search queries containing three or more words, Okapi
displays records in an order which reflects their degree of similarity to thequery (Section 6.6). It does not provide the user with any direct help in
modifying the query. CITE (Section 2.4.4) does use relevance feedback,but uses it in a way which requires the user to play a role which is much
more active than merely answering "yes" or " n o " to questions about therelevance of displayed records. There does not appear to be any other9. Conclusions and recommendations

165

OP AC which makes any significant use of unconventional IR techniques.
RELEVANCE FEEDBACK AND CLASSIFICATIONThe central problem is that of finding not only those records which areindexed by some or all of the terminology of the query, but also other,related, records. Relevance assessments are easy to obtain — they give theuser a sense of useful and positive involvement and need only require a"yes" or " n o " response — and they must often provide useful additionalterminology which can be used to revise the search formulation. In manycatalogues there are no assigned subject headings, but almost all containclassification numbers, and it is almost self-evident that, among records
classified similarly to known relevant records, there will often be somewhich are also relevant. It is equally self-evident that the use ofclassification alone will often decrease the precision of a search. Thiseffect may be minimised by using a combination of classification together
with free and/or controlled terms from relevant records. It may also be
possible to exclude records on the grounds that they contain terminology
in common with record(s) judged non-relevant.
Some real subject searches from the Okapi logs have been repeated underexperimental conditions, and in a considerable proportion of thesesearches additional relevant records were indeed obtained simply bydisplaying records classified in the same area as a record which has been
judged to be relevant, and this can be done with a minimum of user
expertise. It is more or less equivalent to the process of bookshelfbrowsing, with the advantage that all the books are there, and one canbrowse many shelves without moving from the terminal. There is little
doubt that recall certainly, and possibly precision, could be further
increased by adding terms such as title words from relevant records to thequery, and perhaps also some account could be taken of the terminology
and classification of records judged wow-relevant. A course of researchalong these lines has been proposed in some detail in [11].
9.4.2

On data input by the user: spelling and keyboarding

A large proportion of users' time at the OPAC terminal is spent keying insearch statements; many of these are amended by the user before thesearch is started, but many still contain spelling and keying mistakes/])(1) The Okapi transaction logs suggest a mean effective keying speed of well under one
keystroke per second. In a sample of 220 consecutive subject search statements 24
contained one or more spelling mistakes. The mean length of the subject searchstatements was about 20 characters.

166

9. Conclusions and recommendations

This reduces the cost effectiveness of OPACs, and their attractiveness tousers, and is one of the reasons why one probably needs more OPAC
terminals than microfiche readers to service a given population.
It is ironic that we are entering the age when computers are becoming
everyday tools at the same time as the concept of correct spelling is
evaporating and we seem to be reverting to the condition which obtained
before the eighteenth century. It is very easy for the human eye and brain
to cope with orthographical variation, but quite beyond the current stateof knowledge to write computer systems which have the same tolerance.
For OPAC use, misspelling is of far lower importance with catalogues inwhich items are found only, or mainly, by browsing (that is, in the older,pre-coordinate, OPACs), than it is with keyword, post-coordinate
catalogues. This is certainly a feature in favour of the pre-coordinate
OPACs. On the other hand, browsing is time consuming and tiring, so
there is no doubt that any OPAC ought to lead the user straight to therelevant item(s) whenever possible.
Much research has been done over the last twenty-five years on computer
programs for the automatic or semi-automatic correction of misspellings,
and, although there are no methods which give results comparable tothose obtained by an expert human, there are a number of procedures
which are good enough to be worth incorporating in an OPAC. The more"intelligent" methods, such as that of Yannakoudakis and Fawthrop[12,13] need large amounts of memory and processing power, but
methods which rely on storing in an index more or less "fuzzy"representations of words and names are quite feasible with the resources
which are currently available to libraries. An example is the n-gramsimilarity technique given by Freund and Willett [14] and used by Porter[15] in a computerised museum catalogue. The user interacts with suchprocedures by choosing from a list of similar terms.
At the most rudimentary level, it should be noted that if a word entereddoes not appear in a large catalogue, then there is quite a high probability
that the word is misspelt, and the system can give a clear but neutral
response like:"econmics": nothing foundwhich suggests to the user that s/he may have made a mistake (without
making the system look idiotic when the word in question is correctly
spelt).

9. Conclusions and recommendations

167

One of the proposals for further development of Okapi suggests theinvestigation of the effectiveness of a degree of fuzzy matching of
personal names and of words in subject search queries [16].
On the use of keyboards, it is probable that not much can be done except
by making OPACs which enable and encourage users not to type morethan is necessary to retrieve what they are looking for (this applies
particularly to specific item searches). It seems fairly clear that noalternative keyboard would give a significant improvement over the
standard Sholes (QWERTY) layout [17].
On alternative methods of data input, the drawbacks of touch screens
have been mentioned in Section 7.3.2. Input devices like mice (mouses}),lightpens and joysticks cannot do anything, in the catalogue context,
which cannot be done with touch screens. In ten or twenty years we may
have voice input systems adequate for the enormous vocabulary andrange of users which a catalogue must handle. Handwriting recognition is
not so remote, but does not avoid spelling problems.
9.4.3

On OPAC output

OPAC searches which involve browsing records or index terms would be
both quicker and more pleasant if the clarity of the display were improved
and the amount of data on the screen increased. Conventional VDTs havevery poor readability compared to printed text: to obtain comparable
readability it is necessary to use 60%-85% white space as opposed to20%-60% for printed material (Section 7.4.3). The poor quality is caused
by low character definition, limited character set, little or no choice offont, character size and spacing and the paucity of ways of emphasisingand distinguishing between elements of the display (usually limited totwo degrees of brightness and sometimes an underlining facility or theuse of inverse video). Microfiche material suffers some, but not all, of
these drawbacks.
Display devices of much higher quality are now obtainable, and are
becoming less expensive. The greatest improvement in scanning speed
and comfort will probably be obtained by providing much larger,
vertically oriented, black-on-white screens. If the definition is highenough, an A4 sized screen can carry about 70 lines of 80 characters, andcould be used to display up to about 30 brief bibliographic records or two
to four fairly full ones. (Most existing OPACs show six to ten brief
records per screen, or at most one full record.)

9. Conclusions and recommendations

168

A typical browsing display (unless it is an index display) needs one discaccess for each record, so an OPAC equipped with large, high-definition
output devices imposes additional disc access, data transmission andmemory loads, and so designers must take account of these heavier
storage and transmission demands.
OPACs of the near future must also be able to display at least the full
MARC character set, as do some cataloguing systems. It may thenbecome inadvisable to use one of the standard character codes (e.g. ASCII
or EBCDIC) for the internal representation of characters, but rather tore-code so that, for example, e-acute comes between e and f in thecollation sequence.
9.4.4

On user interaction in general

Hildreth has urged that an OPAC should be ''adaptive, collaborative andSocratic" in its interaction with users [18]. It must be readily usable bypeople with varying degrees of experience, proficiency and aptitude,
without only catering for the totally unskilled. It should guide users withany amount of proficiency in obtaining good results and in learning to
make better use of the system. No existing OPAC approaches this idealinteraction, partly because we do not know how to implement it, andpartly because of the demands on computing resources and human design
and programming resources which it would make.
Of existing OPACs, almost all are fairly readily usable by fairly unskilled
people. Some achieve this by offering two levels of interaction, others
have only one, which is aimed at inexperienced users. At the lower (oronly) level, there are few OPACs which offer sophisticated IR features.
(As usual, CITE (Section 2.4.4) is an exception, but is not suitable forvery unproficient users.) There has been no serious attempt to make an
OPAC which adapts, actively or passively, to different types of user or tothe course of an indivdual search.
Such an OPAC would be an example of the type of "highly interactive"or "adaptive" system discussed by, among others, Hayes in [19] andInnocent in [20].
HIGHLY INTERACTIVE OPACS

Marcus and Reintjes [21], cited by Hildreth in [22], list the following
dimensions along which interaction can vary:9. Conclusions and recommendations
1.

169

Verbosity I terseness in the number and comprehensiveness of messages.

2. Instructional I service: system messages and interaction in general canbe set so as to be more or less didactic.
3. Interpreted/strict: in the OPAC context, interpreted mode would
extend a search by using semantically or morphologically related
terms (Section 9.4.1), whereas strict would keep precisely to theformat and terminology of a search statement.
4. Automatic Iassisted: the degree of user involvement, the extent towhich the system requests user guidance.
5. Hidden/expository: the extent to which the system reveals its
workings.
There could be continuous variation along each of these dimensions, andit need not be constant throughout a session or even a search.
The verbosity I terseness scale is probably the one which needs the mosturgent attention. One of the major problems with interactive systems isthat of drawing the user's attention to vital information at the time it isneeded. Experience has shown that casual OPAC users (and manycatalogue uses are and should be casual) take a minimum of notice of
messages on the screen, often reading a message only after several failed
searches.
Further, after gaining a little experience with a system, users commonly
assume that they know all they need to know, and continue to makesearches which are less efficient than they should be. Hence it isimportant that messages and prompts should be limited to what isnecessary for a particular user at a particular point in a particularsearch/ l)
It is therefore necessary to reduce messages to a minimum (one message,
two to four alternative actions depending on how concisely they can be(1) There is a connection here with the poor readability of conventional VDT displays,
mentioned in Section 9.4.3. Without blinking or flashing, which is rather drastic, ormaking a noise, which is unpleasant, there is little which can be done to attract theattention of a user. However, improving the readability and presentation of displays
would undoubtedly alleviate the problem somewhat.

9. Conclusions and recommendations

170

explained) — this should be obvious from analogy with (spoken)
dialogues between humans in similar types of situation (helping,
explaining, advising, searching). The whole process must be highlyinteractive: a brisk exchange of brief messages. The system must knowwhat to do, and when it does not know it must seek elucidation from theuser.
EXAMPLES"There are a lot of Smiths — can you give an initial?""6 under realism, 15 under naturalism, nothing under both:please choose the one you want""There's nothing very similar. Can you think of something
less specific?"(After a longish pause) "Press the HELP key for advice on how toenter your search"(User slowly typing long title) "There's usually no need to enter
the whole title"
To achieve this, the system must be able to infer with high reliability
what the user currently needs to know, what choices of action are sensibleat this point. This would seem to suggest some sort of artificial intelligence
application, but the term seems to have become both too general and too
specific. All it really needs is the application of artificial common sense.
Knowledge of how to make such highly interactive systems is derived
primarily from a study of what people do when using OPACs (Section
9.4.6), with particular attention being paid to searches which failed or
only partially succeeded.
Some preliminary work towards the evolution of a highly interactive
OP AC has been proposed in [23].
9.4.5

On bibliographicfilesfor OPACs

OPACs vary widely in the structure and complexity of their bibliographic
files. Some use a full MARC record (doubtless with the tags embedded
and the directories removed), some, like many of the OPACs derived
from circulation systems, use an overly abbreviated record; others, like
Okapi, retain most of the information from the MARC record but discard
a good deal of its structure.

9. Conclusions and recommendations

171

IMPLICATIONS FOR REMOTE ACCESS

There are two ways in which users of a library could obtain remote accessto the collections of other libraries: one method is to access the remote
OP AC, using local terminals; the other is to use the local OPAC to accessthe remote files.
The first method can be used now — there are a number of OPACs bothin the US and the UK which can be accessed through public networks.
Access to a remote OPAC entails either that the OPAC be instantly usableby those unfamiliar with it, or that there should be an interaction mode
which is standardised. There have been suggestions in the US that there
should be some attempt at drawing up a standard command language for
OPACs. However, one of the main burdens of this report is that the very
concept of a command language is not appropriate for public access(Section 7.4.1), and the approach should rather be to make the OPAC
transparent (in the sense of making the interaction self-evident).
The alternative is to standardise OPAC files and their indexes within theparticipating libraries. This condition already holds for the files oflibraries whose OPACs use a full MARC structure, but it seems too muchto hope that a satisfactory standard for indexing would be achieved. A
compromise which seems just conceivable is that groups of libraries
might be able to agree on a simpler standard, derivable from MARC, for
OPAC files and indexes. The Centre for Catalogue Research report [24]showed convincingly that almost all catalogue uses would be satisfied by arecord very much simpler than the full MARC record (indeed, by arecord which is even simpler than the Okapi record).
A further point which has to be considered in this connection is that many
libraries would wish to be able to output records in exchange format for
use by other libraries. The Okapi source file is an example: although it canbe used to generate catalogue records adequate for the requirements ofmost library users, it would be impossible to produce from it anything
resembling a subset of a MARC record.
ENHANCING THE CONTENT OF BIBLIOGRAPHIC RECORDS

Certainly, few libraries need full MARC records for any purpose exceptthat of exchange, but it may be that there are areas in which the structureof standard bibliographic records could even be elaborated.

172

9. Conclusions and recommendations

The CLR survey [25] found that many OP AC users would like periodical
articles to be accessible. MARC does not specify a method for cataloguingthese. It is possible to catalogue journal issues as records with analyticalentries, within the MARC structure, and this has been done by somelibraries in the UK. Although the UK MARC Manual says that
"analytical entries are made for up to three works contained within an
item" [26, page 5/50], there is no reason for abiding by this restriction.
Rule 1.7B18 and Chapter 13 of AACR2 apply [27,28].
Apart from the matter of periodical articles, there are several ways inwhich additional subject-rich entries could be provided, some for display,others to provide index terms for OPACs. Some of the MARC notes fields,
particularly 505 {contents) and 513 {summary), could be used to improve
both access and the informativeness of displayed records. Apart from its
use in analytical cataloguing, 505 can be used to contain information fromcontents pages, thus enhancing both access and informativeness as in the"BOOKS" project [7]. The MARC 513 field could contain abstracts,
which would be particularly useful for periodical articles (since abstracts
are often available). There is also an index terms field (695), reserved for
future use.
SIMPLIFYING BIBLIOGRAPHIC RECORDS

One recommendation for the simplification of MARC is for theelimination of a feature which must have irritated many users —statements of responsibility. It is possible to repeat the names in these in 700,
710, 711 (name added entry) fields, but these fields do not allow atpresent for recording the role of the person or body named except in thecase of collaborating author; AACR2 [27, Rule 21.0D] allows the optional
addition of designations of function, but this has not been implemented
in UK MARC. There are seven spare values (3-9) of the second indicatorwhich could be used as function designators (translator, illustrator, etc.).
If this were done, those libraries which wished to display something in
the nature of a statement of responsibility could still do so, while avoiding
the present duplication of data.
T H E "MAIN ENTRY" CONCEPT AND THE ORDER OF RECORDS IN OPACDISPLAYS

Going a little further, what would be the implications of eliminating theconcept of main entry} Except for some of the more primitive ones,
OPACs do not, and arguably should not, suggest to the user that there is9. Conclusions and recommendations

173

one physical sequence of records. There is evidence that users who haveslight acquaintance with OPACs feel that there is a sequence, but it seems
likely that OPACs will come to be regarded as entities which behave more
like reference librarians than catalogues. In any case there should be nosingle fixed sequence, but an indefinitely large number of apparent
sequences which depend on the search being made. (How many users of
online reference retrieval systems regard them as providing access toordered files?)Apart from the attitude of users, the elimination of the main entry
concept together with the statement of responsibility would remove aburden from cataloguers by enabling them to enter all names only in 700
and 710/711 (or 100 and 110/111) fields, instead of some in 7nn, some in
Inn or 245 and some in both.
Nevertheless, the question of order in OPAC displays does need attention
so long as considerable numbers of records may result from a search.
There is no space here for a full discussion of this contentious topic, but
one or two of the issues are touched upon. Clearly, the needs are very
different in subject and in specific item searching.
The main function of the apparent ordering of a browsing display is sothat the user knows whether to go forwards or backwards, and, mostimportantly, when to give up on the grounds that the sought item is notthere (think of searching a random telephone directory).
Traditionally, there were elaborate filing rules for catalogue cards, andfor dictionaries and street indexes and the like. Some of these rules
depended on knowledge which it is difficult or impossible to incorporate
in a computer program (even to make " 3 0 " file with "thirty" is a major
piece of coding). These rules are not usually known precisely by users(does D e a t h come before or after D e - a t h ? ) , but there is enoughconsensus that it is usually possible to reach approximately the right area
in whatever is being searched, and then to look at every entry.
In OPACs and other IR systems display order is sometimes imposed by
the fact that the easiest way to add new records to a file is to put them on to
the physical end of the existing file, which leads to records with identicalkeys being output in order of date of addition to the file (preferably
reverse order, since new material is usually in heavier demand than old).
Those who are worried by an apparent lack of order in, say, a display ofrecords of books by a prolific author, will doubtless be satisfied by a form

174

9. Conclusions and recommendationsof indexing which concatenates a portion of the title onto the author key(as in the first example in Section 5.3.1); but the systems which use thistype of indexing are primitive ones where there is no concept of "exact
match": items can only be found, as in a conventional catalogue, by
scanning. In any case, searches for works by an author form only a small
proportion of catalogue uses. In specific item searching browsing is
unnecessary when there is an exact match: being forced to browse is awaste of the user's and the system's time (Section 7.3.1). In subject
searching, on the other hand, browsing is essential, but there is only one
satisfactory way of ordering the displayed records: by probability of
relevance [29].
9.4.6

On the evaluation of OPACs and the study of their use

There is far higher variability among OPACs than there is among
conventional catalogues. There is no received opinion about what
constitutes a good OPAC, although the principles set out by Hildreth[22], and many of the recommendations given above, can hardly be called
contentious. There is no serious theoretical difficulty in making acomparison between two catalogue systems, provided that they serve thesame user population and access the same collections): the comparisonby Siegel and others of CITE with ILS in the National Library ofMedicine demonstrated this [30]. As a step towards the comparison of awide range of OPACs, there is something to be said for the use of a set ofstandard searches, carried out by an experienced researcher.
However, the purpose of evaluation must be to help us to design better
catalogues, and there are far too many variables, and inter-dependences,
for any highly objective and analytic evaluation to be helpful or evenpossible. It follows that the process of producing improved OPACs
necessitates studying what features of existing OPACs appear to beparticularly good, or bad, and why they are so.
Some OPAC properties can be evaluated without regard to the userinterface — these are what we have referred to as OPAC search functions(Sections 2.7.1 and 8.1.2). Available features can be tested without regard
to ease of use or whether they would be used by real searchers. Suchevaluation forms an essential part of the development process for any
OPAC, but since the user is an inseparable part of an interactive system,
the most important evaluation must take account of users' needs andbehaviour. This is not to say that every evaluation experiment has to havereal users: an experienced experimenter will have a very good idea of how

9. Conclusions and recommendations

175

users would react at various points during their interaction with the
OPAC.
So far as possible an OPAC must function efficiently given the often
conflicting requirements of different types of users and needs. Hencesome sort of classification of user types is important: apart fromdemographic data (Section 8.1.2), users can be categorised by theirdegree of familiarity with the OPAC under consideration, by their
exposure to interactive computer systems in general and by theiracademic level (Section 7.2). A classification of search types (user needs)
is also necessary.
WHAT SHOULD BE EVALUATED?

The following can be evaluated with respect to different types of users
and searches:(1)

Speed and effectiveness — in other words efficiency. It is not even
known how OP ACs compare with book, card or microform catalogues(except that OPACs are generally more popular). In particular,
there is an urgent need for a comparative study of catalogue use in alibrary which is in the process of changing from a conventional
catalogue to one of the more advanced OPACs.

(2)

Ease of use and of learning Irelearning, and how it relates to thefeatures available and the ways in which they are offered, andmethods of providing help and advice, of guiding the user throughthe search process, both within the OPAC and externally. Are there
features which are not used? If so, is this because they are notuseful, or because they are too difficult to use? What effect does
experience of one OPAC have on users' behaviour with another?

(3)

User input. To what extent, if any, are devices other than keyboards
useful? What input formats are most efficient and acceptable?
Should subject searches be entered in ''natural language", or is it
feasible to attempt to obtain search statements in a form reducible tosome kind of Boolean expression?

(4)

Display content and presentation, and questions of order, and of theprovision and use of browsing facilities.

9. Conclusions and recommendations

176

EVALUATION METHODS AND SOURCES OF DATA

The most easily obtainable data are from transaction logs, produced
automatically by the search system. Most OPACs incorporate somedegree of logging, but its nature and comprehensiveness vary widely. A
minimum requirement is that the logs should contain sufficient
information for searches to be repeated by an experimenter. It is highly
desirable also that they lend themselves readily to automatic analysis —that their format is such that it is easy to write computer programs toanalyse the logs in ways which were not necessarily foreseen. They should
also contain timings of every phase in the interaction, so that, forexample, the degree of familiarity of a user can be estimated from theamount of time spent reading instructions. One of the shortcomings ofautomatic logging is that, with an OPAC, it is often not possible todetermine session boundaries (that is, the point at which a specific user
starts and finishes a session — see Section 8.2) because it is not reasonable
to expect users to sign on and off as they do in some other types ofinteractive system.
There is much that cannot be deduced from even the most detailed oftransaction logs. This includes data about users' attitudes, needs, degreeof satisfaction and understanding of the system, as well as demographic
data (although some systems — Paperchase is an example — try to obtainsome of this information by asking the user to complete an online
questionnaire at the end of a session). Hence automatic monitoring has tobe supplemented by observation and by structured or unstructured
interviewing. There is also a place for the use of questionnaires — the oneused in the CLR study [25] was fairly comprehensive, although rather"closed" (Section 8.1).
9.5

Concluding remarks

The key problem in OPAC design is that of making sophisticated retrieval
facilities available, while retaining ease of use and reducing learning
effort to a minimum.
It is quite easy to achieve ease of use — we are certainly not under theillusion that Okapi is the only OPAC which is usable at sight. It is less easy
to make a retrieval system which compensates for searchers' lack ofknowledge of the ways in which items in its databases are described, andoutputs results in order of probable relevance to the request. It will be
extremely difficult to produce a system which achieves both goals,
although we do not believe that this is unattainable.

9. Conclusions and recommendations

177

Such an ''ideal" retrieval system would require very large investments infile design and access methods, in functional design and in theinvestigation of modes and structure of user interaction procedures.
Some of the methods for approaching this goal have been outlined in
Section 9.4. They require the use of advanced hardware (large amounts ofcore memory, high data transmission speeds, high quality display
devices), and the application of software techniques which have evolved
since the advent of personal microcomputers and in some of theapplications of artificial intelligence. Above all, they call for the study ofusers' needs, and for a continuous process of evaluation and feedback.
To date, no OP AC or other reference retrieval system even approaches
this ideal, and it is clear that few, if any, organisations in the library world
have the resources to make the required major investment.
New OPACs are appearing almost weekly, as part of integrated librarysystems. Since the market is highly competitive, and the OPAC has only
recently become one of system purchasers' requirements, many of the
suppliers add "public access" to their systems hastily and almost as anafterthought. If library management, from national libraries downwards,
would come to realise the increase in the use of library materials which
would result from the introduction of catalogues which actually help
most users to find what they are seeking, and that the systems they arebuying now may well be totally obsolete in five years' time, it might bethat automation suppliers and national or international bodies would
cooperate in helping to fund research into the development of thecatalogues of the future.
References
12
3
4

Persky G and others. A Geac local area network for the Bobst
Library. Library HI TECH (6), 1984, p37-45.
Markey K. Subject searching in library catalogs: before and after the
introduction of online catalogs. OCLC, 1984.
Kaske N K and Sanders NP.^4 comprehensive study of online publicaccess catalogs: an overview and application of findings. Final report to
the Council on Library Resources. Vol III. OCLC, 1983.
Walker S. The free language approach to online catalogues. In:
Keyword catalogues and the free language approach. Edited by Philip
Bryant. Centre for Catalogue Research. Bath University Library.
1985.

178
5

6

7

8
9

10
11

12
13

14

15

16

17

9. Conclusions and recommendations
Bates M J. Factors affecting subject catalog search success. Journalof the American Society for Information Science 28 (3), 1977, pi 61169.
Hildreth C R. Pursuing the ideal: generations of online catalogs.
In: Online catalogs, online reference — converging trends. Proceedingsof a LITA Preconference, June 23-24 1983, Los Angeles. American
Library Association, 1984.
Atherton P. Books are for use: final report of the Subject Access Projectto the Council on Library Resources. Syracuse University, School ofInformation Studies, 1978.
Markey K. DDC as a library user's tool. MARC Users' Group
Newsletter. Issue 85 No 1, March 1985, pl9-26.
Markey K. Subject-searching experiences and needs of online
catalog users: implications for library classification. Library Resources and Technical Services, January-March 1985, p34-51.
Harper D J. Relevance feedback in document retrieval. PhD Thesis,
University of Cambridge, 1980.
[Walker S ]. On the use of relevance feedback and classification to improve
subject access fin an online catalogue]. Draft proposal submitted to the
British Library Research and Development Department. 1984.
Yannakoudakis E J and Fawthrop D. The rules of spelling errors.
Information Processing and Management 19 (2), 1983, p87-99.
Yannakoudakis E J and Fawthrop D. An intelligent spelling
error corrector. Information Processing and Management 19 (2), 1983,
plOl-108.
Freund G E and Willett P. Online identification of word variants
and arbitrary truncation searching using a string similarity measure.
Information technology: research and development 1 (3), 1982, pi 77187.
Porter M F. Information retrieval at the Sedgwick Museum.
Information technology: research and development 2 (4), 1983, pi 69186.
[Walker S]. Improving access in an online public access catalogue byautomatic word stemming, spelling correction, and limited synonym
generation. Research proposal submitted to the British Library
Research and Development Department. April 1985.
Norman D A and Fisher D. Why alphabetic keyboards are noteasy to use: keyboard layout doesn't much matter. Human Factors 24(5), 1982, p509-519.

9. Conclusions and
18

19

20
21

22
23

24
25

26
27
28
29
30

recommendations

179

H i l d r e t h C R. [Users and online systems: the user interface]. Paper
given at a conference "Online public access to library files", Centre
for Catalogue Research, Bath University Library, 3-5 September
1984.
H a y e s P J and R e d d y D R. Steps toward graceful interaction in
spoken and written m a n - m a c h i n e communication. International
Journal of Man-Machine Studies 19, 1983, p231-284.
I n n o c e n t P R. T o w a r d s self-adaptive interface systems. International Journal of Man-Machine Studies 16, 1982, p287-299.
M a r c u s R S and R e i n t j e s J F. The networking of interactive
bibliographic retrieval systems. M I T Report E S L - R - 6 5 6 . N T I S
PB252407. M a r c h 1976.
H i l d r e t h C R. Online public access catalogs: the user interface. O C L C ,1982.
[Walker S]. The design and development of a self-adaptive user interface
for an online public access catalogue. Draft proposal submitted to the
British Library Research and Development Department. 1985.
S e a l A , B r y a n t P and H a l l C. Full and short entry catalogues.
Centre for Catalogue Research. Bath University Library, 1982.
M a r k e y K. Online catalog use: results of surveys and focus groupinterviews in several libraries. Final report to the Council on Library
Resources. Vol II. O C L C , 1983.
UK MARC
Manual. British Library. Bibliographic Services
Division. 2nd ed., 1980.
Anglo-American
Cataloguing Rules. Library Association, 2nd ed.,1978.
G r e d l e y E (Polytechnic of N o r t h L o n d o n ) . Verbal communication, M a r c h 1985.
R o b e r t s o n S E . T h e probability ranking principle in IR. Journal ofDocumentation 33 ( 4 ) , 1977, p294-304.
S i e g e l E R et al. A comparative evaluation of the technicalperformance and user acceptance of two prototype online catalog
systems. Information Technology and Libraries 5 ( 1 ) , 1984, p35-46.