Appendix A

Okapi system description
This Appendix illustrates and describes the systems used in the City University library and for registered network users. All the Figures except A.21
and A.22 illustrate the system used at the terminal in the library. There
are minor differences in the wording of messages and prompts between thelibrary and the network systems.
Figure A.l: Welcome screen* * * * * * * * * * * * **WELCOME TO OKAPI**************
This is an experimental system that will help you find booksin this library by subjects and keywords

So that OKAPI can learn to adapt to your needs it would be very
helpful if you would enter the rest of your library card numberI

2 8008I
If you don't want to help with our experiment but still want to
use this system please enter the sequence below
2UV7X 2X1W
If you would like more information press the BLUE key

If the user presses the BLUE key the message shown in Figur A.2 issuperimposed over the "Welcome Screen1'
If users enter a library card number a small window (or snippet) appears
telling them the last time they used the system. If they enter the "special
number" the snippet says "Thank you for your cooperation". The special
59

APPENDIX A. OKAPI SYSTEM

60

DESCRIPTION

Figure A.2: Information window for welcome screen
OKAPI
It is helpful to us if you enter your library card number
as this enables the computer to 'recognise' you when you
use the system again. We are hoping to use this information
to develop a system that will adapt itself to the needs ofindividual users.
We cannot and do not want to find out who you are from yourlibrary card number.
The OKAPI project is based in the Information Science Department in Room A223. Please call in if you would like to know
more about our work or if you have any suggestions.
Press the Return key to continue

Figure A.3: Information offered after user identification
OKAPI
In this experiment you will be asked for
information about the relevance of books
you find. OKAPI can then try to findsimilar books using the title and subject
words. This is called automatic query
expansion.
The OKAPI project is based in the Information
Science Department in Room A223. Please call in
if you would like to know more about our work,
or if you have any suggestions.
Type S to start your searchnumber is intended to be unmemorable so that users are more likely to use
their own library card number. It is randomly generated each time thesystem returns to the welcome screen.
Users are given three attempts to enter either their own or the special
number correctly. If they fail a snippet saying "sorry" appears under thenumber box, the screen goes blank for a few seconds and the welcome screen
reappears. Once a number hats been entered correctly users are offered
the choice of either starting their search or seeing more information. The
information at this stage is shown in Figure A.3.
If users do not press the RETURN key after typing in their search asnippet appears reminding them to do so.
When the search has been entered the "Results" screen of Figure A.6
appears

Figure A.4: Search input screen
SUBJECT SEARCH** OKAPIThe computer will look for books which include all (or most) of your words
in their titles or subject descriptions

Type a word or a phrase which describes the books you want

I Press BLUE |I for info |I BLACK key ||to quit|

Figure A.5: Information offered from search input screen
The computer will look for books described by as many as possibleof the words in your search - you can type as much as will fit.
To correct your typing, use the Delete key.
The books which seem to match best will be shown first.

Figure A.6: Search results screen
SUBJECT SEARCH for "insecticides and the environment*'
5 books under "insecticides"
1081 books under "environment"

One book matches your search well
(5 books found altogether)Type Display to look at the books found ?
(the most similar books should appear first)
Type Hew if you want to do a different search
Type Edit to change or add to your search

I BLACK key |I to quit|

62

APPENDIX A. OKAPI SYSTEMDESCRIPTION

The "BLACK key to quit" snippet appears on every display as it isimportant that the user logs off at the end of the session.
A spelling mistake or the entering of a word not in the database resultsin Figure A.7.
Figure A.7: Replacement of misspelt word
SUBJECT SEARCH for "insecticides and the enveronment"
5 books under "insecticides"
CAH'T FIID "enveronment"

Press the return key if you want to change this word
Press the space bar to continue without this wordor type lew if you want to do a new search

I BLACK key I|to quit|

Figure A.8: Brief record display
LIST OF BOOKS
Search: "insecticides and the environment"

Mo. Title

Books 1 to 5 of 5

AuthorDate

1
Organochlorine insecticides : persistent organic. (MORIARTY F)
1975
THE REST OF THE BOOKS MAY I0T MATCH YOUR SEARCH VERY WELL
2Chlorinated insecticides.
BROOKS G T
1974
3
Safe use of pesticides : twentieth report of the.. World Health Org.. 1973
4
Insecticide and fungicide handbook for crop prote.. British Crop Pro.. 1972
5
Insecticides : action and metabolism.
O'BRIEN R D
1967** EID OF LIST *•Type its number to see if a book is relevant
Type Options to see other things you can do
I BLACK key I|to quit|

In Figure A.10 the BLUE option informs users that if they choose somebooks the system will look for similar ones if they type MORE.

63

Figure A.9: Options from brief display
LIST OF BOOKS
Search: "insecticides and the environment"

lo. Title

Books 1 to 5 of 5

I0PTI0IS

|te

,

,__

1
Organochlorine insectici|You can do :THE REST OF THE BOOKS MAY 10 I
2
Chlorinated insecticidesl lew (start a new search)
3
Safe use of pesticides :| Edit or repeat your last search4Insecticide and fungicidl
5Insecticides : action anlPlease choose one** EID OF LIST **|I(or press the return key
I to return to the list of books)

|75
I
|74
173
172
|67
I
I
I

,
I
1

|__
I
,..

Type its number to see if a I

I

I

I

Type Options to see other th|I

I BLACK key ||to quit|

Figure A.10: Full record display
FULL DISPLAY
Search: "insecticides and the environment"

Book 1 of 5

AUTHOR(S): (Moriarty F)
TITLE(S): Organochlorine insecticides : persistent organic pollutants.
PUBLICATION Academic Press, 1975.
SUBJECT(S): Environment. Pollution by pesticides: Organic chlorine compounds.
Pesticides - Environmental aspects. Chlorine organic compounds.
Shelved at : 632.95042 MOR

I Press BLUE |-| for info |Is this the sort of book you are looking for? (y/n) YES|BLACK key |Ito quit|

64

APPENDIX A. OKAPI SYSTEMDESCRIPTION

When the user chooses the book of Figure A.10 as relevant appropriate
fields of the record is "indexed" and the terms extracted from it placed ina list with the original query terms. After this book has been chosen thelist will contain "insecticides" and "environment" from the original query,
together with "organochlorine" and the other title words, "pollution", "pesticides" etc from the subject headings, and the Dewey number 632.95042.
This list of terms, which is added to every time a record is chosen relevantby the user, is the list from which the "best" terms will be used for query
expansion.
As only one book, in this example, was a good match to the search andas that has been looked at and chosen as relevant, the user is now given
the choice of doing MORE (Figure A.11). If the original query results in
at least three good matches MORE is not offered until several of the books
have been looked at as the automatic query expansion usually works betterwith a larger pool of terms.
Figure A.11: Brief record display with "MORE" option
LIST OF BOOKS
Search: "insecticides and the environment"

lo. Title

Books 1 to 5 of 5

AuthorDate

1** Organochlorine insecticides : persistent organic.. (MORIARTY F)
THE REST OF THE BOOKS MAY HOT MATCH YOUR SEARCH VERY WELL
2Chlorinated insecticides.
BROOKS G T
3
Safe use of pesticides : twentieth report of the.. World Health Org..
4
Insecticide and fungicide handbook for crop prot. . British Crop Pro..
5
Insecticides : action and metabolism.
O'BRIEI R D** ESD OF LIST **

1975
1974
1973
1972
1967

Type its number to see if a book is relevant
Type More to look for books similar to the one you chose MORE
Type Options to see other things you can do

I BLACK key

I to quit

When MORE is chosen the system takes the original search terms together with those added from the title and subject fields and the classmarkof any chosen books and automatically expands the original query. The result of the new search is displayed in a window superimposed on the brief
record display (Figure A.12).
Figure A. 13 shows the two books found by the first iteration of query
expansion in this search.

Figure A.12: Results of query expansion
Books 1 to 5 of 5

LIST OF BOOKS
Search: "insecticides and the environment'

Ho. Title

AuthorDate

1** Organochlorine insecticides : persistent organic. (MORIARTY F)
THE REST OF THE BOOKS MAY I0T MATCH YOUR SEARCH VERY WELL
2Chlorinated insecticides
3
Safe use of pesticides : | Looking for more books similar to the4
Insecticide and fungicidlone you chose...
5
Insecticides : action an|** ESD OF LIST **|Found some more books

1975
74
173
J72
167
I

I

I

iType Disp to see the books

I

,
I

,_.
I

1 (or press return if you don't want to see them) | —Type its number to see if aIType More to look for books IType Options to see other th -I BLACK key ||to quit|

Figure A.13: Brief display from query expansion search
LIST OF BOOKS similar to the one you chose(Original search: "insecticides and the environment")

Books 1 to 2 of 2

No. Title

Author

Date

1
2

EDWARDS C A(MATSUMURA F)

1973
1972

Environmental pollution by pesticides.
Environmental toxicology of pesticides.
** EID OF LIST **Type its number to see if a book is relevant
Type Back to return to the books you originally foundType Options to see other things you can do

I BLACK key ||to quit|

66

APPENDIX

A. OKAPI SYSTEMDESCRIPTION

Figures A.14 and A.15 show that the records from the expanded querycontain many of the terms extracted from the originally chosen record (Figure A.10). Although both these books are indexed under "environment" therecords do not contain "insecticide" and the searcher would not have foundthem easily without query expansion.
Figure A. 14: Record from query expansion
FULL DISPLAY of books similar to the one you chose(Original search: "insecticides and the environment")

Book 1 of 2

AUTHOR(S): Edwards C A
TITLE(S) : Environmental pollution by pesticides.
Environmental science research.
PUBLICATIOI: Plenum Press, 1973.
SUBJECT(S): Environment. Pollution by pesticides: Organic chlorine compounds
Reviews of research. Pesticides - Environmental aspects. Chlorine
organic compounds.
Shelved at : 363.7384 EDW

I Press BLUE |I for info |Is this the sort of book you are looking for? (y/n) YESI BLACK key ||to quit|

The user chooses both the books shown in Figures A.14 and A.15. A
second iteration of query expansion results in the list shown in Figures A.16
and A.17.
The initial ''options" window (Figure A.9) just gives the user the choiceof doing a new search or editing the existing one. Once a book has beenchosen there are more choices as shown in Figure A. 18.
The VIEW option displays a list of the titles which the user has selected
as relevant. Choosing the PRINT option results in Figure A. 19.
The EDIT option displays the original search and allows the user to add
or delete terms. The screen of Figure A.20 shows the information snippet
for this screen after the BLUE key has been pressed.
The NEW option allows the user to type in a completely new search
while allowing the user to see up to their last three searches

67

Figure A.15:

FULL DISPLAY of books similar to the ones you chose(Original search: "insecticides and the environment")

Book 2 of 2

AUTHOR(S): (Matsumura F); (Boush G M); (Misato T)
TITLE(S) : Environmental toxicology of pesticides.
PUBLICATION: Academic Press, 1972.
SUBJECT(S) : Environment. Pollution by pesticides. Pesticides and theenvironment. Pollution.
Shelved at : 632.95042 MAT

I Press BLUE I| for inf0 |Is this the sort of book you are looking for? (y/n) YESI BLACK key ||to quit |Figure A.16: Records from second iteration of query expansion
LIST OF BOOKS similar to the ones you chose(Original search: "insecticides and the environment")

Mo. Title
1
2
3
4
5
6
7
8
9

Books 1 to 9 of 85

AuthorDate

Organometallic compounds in the environment.
CRAIG P
The use and significance of pesticides in the en . MCEWEH F L
Lead pollution : causes and control.
HARRISOI R M
Lead pollution : causes and control
HARRISOI R M
Lead and health : the report of a DHSS Working .
Great Britain. ..
Methodological approaches to deriving environmen . CALABRESE E J
Analytical aspects of mercury and other heavy me . (FREI R W)
International Co..
Environmental pollutants : selected analytical .
Further review of certain persistent organochlor . Great Britain. ..

1986
1979
1984
1981
1980
1978
1975
1975
1969

Down (next)
Type its number to see if a book is relevant
Type Back to return to the books you originally foundType Options to see other things you can do

I BLACK key |I to quit|APPENDIX A. OKAPI SYSTEM

68

DESCRIPTION

Figure A.17:
LIST OF BOOKS similar to the ones you chose(Original search: "insecticides and the environment")

Ho. Title
1
2
3
4
5
6
7
8
9

Books 10 to 18 of 85

AuthorDatelitrogenous air pollutants : chemical and biolog. . (GROSJEAI D)
Effects of airborne sulphur compounds on forests.. Discussion Group..
The non-agricultural use of pesticides in Great .. Great Britain. ..
The Bhopal syndrome.
WEIR D
Pesticides, boon or bane?.
GREEI M B
Pesticides and pollution.
MELLAIBY K
The use and significance of pesticides in the en.. MCEWEI F L
Chlorofluorocarbons and their effect on stratosp.. Great Britain. ..
Chlorine dioxide : chemistry and environmental .. MASSCHELEII W

1979
1976
1974
1988
1976
1970
1979
1976
1979

Down (next), Up (prev)
Type its number to see if a book is relevant
Type Back to return to the books you originally foundType Options to see other things you can do

IBLACK key |I to quit|

Figure A.18: Options after query expansion
LIST OF BOOKS
Search: "insecticides and the environment"

lo. Title

Books 1 to 5 of 5

I0PTI0IS-I
1** Organochlorine insecticiI You can do :THE REST OF THE BOOKS MAY 10|
2
Chlorinated insecticidesl lew (start a new search)
3
Safe use of pesticides :| Print out your list of chosen books
4
Insecticide and fungicidl View your list of chosen books
5
Insecticides : action an| Edit or repeat your last search*• EHD OF LIST **|(your selections will be lost)I
iPlease choose one
I(or press the return key-| to return to the list of books)
Type its number to see if a I

I
Type More to look for books IType Options to see other th|

IBLACK key-1 to quit

75
74
73
72
67

Figure A.19: Choosing the PRINT option
SHORT DISPLAY of the book you chose(Original search: "insecticides and the environment")

Book 1 of 1

1 Your list of books will be available in
Bo. Til the Information Science Research Room A223

te

1** 0r| You can collect it between 10.00 and 12.00**| or 14.00 and 17.00 Monday-Friday.
I Ve will keep your list for a week from today.
hosen books
I
I Type P to print your list
IsearchI
I or type 0 to go back to OPTIOHS
be lost)75

iPlease choose one P R U T
I(or press the return key-| to return to the list of books)
Type 1 to see details of thelI
Type Back to go back to the |Type Options to see other th|

I BLACK key-|to quit

Figure A.20: Editing a search
SUBJECT SEARCH

I Previous search(es)I 1"insecticides and the environment"** OKAPI

Books found |5 I

Type a word or a phrase which describes the books you want

I insecticides and the environment

I Use the Delete key (above the Return key) |I to get rid of anything you don't want.
I
I Type in any new words you want to use.
|

I BLACK key |I to quit|APPENDIX A. OKAPI SYSTEM DESCRIPTION

70

Figure A.21: Full record display: INSPEC
1 of 568

FULL DISPLAY
Search: "machine learning in information retrieval"
TITLE: Inference technique for distributed query processing in apartitioned network.
SOURCE: (Report) 1990)

HEADIIGS: Incomplete knowledge; knowledge based approach; query processing;
network partitioning; entity-relationship model; correlated
knowledge; attributes; database content; machine learningtechniques; rules; data inference; objects; correctness
criterion; toleration; algebraic tools; distributed databasesystem; ship database; model-based knowledge acquisition
methodology.
ABSTRACT: "A new knowledge based approach to query processing during
network partitioning has been proposed. The approach is based on the given
query and the use of available domain knowledge to infer inaccessible data.
Down to see the next page
I CTRL-Y ||to quit I

or press Return to continue

Figure A.22: Brief record display: INSPEC
LIST OF PUBLICATIONS
Search: "machine learning in information retrieval"lo. Title

1 to 9 of 568

AuthorDate

1** Inference technique for distributed query processing..
2
Using type inference and induced rules to provide in.. Chu, V W
3Structured information management using new techniqu.. Gibb, F
4** Proceedings of the 11th BCS IRSG Research Colloquium..
5
Machine readable information systems of learning opp.. Allred, J
6
Semantics of user interface for image retrieval: pos.. Crehange, M
7Image progressive retrieval from a videodisk: a raach.. Crehange, M
8Machine learning for 'intelligent' information retri.. Goker, A
9Automatic thesaurus construction by machine learning.. Guntzer, U

1990
1990
1990
1989
1989
1989
1989
1989
1989

Down (next)
Type its number to see if a publication is relevant
Type More to look for items similar to the ones you choseType Options to see other things you can do

ICTRL-Y I
I to quit |Appendix B

Okapi transaction logs
Log text is in Roman, annotations in italic
I User special.no Database city.feb91 Date Wed Jun 19 13:12:32 1991
Program /usr/local/bin/s.city.3.1.3 Description Okapi rf
Version 3.1.3 Rev date 31 Jan 1991 Pid 12278

Identification

T 910619131232

Timestamp.

The E lines hereafter are elapsed times in seconds from the previous E or T.

U s
C ** startT 910619131239
S SCR7.1

SCR7.1 is the search input screen.
U strucu<LEFT>ture<LEFT>al vibta<LEFTXLEFT>ration<RET>
E +16
L s t r u c t u r a l (744) , v i b r a t i o n (232) ,E +0User has typed a search with corrections, taking 16 seconds to type it. The system
744 postings for "structural" and 232 for "vibrationfs]'''.

found

0 NMPW = 10 NGW = 10 NAW = 10

System has reported "10 books match your search well.''
looks at a screen of brief records for 21 seconds.

71

User types d for Display and72

APPENDIX B. OKAPI TRANSACTION LOGSu d .
S BRIEFS
1
2
3
4
5
6
7
8
9

Sound and structural vibration : radiation, tran., FAHY F J
Structural vibration analysis : modelling, analy., BEARDS C F
Structural vibrations due to wind. Wind tunnel .. (FLINT A R)
Stresses, vibrations, structural integration and., BEITCH L
Formulas for natural frequency and mode shape.
BLEVINS R D
BLEVINS R D
Flow-induced vibration
Vibrations of engineering structures.
(BREBBIA C A)
GORMAN D JFree vibration analysis of beams and shafts.
Flow-induced structural vibrations : proceedings. (NAUDASCHER E)

1985
1983
1980
1979
1979
1977
1976
1975
1974

U 5
E +27
S FULLAUTHOR(S): Blevins R D
TITLE(S): Formulas for natural frequency and mode shape.
PUBLICATION: Van Nostrand Reinhold, cl979.
SUBJECT(S): Structures. Vibration. Formulae. Structural dynamics.
Vibration.
Shelved at : 624.171 BLE

U y
E +33

User has looked at the fifth record in full and answereduy" to the relevancequestion.

S BRIEFS
U 7
E +6
S FULLAUTHOR(S): (Brebbia C A)
TITLE(S): Vibrations of engineering structures.
PUBLICATION: Computational Mechanics Ltd, 1976.
SUBJECT(S): Structures. Vibration. Structural dynamics. Vibration.
Shelved at : 624.171 BRE

U y
E +7
S BRIEFS
U d
E +12
S BRIEFS
1
Structure-borne sound; structural vibrations and.. CREMER L** END OF LIST **U o
E +9
S OPTIONS
U v

1973

73
E + 12
S VIEWING
S BRIEFS
1** Formulas for natural frequency and mode shape.
2** Vibrations of engineering structures.
** END OF LIST **BLEVINS R D(BREBBIA C A)

1979
1976

Reviewing chosen records
U o
E +4
S OPTIONS
U e
E +14

User chooses to edit the search, adding the term

"aircraft".

S SCR7.2 ( e d i t )U structural vibration aircraft<RET>
E +22
L C0023 (430), structur (2433), vibration (232),
E +0
0 NMPW = 0 NGW = 1 NAW = 50
U ds BRIEFS
1
Introduction to the study of aircraft vibration . . SCANLAN R H
THE REST OF THE BOOKS MAY NOT MATCH YOUR SEARCH VERY WELL
BLAKE W K
2
Mechanics of flow-induced sound and vibration.
Mathematics of random phenomena : random vibrati, . KREE P
DOWDING C HBlast vibration monitoring and control.
Sound and structural vibration : radiation, tran. . FAHY F JFERTIS D G
Dynamics and vibration of structures.
Structural vibration analysis : modelling, analy. . BEARDS C F
Vibrational spectra and structure : a series of . . (MARTIN A E)
BLEVINS R D
Formulas for natural frequency and mode shape.

U 1
E +37
S FULLAUTHOR(S): Scanlan R H; (Rosenbaura R)
TITLE(S): Introduction to the study of aircraft vibration and flutter.
PUBLICATION: Macmillan, 1951.
Shelved at : 629.132362 SCA

y+7
BRIEFS6+GFULL
AUTHOR(S): Fertis D G
TITLE(S): Dynamics and vibration of structures.
PUBLICATION: Krieger, 1984.

1951
1986
1986
1985
1985
1984
1983
1980
1979

74

APPENDIX B. OKAPI TRANSACTION LOGS

Shelved at : 624.171 FER

U "I
U y
E +9

User requests information, which at this point informs user that the system can look formore records similar to the ones which have been chosen.

S BRIEFSU 7
E +7
S FULLAUTHOR(S): Beards C F
TITLE(S): Structural vibration analysis : modelling, analysis anddamping of vibrating structures.
Ellis Horwood series in engineering science.
PUBLICATION: Ellis Horwood, 1983.
SUBJECT(S): Structures. Vibration. Structural dynamics.
Shelved at : 624.171 BEA

U y
E +4
S BRIEFS
U m
E +1
S MORE
E +1
0 NMPW = 0 NGW = 5 8 NAW = 5 8User has chosen three records and now requests query expansion ("MORE"). System reports that some records have been found. It doesn't say how many unless there are lessthan a screenful.

Ud
S BRIEFS
1
Formulas for natural frequency and mode shape.
BLEVINS R D
1979
2
Vibrations of engineering structures.
(BREBBIA C A)
1976
3
Flutter suppression and structural load alleviat.. (Agard. Structu..
4
Stochastic methods in structural dynamics.
(SCHUELLER G I) 1987
5
Schaum's outline of theory and problems of dynam.. TUMA J J
1983
6
Analysis of structures by the force-displacement.. SMOLIRA M
1980
7
Sound and structural vibration : radiation, tran.. FAHY F J
1985
8Flow-induced vibration
BLEVINS R D
1977
9Several techniques for flight flutter testing : .. (Agard. Structur. 1960

U 9
E +21
S FULLAUTHOR(S): (Agard. Structures and Materials Panel); (Agard)

75
TITLE(S): Several techniques for flight flutter testing : papers
presented at a special symposium during the eleventhmeeting of the AGARD Structures and Materials Panel heldin Athens, Greece... 1960.
Agard AG56.
PUBLICATION: Agard, 1960.
Shelved at : 629.132362 AGA

U y
E +23
S BRIEFS
U d
E +11
S BRIEFS
1
Stresses, vibrations, structural integration and. BEITCH L
2Structural dynamics for the practising engineer.
IRVINE H M
3Dynamics of structures and machinery : problems . SZULADZINSKI G(DONEA J)
4Advanced structural dynamics.
5
Structural dynamics : theory and computation.
PAZ M
6The component element method in dynamics : with . LEVY SWARBURTON G B
7The dynamical behaviour of structures.
CLOUGH R V
8Dynamics of structures.
K0L0USEK V
9Dynamics in engineering structures.

1979
1986
1982
1980
1980
1976
1976
1975
1973

U 8
E +20
S FULLAUTHOR(S): Clough R W; (Penzien J)
TITLE(S): Dynamics of structures.
PUBLICATION: McGraw-Hill, [1975].
SUBJECT(S): Structural dynamics
Shelved at : 624.171 CL0

U y
E +4
S BRIEFS
U o
E +5
S OPTIONS
U v
E +3
S VIEWING
S BRIEFS
1951
1** Introduction to the study of aircraft vibration . .SCANLAN R H
1984
FERTIS D G
2** Dynamics and vibration of structures.
1983
3** Structural vibration analysis : modelling, analy.. BEARDS C F
4** Several techniques for flight flutter testing : .. (Agard. Structu. . 1960
1975
CLOUGH R W
5** Dynamics of structures.
** END OF LIST **Now reviewing the three records originally chosen and the two chosen from query

expansion.

76

APPENDIX B. OKAPI TRANSACTION LOGS

U 4
E +10
S FULLAUTHOR(S): (Agard. Structures and Materials Panel); (Agard)
TITLE(S): Several techniques for flight flutter testing : papers
presented at a special symposium during the eleventhmeeting of the AGARD Structures and Materials Panel heldin Athens, Greece... 1960.
Agard AG56.
PUBLICATION: Agard, 1960.
Shelved at : 629.132362 AGA

U <RET>
E +11
S BRIEFS
U -Y

Quit,

finished.

E +3
C ** Record display summary - this search
Total unique briefs: 25
Fulls 5, chosen 5, rejected 0

Appendix C

Questionnaires
C-lQuestionnaires for library users

C.l.l

E x h a u s t i v e n e s s of search

To determine the level of exhaustiveness of search intended by the user. Not
introduced until quite late in the experimental period.
Are you looking for:1. One or two books on a subject
2. A reasonable selection of books
3. As many books as possible
4. Just one particular title
5. None of the above—please say what you do want

C.1.2

P r e v i o u s experience

To determine what, if any, previous experience users had of computerised
catalogue systems.
1. Have you ever used the other computer catalogue in this library? (i.e.
CLSI)
2. Have you ever used any other computer catalogues? If so, which ones?
3. Have you ever used this one before?
77

APPENDIX C.

78
C.1.3

QUESTIONNAIRES

Post-search questionnaire

For users who declined to have a structured interview, a short questionnaire
to determine level of satisfaction with the system.
1. Did you find the system easy to use?
(on a scale of 1-5, with 5 being very easy)
2. Do you think the system is easier/less easy to use than the other librarysystem?
(this was only asked were the users had indicated use of the othersystem)3. Did you need help anywhere where it was not provided?
4. Were you pleased with the results of your search?
(on a scale of 1-5, with 5 being very pleased)
5. Did you have to look at a lot of books in order to find the useful ones?
6. Did you use the "MORE" option?
7. Did you find it useful?

C.1.4

Structured interview/ / user had used the "MORE" option then the following questionnaire wasused to structure the interview. Questions were asked in conjunction withthe printed log of the user's search. A sheet was completed for each searchin the session.
If the user had not used the "MORE" option the search was replayed fromthe log with the user, and query expansion was carried out. The questionnaire used was the same as the one below as far as question 5. Users wereasked why they had not used "MORE". They then looked at the results of thequery expansion and were asked for relevance judgments on additional items
found. Questions 8, 9 and 10 were the same as the questionnaire below.
1. Did you find the system easy to use?
2. Do you think this system is easier/less easy to use than the otherlibrary system?
3. Were you pleased with the results of your search?
4. Why wrere some of the books in your not looked at in full?
5. How did you decide the ones you chose were relevant?
6. Why did you decide to do "MORE" here?

C.2. QUESTIONNAIREFOR NETWORK

USERS

79

7. Did it work as you hoped it would?
(here the interviewer looked at the results of query expansion on theprinted log with the user)
8. (A general question about the use/non-use of the Options)
9. If this is not the first search how is it related to the previous one?
(The categories, allocated by the interviewer, were Not related, Broader,
Synonymous, Narrower, Other relationship^
10. If a further search was carried out was it related to this one?
(The categories were the same as for the previous question—again allocated by the interviewer)

C-2

Questionnaire for network users

1. Context(a) What is your field of work, and what is the aim of your work?
(b) What stage are you at?
(c) Do you belong to a particular research group?
2. Okapi—general(a) Did you find the system easy to use? Why?/why not?
(b) Did you try all the databases?
(c) Which did you find most useful?
(d) Did you find that you had to adapt or change you search behaviour or expectations over time?
3. Search—details(a) How did you decide which brief records to look at in full?
(b) How did you decide whether the full records were relevant?
(c) Did you notice any indication in the list of brief records when
they became less relevant?
(d) Did you have to look at a lot of references to find the useful ones?
(e) Did you use the "MORE" option? Why?/Why not?
(f) Did you find it useful?
4. Conclusion(a) Have you ever performed similar searches manually or other online or CD-ROM sources?
(b) Are there any features about the system you particularly liked or
disliked?

Appendix D

Notes on a frequent user
The following is a slightly edited transcription of the experimenter's notes
on one of the frequent Okapi users discussed in Chapter 4.

Frequent User: N E T - D
Last Log: 12/10/90, Day Printed: 27/11/90, Logs: 1-25, Total Bytes: 293
K, Total Online Time: 1 hr 38 mins 38 sees

SUBJECTS:• Hi-fis, sound.
• Bartok, Bach, St. Matthew passion, celesta, concerto, post horn serenade• Unix, OS.
• Robot vision.
• Environmental geology, chemistry.

GENERAL COMMENTS:• Searches INSPEC and City Databases.
• Used "MORE" in early sessions.
• Views after choosing references and then Prints.
• Quite a few short sessions with resulting short reference list from query.
Possibly screen dumps or makes notes of references.
80

81• Seems to read most of the references on the screen when looking at
brief records.
• Has few sessions with quite several "new" searches which are contextually new as well.
• Repeats some searches.
SPECIFICS:• When typed in "sound" spent 1-6 sees looking at the brief record
screens called by Up and/or Down (there was no "THE REST OF..." point). Chose audio related references on whole and then did"MORE". From the list generated after the query expansion the user
then chose "hi-fi" and "speaker" related topics. Perhaps some savings
could have been made if a link was formed with previous usage of hi-fi.
• Log 2 and 9: Again seemingly looking through a lot of screens (Downs)
to find the "THE REST OF..." point. In log 9, however, the user chosea few of the records to be relevant and did "MORE" twice for the same
query. Most of the ones that the user looked at in full were generally
chosen to be relevant (query: "robot vision" and subsequently "robot
vision control motion").
• Log 13: Would have been possible to detect the most likely context
of the query "speaker" from previous searches i.e. "loud speakers"
as opposed to "Mr. Speaker" and so on. The references could havebeen presented in the appropriate order of the context identified, forexample.
• Logs 14, 15, 16 and 17 can be disregarded. "CTRL-S" seems to berepeated not just in start of session but in mid of a session as well (log16).1
• Log 18: As per log 13 could have reordered the brief reference list for
the query "transmission speaker" for the same search.
• Searches on "loud speaker" repeated in form of short sessions.
• Log 21: One very good match of a reference title to a query. Again,
perhaps putting such a reference at top of list might be an improvement.
• Log 25: An author search ("Bartok") on the City database. However,as the system does not enable an author search on the City databasethe user was presented with a list of biographies of Bartok. Hence,
the user then typed in the query "celesta" as this is part of the titleof Bartok's books.
Presumably communication problems. CTRL-S is a flow control a X-OFF" code, nota command in the Okapi system.

82

APPENDIX D. NOTES ON A FREQUENT

USER• With another search of "Bach St. Matthew passion", in this session,
the user did not seem to be able to find what he/she was looking for.

Bibliography[1] CROFT W B and HARPER D J. Using probabilistic models of document retrieval without relevance information. Journal of Documentation 35 4 1979.
[2] DOSZKOCS T E. CITE NLM: Natural-language searching in an online
catalog. Information Technology and Libraries 2 4 Dec 1983 364-380.
[3] HANCOCK-BEAULIEU Micheline. Subject searching behaviour at thelibrary catalogue and at the shelves: implications for online interactive
catalogues. Journal of Documentation 43 4 Dec 1987 303-321.
[4] HANCOCK-BEAULIEU Micheline, ROBERTSON Stephen and NEILSON Colin. Evaluation of online catalogues. British Library 1990.
(British Library Research Paper)[5] JONES Richard M. A comparative evaluation of two online public accesscatalogues: user opinions about the design of online catalogues. British
Library Research and Development Department, 1988. (British Library
Research Paper 39)[6] MICHALSKI R S, CARBONELL J G and MITCHELL T M (eds) Machine learning: an artificial intelligence approach. Morgan Kaufmann,
1983.
[7] MITEV N N, VENNER G M and WALKER S. Designing an online
public access catalogue: Okapi, a catalogue on a local area network.
(Library and Information Research Report 39). British Library, 1985.
[8] MITEV N N and WALKER S. Information retrieval aids in an online public access catalogue: automatic intelligent search sequencing.
In: Informatics 8: Advances in intelligent retrieval. Proceedings of an
Aslib/BCS conference. Oxford, 16-17 April 1985. London: Aslib, 1985.
[9] ROBERTSON S E. The Probability Ranking Principle in Information
Retrieval. Journal of Documentation 33 294-304 1977.
[10] ROBERTSON S E, On Term Selection for Query Expansion. Journalof Documentation 46 4 Dec 1990 359-364.
83

84

BIBLIOGRAPHY[11] ROBERTSON S E and SPARCK JONES Karen. Relevance weighting ofsearch terms. Journal of the American Society for Information Science
May-June 1976 129-146.
[12] ROBERTSON S E and THOMPSON Catherine L. Weighted searching: the CIRT experiment. In: Informatics 10: prospects for intelligent
retrieval. University of York, 21-23 March 1989, pl53-166. Aslib, 1990.
[13] WADE S J and WILLETT Peter. INSTRUCT: a teaching package forexperimental methods in information retrieval. Part III. Browsing, clustering and query expansion. Program 22 1 Jan 88 44-61.
[14] WALKER Stephen. Interactional aspects of a reference retrieval systemusing semi-automatic query expansion. In: Informatics 10: prospects
for intelligent retrieval. University of York, 21-23 March 1989, pi 19136. Aslib, 1990.
[15] WALKER Stephen. The Okapi online catalogue research projects. In:
The online catalogue: developments and directions. Edited by Charles
R Hildreth. Library Association, 1989. p84-106.
[16] WALKER Stephen and JONES Richard M. Improving subject retrieval
in online catalogues: 1. Stemming, automatic spelling correction andcross-reference tables. British Library, 1987. (British Library Research
Paper 24)
[17] WALKER S and DE VERE R. Improving subject retrieval in online
catalogues. 2. Relevance feedback and query expansion. British Library,1990. (British Library Research Paper 72)[18] American National Standards Institute. ANSI/NISO Z39.50-1988. Information retrieval service and protocol: American National Standard
for information retrieval service definition and protocol specification forlibrary applications. New Brunswick and London: Transaction Publishers for National Information Standards Organization, 1988.