H p p e? m cil ± ><F I L B1

E5 T R L J C T L J R E E 5

Bibliographic

fiLes

The r u n - t i m e database c o n s i s t s of records each c o n t a i n i n g a d i r e c t o r y ,one f i x e d f i e l d and seven v a r i a b l e f i e l d s . The d i r e c t o r y c o n t a i n s theLength of each of the v a r i a b l e f i e l d s . The f i x e d f i e l d c o n t a i n s an
encoded c o n t r o l number, language, d a t e , Dewey e d i t i o n number andh o l d i n g s i n f o r m a t i o n . The v a r i a b l e f i e l d s are
Ruthor(s) and other names extracted from MRRC 100/110/111/700/ 710/711.
Personal names are held as surname and i n i t i a l s . Personal and corporate names
are distinguishable.
Main t i t l e and subtitle from 245, or occasionally 240 (uniform t i t l e } .
This is the only f i e l d apart from the directory and fixed f i e l d which isalways non-empty.
Publisherand edition

from 260 and 250.

Series and part titles
i n the following order: 440, 490, 248, 250, 260.
For records w i t h a n a l y t i c a l entries any contents note CMHRC 5053 i s incLuded
here.

Dewey numbers from MRRC 082 or a local field. The first Dewey number isfollowed by a spine label from a local field Cthe structure does not cater fordifferent copies of the same work having different shelfmarks].
Sites and number of copies at eachsite

Subject Headings from 600, 610, 511, 650, 651, 083. (083 in UK MRRC
contains verbal feature headings derived from PRECI5 fields.)
Indexes to bibliographic files
R fairly conventional inverted index system is used. Each index consistsof separate primary and secondary indexes, and a postings file.
Secondary indexes are blocked files (block size is four kilobytes], witheach block containing a variable number of variable length index
term
records.
Each index term record includes a term (which may be a word,a stem, a phrase or a token representing a class of "synonymous" terms],
a pointer to the Location of this term's postings in the postings file,
pointers to the next and previous index term record, and some
information about the roleCs] of this term in bibliographic records andits number of postings. CWhen a term has only a few postings these are
directly stored in the secondary index instead of in the postings file.
This is not of much significance in a subject search system, but saves
disk access when searching, for example, a titLe phrase index, where thegreat majority of index terms occur in only one bibliographic record.]
fl primary index is an indexed sequential file whose records are the last index
term in each block of the corresponding secondary index. It is normally heldin fast internal memory while a search program is running.
Postings are only 32 bits long. The first 24 bits represent the position

-83-

Hpp

1

File

structuresof a bibliographic record, and the remaining eight give some indicationof the role or position of the index term in this record. The exact
function of the role bits varies from index to index.
Rn index search proceeds as follows. The sought term is Looked up in the
primary index using a modified binary search. Usually it wilL not befound, but what is returned by the primary search is the number of thefirst primary index record which is greater than or equal to the soughtterm. Thus if two successive primary index records are i cat i and "dog"and the sought term is "catalogue" the primary search will return therecord number of "dog". This will be the number of the secondary index
block which will contain "catalogue" if it occurs in the index.
Following the primary search the appropriate secondary index block isread from disk. This is scanned sequentially record by record until
either an index term record for "catalogue" is found or an index term
greater than the sought term is reached. The procedure returns "success"or "failure" and the address of the index term record which has beenfound.
This very simple structure does not lend itself to easy updating, but it
is quite fast, requiring one disk access and a mean of about a hundred
string comparisons for each term search. Rn otherwise idle machine (Sun3/50 D can Look up about 30 terms a second.

-84-

R p p> e? m czl ± ><I N P U T

2

P R O C E S S I N GThis is a more detailed description of the procedures applied to user
input outlined in 3.5. Much the same processing is done on bibliographic
input during the generation of index terms.
Input is put into Lower case and punctuation and spacing are normalized.
Repeated blanks are reduced to single blanks. Punctuation not followed
by a blank is replaced by a blank Cunless it is in a number, or part ofan apparent "initiaLism" Like "u.s.a." in which case it is removed).
Initialisrns containing blanks are also preserved Cnu s a" --> "usa").
Rfter this stage the search statement consists only of alphanumerics,
blanks, hyphens and apostrophes, the processing of hyphens is fairly
elaborate, although they are not very common in user input. Most hyphens
are either removed or replaced by a single blank. The sequences " - "," -" and •- " are all replaced by a blank except when they areapparently part of a range of dates, in which case they are replaced bya single hyphen. Some hyphens are dealt with by look-up Cso the arabic"al-" can be removed, and the system also knows some common prefixes
like "post", "micro" and "neo"). Two hyphens representing a dash are
replaced by a blank. In other cases hyphens are replaced by blanks if
there is more than one of them in a "word" or if a single hyphen
occupies position five or greater in a word. RLL other hyphens are
removed. Rpostrophes are removed unless they are in the second or the
penultimate position in a word. Rmpersand with a blank on each side
becomes " and ". RLl other characters are removed. If at this stage thesearch is "empty" the input box clears and the system awaits further
input.
The text now consists only of lower case letters, digits, blanks and theoccasional hyphen Cin date ranges) and apostrophe. In searching, theresulting pre-processed string is what will be shown to the user as thesearch. Following the pre-processing stage the text undergoes "weak"stemming and spelling standardization. Weak stemming removes regular
english plural and "ingB endings with reasonable reliability. Spellingstandardization copes with many of the orthographical differences
between British and Rmerican english - for details see CWRLK87b, p653.
The additional "strong" stemming - removing suffixes such as
8
Lizlation", "ous" and "ly" - discussed in the earlier report is not
used in the current systems, as we concluded that it was rarely
beneficial and sometimes detrimental. Strong stemming certainly has aplace in searching small or highly specialised databases.
One further point about search processing is worth making. In previous
Dkapi systems we have always checked searches for duplicate terms["history of spain and history of portugal" was treated as "history of
spain and portugal"). This gave rise to complications in the [admittedlyrare) cases when two members of the same synonym class occurred in thesame search. When processing "haiti and haitian politics" the system had
to display '"haitian" incLuded under "haiti"'. The present systems do
not look for duplication. This results in a term which occurs more thanonce being given Cat Least) twice the weight it would have if it only
appeared once. This only applies to the initial search, not to expanded
queries, which never contain repeated terms.

-85-

R p p e n d ± x

3

Q U E 5 T I D N N R I R E 5

1

Questions asked before session on dumb system1.

Rt which college are you a student?

2.

Which course are you studying?

3.

Which year are you in?

4.

Do you use a library in connection with your college work? YE5/N0

I f YE5
5. Which library/ies do you use?

If MORE T H m ONE
6. Which library do you use most frequently?

7. Does this library have a computer for you to use to look up books?
YES/NO
If NO
8. What do you use to look up books?

If YE5
9. Do you ever use it?

YES/NOIf YES-86-

Rpp 310.

Questionnaires

How often do you use i t ?

a. once a termb. once a monthc. once a week
d. a few times a week
e. every day

11. What sort of books do you look up on the computer?
a. specific books
b. books on a topic
c. both of these
12. Have you ever used a computer in any other Library to look up books?
YES/NOIf YE5
13. Which library was it?

14.

Do you ever use any other type of library catalogue?

YE5/N0a. cardb. microfichec. printed book ...
d. other2

Questions asked a f t e rs e s s i o n o n dumbsystem1.

About what proportion of the time did you feel that the computer
was useful i n helping you to find bocks which 'were relevant to theessay t i t l e s you were given?

a. 0%
b. 25%
c. 50%
d. 75%
e. 100%Please give reasons for your answer

2.

Did you find it helpful to be able to ask the computer to savedetails of books for you to look at later? YE5/N0
Comments/if NO please explain why not

3. Did you have any problems using the computer?
If YE5
4. Please describe the problems you had.

-87-

YE5/N0Rpp 3

5.

Questionnaires

Can you think of any improvements which could be made to the system?
YE5/N0If YES
6. Please describe them3

Questions asked a f t e r s e s s i o n on qe system

1.

Rbout what proportion of the time did you f e e l that the computer
was useful i n helping you to f i n d books which were relevant to theessay t i t l e s you were given?

a. 0%
b. 25%
c. 50%
d. 75%
e. 100%Please give reasons for your answer.

2. Did you find it helpful to be able to ask the computer to savedetails of books for you to look at later? YES/hU
Comments/if NO please explain why not

3. Did you use the •MORE1 option?

YE5/N0If YES
4. Did it help ycu to find more useful books?

YES/NO

Comments/if NO please explain why not

5. Did you have any problems using the computer?

YES/NDIf YE5
6.

Please describe the problems you had.

7. Can you think of any improvements which could be made to the system?
YE5/N0If YES
8. Please describe them.

-88-Ppp 3
4

Questionnaires

Questions asked a f t e r session on f u l l system

1. About what proportion of the time did you feel that the computer
was useful in helping you to find books which were relevant to theessay titles you were given?
a. 0%
b. 25%
c. 50%
d. 75%
e. 100%Please give reasons for you answer

2. Did you find it helpful to be able to ask the computer to savedetails of books for you to look at later? YE5/N0
Comments/if NO please explain why

3. Did you use the 'MORE' option?

YE5/N0If YES
4. Did it help you to find more useful books?

YE5/N0Comments/if NO please explain why not

5. Did you opt to look at books shelved near the one you had chosen?
If YES
6. Did this help you to find more useful books?

YE5/N0Comments/if M ] please explain why

7. Did you have any problems using the computer?

YES/NOIf YES
8. Please describe the problems you had.

9. Can you think of any improvements which could be made to the system?
YES/NO
10. Please describe them.

-83-

YE5/N0Flpp 3

Questionnaires

5Comparison q u e s t i o n s asked a f t e r

s u b j e c t had used two systems

1.

Which system did you find the easiest to use?
a. First
b. Second
Why was this?

2. Which system did you find the most helpful in finding books whichwere relevant to the essay titles you were given?
a. First
b. Second
Why was this?

-30-

H p> p> ee m c::l ± ><
T HSK

-4

S H E E T 5

Rrt 1
1.

Comment on the contribution of art to political propaganda.

2.

Did British pop art differ significantly from its Americancounterpart?

3.

What kinds of Gothic were revived in the 19th century?

4.

Describe the development of the decorative arts in Britain since
1800.

5.

Assess the contribution women have made to art.

Hrt 2
1.

Was it possible for surrealist painting to fulfil surrealist
theory?

2.

How do the mass media affect popular arts and culture?

3.

Discuss the development of parish church architecture in Britain
during the middle ages.

4.

Discuss painting as a vehicle for social comment through the ages.

5.

Hew important was classical Reman architecture for Bruneileschi?

Computing 1
1.

Describe how files are organised under the CP/M operating system.

2.

Describe the main components of a digital computer system
highlighting the main connections between them.

3.

What are the strengths and weaknesses of COBOL compared with otherstructured programming languages?

4.

What factors should be taken into consideration in dialogue design
if human-computer interaction is to be problem-free?

5.

Describe the role of computers in medicine.

Computing 2
1.

Describe a model of a typical microcomputer indicating whichlevels use compilation and which use interpretation as their
method of program translation.

2.

Describe some computer architectures which differ from the
von Neumann model.

-31-

Rpp 4

Task

sheets3.

How could the installation of a local area network be of
benefit to an organisation?

4.

What factors should be taken into account when designing amanagement information system?

5.

How can computer data be transmitted through the telephone network?

Engineering 1
1.

What is the function of a comparator and what design criteria
should it meet?

2.

What precautions should be taken when casting metal items toensure a good quality product?

3.

What is the difference between "static pressure" and "stagnationpressure" applied to a fluid and how do they help determine
velocity?

4.

What equipment is used to manufacture electronic circuit boards?

5.

Describe the principles of strengthening processes in metals.

Engineering 2
1. What properties must a cutting tool material exhibit in order toremain effective during the metal-cutting process?
2.

What factors should the engineer be aware of to ensure safety inthe machine workshop and with what legislation should he comply?

3.

Describe the high pressure polymerisation of polyethylene.

4.

Describe the Frank-Read mechanism of dislocation generation.

5., When are radioactive isotopes used in engineering?

Life sciences 1
1.

Describe the effects insecticides can have on the environment.

2.

Compare the relative influence of heredity and culture on intelligence.

3.

Discuss the processes by which water, ions etc. move acrosscell membranes.

4.

In what ways has man been seen to adapt to his enviroment?

5.

Outline the mechanisms and principles of plant nutrition.

Life sciences 2
1. Outline the main issues in the biological control of disease-92-

Rpp 4

Task

sheetsin plants.

2.

How accurate are personality tests as predictors of
behaviour?

3.

Discuss the relationship between host and parasite withspecific reference to man.

4.

Describe the effects of pollution on the ecosystem offreshwater resources.

5.

Discuss the molecular and cellular basis of the immune system.

Social sciences 1
1.

Discuss the role of multinational companies in the economicsof the third world.

2.

Were the Germans right to criticise the peace settlement of
Versailles in 1319?

3.

Outline the role of social factors in illness.

4.

What affects the human's ability to remember?

5.

Would the introduction of an electoral system based on
proportional representation Lead to a more representative
government in Britain?

Social sciences 2
1. Discuss the role of the mother in the development of thechild.
2.

How do sociological perspectives shed light on the role
education in society?

3.

Would perfectly competitive markets ensure maximisation ofsocial welfare?

4.

How widespread was the Slump by 1332?
national recovery programmes.

5.

Rssess the significance of the Alliance as a "third force" in
British party politics.

Compare the various

General 1
CThe general question sheets were used by library andinformation science students. They are made up from the otherquestion sheets.3
1. When are radioactive isotopes used in engineering?
2.

Outline the main issues in the biological control of disease
in plants.

-33-

Rpp 4

Task

sheets3.

Discuss painting as a vehicle for social comment through the
ages.

4.

Discuss the role of the mother in the development of thechild.

5.

What factors should be taken into account when designing amanagement information system?

General 2
1. Describe the role of computers in medicine.
2.

Outline the role of social factors in illness.

3.

What kinds of Gothic were revived in the 19th century?

4.

Compare the relative influence of heredity and cuLture on
intelligence.

5.

Describe the principles of strengthening processes in metals.

-34-

H (ZD p e? m cd ± ><5

R N N D T R T E D

E X T R R C T 5

FRCDM

F)

L O G

F I L EKey
Lines from the Log are in roman, comments and annotations in italic.
The first character in each Line identifies the type of data:
I Identification
T Date and time to nearest second
C Comment to improve readability
S Screen identification
U User searches and commands
E Elapsed time to nearest second since Last "E" or "Tu Line.
L Lookup. Contains enough information to reconstruct what was
displayed on the screen.
0 Only used here to show number of items with maximum possible
weight, minimum good weight and minimum acceptable weight.
These figures can be used to deduce how the result of a search
was presented to the user.

The log
I 22732 - May 25 1988
Log number and date
T 880525GS5930
Times tamp. Date and time to nearest second.
C 15 f
Subject number and system identification- inserted by theexperimenters.

C
C ** start
T 880525100135
C ** new topic art 1.2
Question reference (see Appendix * ) . The question is 'Did British pop art
differsignificantly from its

Americancounterpart?'.

5 SCR7.1
Screen is new subject

search input

screen (Fig

3.*J.

U poo ary<DEL>t bntish<RET>
Subject enters search,making one correction,and presses

Return.

E +62
62 seconds has elapsed since thetimestamp.

L pop (10), art C1822), §0227 C12666),
Lookup results. The 'token'§0227 represents

the class of terms treated asynonymous with "british" ("Britain", "Great Britain',:+1
Lookup and merge took a second.
C start
0 NMPW = 1 NGttl = 10 NRW = 10

"GB" etc).

One record contains all the terms and there are W of 'good' weight. The
system has retrieved all the records for 'pop'. Six of them also contain'art'.
The last four records contain 'pop' only, and are all false drops (the Last isabout a computer programming language).

System reports "1 book matches your search well HO books found altogether)".
Ud

-95-

Rpp5

Extracts

froma transaction

Log

User chooses 'Display* o p t i o n .
5 BRIEFS

Brief record display.
U1
E +41
5 FULL
Having spent nearly 41 seconds looking at the brief display, the user chooses
to see the first record in full. It is George Melly's 'Revolt into style: thepop arts in Britain'.
Uy
Yes, it is relevant.
Uy'Yes, I would like to see books classified near this one.'
E +7
Seven seconds looking at full record and answering the two guestions.
5 BR0W5EJEWEY at 700.341
5 BRIEFS
Ud
E +33
User spends 33 seconds looking at the first screen of brief records, then goesforward one screen (nine records) in classified order. Unfortunately 700.341 is
a very general number representing art in the British Isles.
S BRIEFSUd
E +4
S BRIEF5
Ud
E +4
S BRIEFS
Ub
Having gone forward two more screens the user reguests 'Back' and returns to
the first
screen of the records originally found. Reading time for each screen
is probablv about 3 112 seconds 14 seconds less display time).

E +7
5 BRIEFS
U 4
E +7
Looks at fourth original record in full ...
5 FULLUy... and judges it relevant.
Uy
E +6
5 BR0W5EJBEY at 703.046
Despite previous experience again chooses to see classified list ...
5 BRIEFS
Ub
E +14... but after looking at it for 14 seconds goes back to the original list;
although some of the records could welt have been useful this is not particularly
evident from the one-line displays.
5 BRIEFSU5
E +10
S FULLUy
Un
E +14
Chooses another record, but not to see books classified near it.

-96-

Rpp5

Extracts

froma transaction

Log

5 BRIEF5Um
E t9
5 MORE
E +2
0 NMPW = 0 NOW = 0 NRW -- 6
Tries the "Nope* (query expansion) option. The search takes 2 seconds. It
finds six records, but they are apparently not very similar to any of the threerecords chosen so far.
Ud
5 BRIEFS
Ub
E +33
S BRIEF5Up
E +4
5 RESTRRTUq
E +6
E finishT 880525100553
E ** Finish - duration 2G4 sec(s)C ** start
T 880525100553
E ** new topic art 1.3'What kinds of Gothic wepe revived in the 19th centupy?"
5 5ER7.1
U 13th century gothic<RET>
E +34
L §0013 (738), centuri 12302), gothic (49),§0013 peppesents "13', '13th', 'XIX' etc. 'centuri'
is what the stemming
procedure makes of "centupy' OP 'centuries' (it is not shown to the usep
like this).
E +1
C start
0 NMPW = 2 NGW = 2 NOW = 43
'2 books match your search well (43 books found altogether)"li d
5 BRIEFSU1
E +22
5 FULLUyuv

Chooses firstE +10

record and looks at books classified neap it.

5 BRCWSEJEWEY at 720.342

5 BRIEFS
U1
E +15
5 FULLUy
E +6
S BRIEFS
Ub
Having chosen one book from classified display petupns to original list.
E +3
S BRIEFS

-97-

RppS

Extracts

fromatransaction

Log

U 2
E +6
5 FULLU (RETxDEDy
Here user makes two inappropriate keystrokes. Since neither of these keys isused except during input of a search it is possible that the user was trying to
decline to answer the relevance question, an unimplemented feature.
Uy
E +7
5 BROIIEEJEWEY at 726.50342
5 BRIEF5Ub
Has chosen another of the original list, then looked at classified display
without choosing any.
E +19
5 BRIEF5Um
E +21
5 MORE
E +3
0 NMPW = 0 NGW = 3 NflUI = 12'More' finds 12 books, three of them probably matching quite w e l l . The system
only reports 'Found some more books, 12 altogether*.
Ud
5 BRIEFS
U6E +34
5 FULLUy
U n
Chooses the s i x t h book from the query expansion search. It i s 'The High
Victorian movement inarchitecture'.
E +5
5 BRIEFSUp
E +6
5 RESTARTUq
E +4
C finishT 880525100915
E * * Finish - duration 196 secCs3
T h i s was a b r i s k a n d d e c i s i v e s e a r c h e r who seems t o f a v o u r s h o r t r e a d i n g
l i s t s . He c o m p l e t e d n i n e o f t h e t e n q u e s t i o n s d u r i n g h i s t w o s e s s i o n s , a
f e a t e q u a l l e d by o n l y one o t h e r s u b j e c t and e x c e e d e d by n o n e .

-98-

H pD p> 6B m cd ± ><E3

n 5 5 E 5 5 D R 5 'I N S T R U C T I O N S

InstructionsFirst of all, thank you very much for agreeing to take part inthis study.
THE BACKGROUND
Over the past month we have been running experiments to assess
the efficiency of a new version of Okapi, the online publicaccess catalogue. The experiments have consisted of a number ofstudents performing subject searches to produce Lists of books toassist in answering hypothetical essay questions 'which we gavethem.
We now need the help of people with some reference experience tolook through these lists of references and decide to what extent
each item would be useful to the person writing the essay.
WHAT TO DO
Ycu have been given one cr more lists of bibliographic
references. Each list represents the combined results of all thesearches conducted on the online catalogue on one topic. The
appropriate essay question is printed at the head of each list.
Please read the question carefully.
Then, far each reference:a!) Use your experience as 3 Librarian to make a quick
judgment about the usefulness Cor otherwise) of eachbook to a student writing the essay.

b) On the scale printed immediately underneath eachreference, tick the option which best describes how you
feel about the book Cwe have found that most people takeabout 10-20 seconds per reference}.

When you have finished please return your reference lists to:

Rachel de Vere, Polytechnic of Central London, 74 Gt Portland 5t, W1.

Rpp 6(Assessors'

instructions

Example r e c o r d sent t o assessors
Page 1
Outline the mechanisms and principles of plant nutrition
CD[16515713
RUTHORCS): SMITH H
TITLEC5D: Regulation of enzyme synthesis and activity in higher plants :proceedings of the Phytochemical Society symposium, Oxford, April
1976.
Rnnual proceedings of the Phytochemical Society. Symposia series.
PUBLICATION: Rcademic Press, 1977.
5UBJECTC53: Plant proteins - Congresses. Enzymes - Congresses. Plants.
Biochemistry.
Shelved at : 581.1925 REG— > Very useful/quite useful/slightly useful/not useful/other--please specify-100-

H p FD 6=? m c:l ± ><C L R 5 5This i s
5.6~7

BROWSING

RESULTSan e x p a n d e d v e r s i o n o f

Question i s

the t o p i c

Search statement

referencei s whatist h e d a t a summarized i n

CRppendix

the userNumber c h o s e n i s t h e number o ffrom the c l a s s i f i e d d i s p l a y .
Rssessment

some o f

typed

43.

in.

r e c o r d s chosen r e l e v a n t

the experimenter'so p i n i o n of

the d i s p l a y

Number chosen Dewey number
total
1st scrby t h e

user

Csee

5.5.23.

Rssessment

Question

5earch statementcomp1.500Programming languages cobol 00Human computers [ i e human-

0
0
0
D

362.172..
remote610.2854.. good
CO1.6424
scattered001.64scatteredcomputer i n t e r a c t i o n ]computer data i n telephone
network

2

0621.38scattered

0

0
0
0
0
0
G
I

025.002..
001.54044
621.38001.64001.64404
658.40388
001.64

remoteremotescatteredscatteredscatteredgoodscattered00001.54610.285..
001.6425
703.22

comp1.3
comp1.4
comp2.5

Computers i n medicine

1

0
G
0

comp2.2

computer architectures

ccmp2.4

comp1.2i0Management information systemO
1
Computer a r c h i t e c t u r e andinterfaces

ccmp1.5
compl. 1
art1.5computers i n medicine
cpm operating system
women art

0

0
0
G
0

G
1
G
0
0
0

scatteredgoodscatteredpossiblepossible759
possible703.22possible753
possible704.042763.4394.. possible

0
1

0
1

940.5488
303.375

0
2
0
0

0
0
0

769.4994.. possible
303.375.. possible
309.097..
remote338.891.. possible

40
3

153.12324.63324.211
101

art1.1

art's contribution topolitical propaganda
contemporary politicalpropaganda

SGC1.1

sod.4
sad.5
sod.5

multinational companies
third world7
memoryproportional representation 0
proportional representation 6
electoral systems

I

1

101

Table

possible
good

good
good
possiblePpp 7
soc2.2

art1.2art 1.3
art1.1
art1.4CLass

G

370.1934.. possible

3
7
pop art british
0
0
1
19th century gothic
01
art politics propaganda
decorative arts britain 1800 1
1

370.1901
370.19700.941709.046720.942726.50942
769.499..
747.05747.213745.03

0
1
G
G
1
17
0
0
1
0
2

3
G
0
G
1
0
1
1
1
1
G
1
G
G
1
7
G
0
1
G
1

943.085
943.085
943.085616.85232
153.12324.630..
324.210..
324.25324.21
401.9

possiblepossiblepossiblepossiblegoodpossiblepossiblepossiblepossiblepossible

1
0
1
2

1
G
0
2

155.422155.422155.418324.53

possiblepossiblepossible
good

4
2
G
2
0

4
2
G
2
G

153.12
153.12616.895
306.46338.888..

good
good
remotepossiblepossible

0
0

0
G
1
0
G
G
G
0
1
2
G

338.880..
338.880..
155.422155.422330.126G
3
0001.6425
001.6425
001.64574.875
541.3724
574.875
581.875
574.876620.0044

possiblepossiblepossiblepossiblepossibleremotescatteredscatteredscatteredgoodremotegood
possiblegoodremote12

12541.388632.96possible
good

0
1

0
1

632.96632

goodpossible1
women art
treaty of Versailles 1913

sod.4

human memory

sod.5

electoral reform

soc2.1

influence of the mother on
the childsod.4

electoral system proportional representation
british government
memory in humans

sod.3

illness social factors

sod.1

third world multinational
companies economic development

soc2.1
soc2.3
comp1.1
comp1.2
life1.3

eng2.5resuLts0sociological perspectives
education

art1.5
sod.2

sod.5

browsingrole of mother10
welfare economics
0G
cp/m file00
operating systems
1computer architecturesemi [permeable] membranes 5
G
ion transport
1
0
7
engineering and radioactive 0

•

good
good
remotegood
goodpossiblepossiblepossiblepossible
good

759

332

isotopes
life2.1

biological control andplants and disease
plants and disease-102-

Rpp 7
comp2.4

Classbrowsing

management information

results
658.403possiblesystem design

soc2.1

comp2.2

comp2.5
sod. 5

0
0
0
0
2

0
0
0
0
1

001.64658.403658.403658.403
155.422

remotepossiblepossiblepossiblepossible

3
3
3
0

1
3
3
0

155.422155.418
155.418001.64

possiblepossiblepossiblescattered000computer architecture0
computer architecture0
Ian1
proportional representation 2

0
0
0
0
0
1
2

001.64001.64044
001.64001.64001.64001.64404
324.63

scatteredscatteredscatteredscatteredscatteredpossible
good

1
0
1
2

324.630..
354.410..
324.63324.21

possibleremotegood
possiblechild developement [sic]and mothercomputers parallel architecturebritain1
electoral reform
0
british government
proportional representation 1

2

-103-

R f 3 £D E2 m ClI ± ><E X A M P L E

1Extracts

S
5 E R R C H E 5

f r o m a L i v e s e a r c h on t h e q u e r y e x p a n s i o nsystemBooks 1 to 9 of 19

LI5T OF BOOKS
Search: "slump 1932"

Author

No.

Title

1
2
3
4
5
6
7
B
9

Slump c i t y : the p o l i t i c s of mass unemp. . FRIEND R
The second stump : a Marxist analysis .. MRNDEL E
The o r i g i n and nature of the Great Slum. . FERRQN P
The slump.
STEVENSON J
The slump : society and p o l i tics during. . STEVENSON J
How to survive the slump : a guide to .. BGNNOCK G
P o l i t i c i a n s and the slump : the Labour . . SKIDELSKY R
Money i n boom and slump : an empirical . . WRLTER5 P P
P o l i t i c i a n s and the slump : the Labour . . SKIDELSKY R

ClassmarkDate

331.137341 1981
330.3047 1980
338.54034 1979
1979
341.0831977
341.083
330.3410.. 19751970
941.083
332.4342 19701967
941.083

Type i t s number to see i f a book i s relevant 3
or type Down ( n e x t ) , Restart/new search/quit

Numbers 3 , 4 and 5 , and p o s s i b l y o t h e r s , l o o k r e l e v a n t .
books a r e n o t r e l e v a n t , c o n t a i n i n g o n l y " 1 9 3 2 " .

FULL DI5PLRY
Search: "slump 1332"

The

other

Book 3 of 13

RUTH0RCS): FEflRDN P
TITLEC5): The origin and nature of the Great Slump, 1323-32.
Studies in economic and social history.
PUBLICATION: Macmillan, June 1373.
5UBJECTC5): Depressions - 1323 - Europe. Depressions - 1323 - United States.
Economic depressions, 1323-1338. Related to economic depressions
in United States. Related to economic depressions in Europe.
Not in this branch
No. of copies in other PCL libraries : RHS C23
Shelved at : 338.54034 FER 338.540373
Is this at all the sort of book you are looking for? Cy/n3 YES

This brings in useful terms such as "depression","1929 1 and "1329-32", as well as unhelpful o n e s .

-104-"economic", Uni

S y s t e m has g o n e b a c k
f o u n d two b o o k s .

Rpp 8

Examplesearches

to b r i e f

display,a n d u s e r has c h o s e n

LIST OF BOOKS s i m i l a r to the one yau chase(Original search: "slump 1332"3•More",

Books 1 to 2 of 2

No. Title

RuthorClassmark

1
2

GRLBRRITH J K
GRLBRRITH J K

338.540973 1980
338.540973 1978

The Great Crash, 1329.
The great crash, 1929.
* * END OF LI5T * *DateType i t s number to see i f a book i s relevant 1
Type Back to r e t u r n to the books you o r i g i n a l l y foundor type Restart/new search/quit

FULL DISPLAY of books similar to the one you chose(Original search: "slump 1932')

Book 1 of 2

FUTHORCS): GRLBRRITH J K
TITLECS3: The Great Crash, 1929.
PUBLICATION: 50th anniversary ed. Deutsch, 1980.
SUBJECT(S3: Depressions - 1929 - United States. Economic depressions.

Not in this branch
No. of copies in other PCL libraries : RHS (23Shelved at : 338.540973 GRL
Is this at all the sort of book you are looking for? Cy/n) YES

We n o w have the term "crash" as w e l l .

-105-

which

Rpp 8User has gone b a c k

Exampleto the o r i g i n a lsearches

list

and c h o s e n a n o t h e r

FULL DISPLAY
Search: 'slump 1332"

record.

Book 5 of 13

RUTHORCSh STEVENSON J ; CCDOK CD
TITLEC5D: The slump : society and p o l i t i c s during the depression.
PUBLICATION: Cape, 1377.
SUBJECTS): Great B r i t a i n - History - George V, 1310-1336. Great B r i t a i n History - Edward V I I I , 1336. Great B r i t a i n - History - George V I ,1336-1352. Great B r i t a i n , 1330-1333.

No. of copies i n t h i s branch : C13
No. of copies i n other PCL l i b r a r i e s : MRd CD RHS (4)Shelved at : 341.083 5TE
Is t h i s at alL the sort of book you are looking for? Cy/n) YE5

T h i s has b r o u g h t i n some s i n g u l a r l y u s e l e s s LC s u b j e c t
a l m o s t u s e l e s s PRECIS v e r b a l f e a t u r e h e a d i n g .
User does " M o r e " a g a i n , w h i c h t h i s t i m e f i n d s
t h r e e w h i c h t h e u s e r h a s now s e l e c t e d } .

five

books

a n d an

Cbased on t h eBooks 1 to 5 of 5

LIST DF BOOKS s i m i l a r to the ones you chose(Original search: 'slump 1332')

No. T i t l eheadings,

RuthorClassmarkDate

338.540373 1378

1

The g r e a t c r a s h , 1323.

GRLBRRITH J K

2The g r e a t c r a s h , 1323.

GRLBRRITH J K

338.540373 1375

3
4

Rmerica's greatest depression, 1323-1341. CHRNDLER L V
The world economic and social c r i s i s : . . CRSTRO F

338.540373 1370

5

We're i n t h e money : d e p r e s s i o n R m e r i c a . . BERGMFM R

3 3 8 . 5 4 0 3 . . 1383
731.430373 1372

* * END OF LIST * *Type i t s number t o see i f a book i s r e l e v a n t
Type Back t o r e t u r n t o t h e books you o r i g i n a l l y

found

or type Restart/new search/quit

We now have two more e d i t i o n s of G a l b r a i t h . One was not p r e v i o u s l y
r e t r i e v e d because i t had a d i f f e r e n t s u b j e c t h e a d i n g . The book by
Chandler i s p r o b a b l y r e l e v a n t , and the user s e l e c t s i t . The C a s t r o i s a
f a l s e d r o p , and t h e l a s t book i s about making f i l m s i n t h e d e p r e s s i o n .

106-Rpp 8

Examplesearches

The user now has what is probably a reasonable reading list, given the
Limitations of the database.

LIST OF BOOKS YOU HRVE CHOSEN[Original search: 'slump 1932')

No. Title

Books 1 to 4 of 4

Ruthor

1** The origin and nature of the Great Slum.. FERRON P
2** The Great Crash, 1S29.
GRLBRRITH J K
3** The slump : society and politics during.. STEVENSON J
4** America's greatest depression, 1923-1941. CHRNBLER L V** END OF LIST **CLassmark Date
338.54094 1979
338.540973 1980
941.0831977
338.540973 1970

Type its number to see further details of one book
Type Back to go back to where you were before viewing the 'chosen' Listor type Restart/new search/quit QUIT

5ome books which were not retrieved are yet another edition of the
Galbraith with no subject headings at all, and Kindleberger's "The worldin depression, 1329-1939". The latter would be retrieved if a somewhat
lower cutoff were used.
In fact there are a number of possibly relevant books. Most of them canbe retrieved by sifting through the 87 books indexed under "depression"or "slump", but many Library databases would be too large for this to befeasible.

2

Notes on a search of the "full" system

The subject, a student of social sciences at a London college, wastackling the question "How do sociological perspectives throw light on
the role of education in society?". She entered "perspective role ofeducation in society". This finds one book matching "well", 135 books
altogether. The first book is "The sociology of education: introductory
analytical perspectives", with a subject heading "Society - role ofeducation". CIt is worth noting that this is the only book which
would have been found by a system which combined words with an implicit
END.) The "noisy" words "role" and "perspectives" are in some ways
helpful here: a search for "education in society" yields many false
drops, with the above mentioned book at about position 100 in thesequence, because it is fairly old and items of equal weight display inreverse publication date order.
Dewey classifies "Social aspects of education" at 370.19, and the firstbook retrieved is classified at 370.123 ("Relations between school andsociety"3. It is interesting that only eight in the first 100 books
retrieved by this search are at 370.19. Some clearly relevant books areclassified at 373.41 (Secondary education in the United Kingdom). The

107-

Flpp 8

Examplesearches

user chose the first book, and then Looked at books "shelved near".
Several on the first screen were clearly relevant, and the subject chose
two of them. These brought in the terms "sociology", "sociological" and"educational". Unfortunately, because of the date of publication
sequence, they were as old as the initially chosen book. To see more
recent works it would have been necessary to go back several screens of
370.193. To see books, equally likely to be relevant, at the moregeneral number 370.19, one would need to go back several more screens.
This would have been a more serious problem in a bigger library, buteven so, the user, understandably, selected from what was before her
eyes. This raises the question of whether it is better to use random
ordering in this type of browsing display, or to start the display atthe beginning of the sequence for the selected class number. The
disadvantage of the latter is that it is impossible to show the "pivot"
item in context.
Having chosen two books classified near the one originally chosen, thesubject then requested "More". This gives a List of 48 books, mostly at
370.193 or 370.19, with one book at 373.41. She chose number five fromthe first screen, "The sociology of educational ideas" (indexed by only
one of the terms of the original search), and requested books classified
near. She chose three books from the first screen and the succeeding
one, then went down four screens (she was now back at 370.193) where shechose "Rn introduction to the sociology of education" and then again
requested "More". This gave a list of 95 books, mostly at 370.19 and
370.1. Many of them were, as would be expected, included in the previous
query expansion List Cof which the subject had only looked at the firstscreenful and selected one). Near the bottom of this List, and not seenby the user, there were two alarming faLse drops ("What's Left for
immunology" and "The management of climatic resources"). These were
unfortunately indexed under "inaugural", "delivered" and "February", aswas Banks on "Sociology and education", one of the books previouslychosen. Qn the whole, though, "noise11 words have remarkably Little
effect on the results of query expansion searches.
Without Looking at the rest of the List, the user chose the first record(Morrish on "Sociology of education"), then looked at records shelved
nearby C370.13 again). Before going beyond the first screen she checked
on what she had selected using the "View" option - nine records, allwith "sociology" and "education" in the title. 5he then went rapidly
forward no less than 48 screens, spending between three and eight
seconds looking at each screen. The whole session took about 11 minutes,
with nine records chosen, all of which were judged relevant by theassessors.
The 14 subjects who searched for this topic chose a total of 67 distinct
titles, and it is not difficult to find more than this. One subject,apparently short of time, chose no records. The other 13 chose an
average of 12 books each, with a mean assessed precisian of 87% andabout 10 brief records seen per choice. The comparative figures for all
topics were 6.9 books chosen, mean precision 65% and 9.8 brief records
per choice. This is one of the three (out of 50) topics which we rated
as easy. Rny of the six combinations of a term from "society","sociology", "sociological" with "education" or "educational" retrieves
a List from which one can select some relevant books. These two-term
combinations find between 15 and 130 records. Rs we have seen, theaddition of a few woolly qualifiers may not be harmful. It might seemthat expansion facilities are not helpful for topics like this one.
However, the Lists obtained by expanding from one or two good records,-108-

Rpp 8

Examplesearchesor even the shelf list, are usually very much more "concentrated" thanthe original Lists. Rn example is the list obtained by the first
expansion in the session described above. It retrieved 48 records, ofwhich 16 of the first 18 Ctwo screens] are worth considering. In
contrast, the original sequence of records retrieved by "Perspectiverole of education in society" contains only four possibly-relevant itemsin the first 18. It is easy to obtain these more concentrated Lists, so
it is likely that the use of query expansion often reduces the effort
involved in even an "easy" search Like this one.
It is difficult to see what additional information would help users tochoose between one title and another. Dur subjects were of course
acting more as search intermediaries than as end users. Perhaps anexperienced intermediary wouLd have tried to choose a selection of
titles which Look as though they may treat the subject from different
viewpoints - one might choose, for example, texts such as "Schooling andcapitalism", "Education and the state", "Family, work and education" aswell as some standard-looking books on educational sociology. In
contrast, users Looking for books to satisfy barely-verbaLized personalneeds will be influenced in their choices by subjective factors such as
knowledge of the author, previous acquaintance, and other, lesstangible, aspects, none of which is very useful as feedback information
for the system.

-109-