Chapter 3

Transaction log analysis
3.1

Introduction

3.1.1

Source of the data

This chapter presents, mainly in tabular form, the results of statistical analysis of the transaction logs gathered from use of the Okapi system describedin Appendix A. The logs record sessions on the City University library catalogue and on a section of the INSPEC database from May 1990 to the endof June 1991. They are treated as three separate subsets:• CAT1: library users identified by their library card numbers• CAT2: library users who chose not to identify themselves• INSP: registered INSPEC users
The logs forming CAT2 were gathered from a single terminal placedin the university library among the terminals to the university's integrated
library system CLSI. CAT1 comprises logs of searches at the library terminal
conducted by users who identified themselves and also logs of catalogue
searches by people who had registered to use the system from anywhereon the campus network. These users also had access to INSPEC. It is notknown how many of the searches in CAT1 were carried out in the libraryand how many from other terminals. The INSPEC searches were all doneby registered users from terminals somewhere on the campus network (otherthan the library).
The sets comprise almost all the logs generated between May 1990 and
end June 1991. Logs of searches performed by the experimenters have been
removed as far as possible, although a few may remain in CAT2.
24

3.1.

INTRODUCTION

3.1.2

S e s s i o n s and searches

25

For the purposes of machine analysis sessions and searches are defined asfollows.
• A session or user session starts when a user invokes a search input
screen (Figure A.4) and finishes when the QUIT key is pressed or thesystem times out or becomes disconnected.
• A search starts when the system displays any of the search input
screens (such as those shown in Figures A.4 and A.20) and finishes
when the session finishes or when the user requests a new or edited
search.
— A search is aborted if the user pressed the QUIT key or allowed
the system to time out or chose the "new" or "edit" options before
the system displayed a "result" (Figure A.6) for the search. This
also covers the case where none of the terms in the search was
found during the lookup and the user chose not to replace them
(Figure A.7) so that the search was effectively empty.
- A search failed if it was not aborted but retrieved no records.
The above definitions of sessions and searches are artificial, and are used
simply because they enable sessions and searches to be algorithmically identified. Machine analysis has the advantage of objectivity, but sessions as
defined are often far from being a natural unit of search activity. A moremeaningful definition would be something along the lines of "a session is asequence of searches by a single user for items on a single topic or closely related topics". But looking at log shows that it is not at all unusual for a user
to do one or more searches on a subject, then some more on another subject
which is apparently not closely related, followed by reversion to the original
topic. Nor is it possible in general to decide on the degree of relationshipbetween two searches, or even what we mean by "closely related".
Recognition of session boundaries is another problem. Although Okapi
users had to go through a kind of login procedure it was not unusual for auser to logout and reconnect almost immediately, rather than use the "newsearch" or "edit search" commands. Probably we should have amalgamated"sessions" known to be by the same user when there was no more than a
few minutes time lapse between them.

3.1.3

T r a n s a c t i o n logs

Virtually all sessions were automatically logged (it is of course possible to run
the system without logging). Some details of log layout changed during thecourse of the experiments, but the general style and layout have remained

26

CHAPTER

3.

TRANSACTIONLOG

ANALYSIS

unchanged since 1984 [7]. The logs record all user keystrokes, all record
displays more or less as the user saw them, and enough coded informationa b o u t the state of the system to enable the interaction to be completely
reconstructed. An annotated example is shown in Appendix B. There is
one log for each user session.
T h e logs are designed rather more for human readability t h a n for automatic analysis and it is quite difficult to write reliable analysis programs.
These programs are likely to fail on logs which are defective, usually becauseof a d a t a communication breakdown or because the host machine crashed.
T h e programs used for the analysis in this chapter were written in the unix
awk language, and were verified by hand on a reasonable sample of logs.
Analysis took place in two stages. The first stage digested a log into a few
s u m m a r y lines. Figure 3.1 shows an example.
Figure 3.1: Transaction log summary

User: 098920007
F i l e : userlog.0002.Z
Database: c i t y
S t a r t _ t i m e : 900903142242
Search 1 ( f i r s t )accounting management control<RET>
Lookup: account (1622), management (6036), control (2841),
Terms: 3

R e s u l t o r i g : OK 535 60 26
R e s u l t more: OK 157 157 0
Orig: Bfs 90 Fls 3 Chos 2 Bandl 2 Band2 0 Band3 0 Mores 1 Clss 0
Mores: Bis 54 Fls 1 Chos 1 Bandl 0 Band2 0 Band3 1
End of s e s s i o n . Time 395

This user typed in the search "accounting management control". The system
stemmed "accounting" to "account". The next line gives the number of postingsfor each of the three terms. The system reported "26 books match your search well
— (535 books found altogether)" (cf. Figure A.6). The user looked at the first
90 records in brief (10 screens), three in full, and chose two as relevant. Both thechosen records were among the first 26, those which matched the search well. The
user did one iteration of query expansion, displaying 54 brief records and looking
at and choosing one, which was not from the first two screens of records.
These summaries identify time, user and database, search statements,
term lookup results and overall search result. For each search the summary
shows the number of brief and full records which the user saw, followed by
the number chosen relevant. There is some information about the sourceof the records chosen as relevant. In the case of an "original" search (oneentered by the user), chosen records are divided into those which containall the terms of the search, those which match the search fairly well but not
perfectly and those which do not match the search closely. In the case of a

3.2. SESSIONS AND

SEARCHES

27

query expansion search ("More") records are divided into those chosen fromthe first screen, the second screen and subsequent screens.

3.2

Sessions and searches

Table 3.1 gives summary information about the numbers of users, sessions
and searches in each of the three datasets.
Table 3.1: Sessions and searches
CAT2

CAT1
Users
Sessions

705
2212

Searches- aborted- failed- no recs disp- records disp

4952
462
201
458
3830
2.03

Mean srches/session
Searches/session

N/K(9.3%)(4.1%)(9.2%)(77.3%)

1401
2574
211
117
204
2042

INSP
86
562

(8.2%)(4.5%)(7.9%)(79.3%)

1.69

1763
126
36
160
1441
2.91

(7.1%)(2.0%)(9.1%)(81.7%)

is calculated on non-aborted searches only.

No display means that the search found some records but the user did not choose to see
any of them.

There are some noticeable differences between the three sets. In particular, the number of searches per session differs quite markedly (significantly
at 0.1% on a t-test) between each pair of datasets, with INSPEC having
the highest number and unidentified catalogue users (CAT2) the lowest. It
will be seen later that CAT2 searches also had a much lower usage of query
expansion than the other two sets (Tables 3.7 and 3.8). This is discussed
further 3.6.

3.2.1

Aborted searches

The proportion of aborted searches looks very high, but this is mainly due
to our definition. In up to three-quarters of aborted searches the user had
not in fact entered a search at all but had pressed the QUIT key or left
the system to time out from a search input screen. In most other cases
the aborted search would count as a failed search on systems which do notcompel the user to negotiate terms which it cannot find (Figure A.7).

28

CHAPTER 3. TRANSACTION

3.2.2

LOG

ANALYSIS

Failed s e a r c h e s

The proportion of (non-aborted) searches which failed to retrieve any records
is very low indeed (3.9% overall), and significantly lower for INSPEC thanfor the catalogue sets. Some keyword-type online catalogues have returned
failure rates of up to 40%. Some of this difference is accounted for by ourdefinitions of failed and aborted, but in the main it is due to• Okapi's "best match" search• stemming and automatic cross-referencing.
These features were not under investigation in the present project — see[16] for an account of the relevant experiments. However, Table 3.2 suggeststhat the overall failure rate might well have been over 30% without thesedevices.
Table 3.2: Distribution of number of records retrieved
Search result
Exact match:"high" (more than 180 recs)"mid" (10-180 records)"low" (1-9 records)
Subtotal
Fairly good match
Poor match
Total searches

CAT1

CAT2

INSP

12.3
30.9
34.2
77.4
8.6
14.1
4288

13.6
30.7
32.7
77.0
8.2
14.8
2246

16.7
40.1
28.6
85.4
9.0
5.6
1601

The figures are percentages of the number of searches which retrieved some records.
Exact match means records which would have been retrieved by a boolean AND — they
contain all the terms in the search.
Fairly good match generally means records attaining about 2/3 of the weight they would
have if they contained all the terms of the search.
Poor match generally means records attaining between 1/2 and 2/3 of the weight theywould have had if they contained all the terms of the search.

3,3

Records retrieved, displayed and chosen

From here on failed and aborted searches are excluded from the datasets.

3.3.1

N u m b e r s of r e c o r d s r e t r i e v e dTable 3.2 shows the distribution of the number of records retrieved for eachof the three datasets. The two catalogue datasets give very similar figures,
but INSPEC searches have a marked tendency to retrieve more records.

3.3. RECORDS
3.3.2

RETRIEVED,DISPLAYEDAND CHOSEN

29

Effect of number of records r e p o r t e d on record display

Overall, in about 10% of "successful" searches (i.e. searches which retrieved
some records), the user chose not to look at any records, even in a brief
display (this figure is derived from Table 3.1). The Okapi system did not
automatically go into record display on retrieval as it was felt that the user
should have a picture of the overall results of the search (Figure A.6). One
would expect some searchers to be put off if the system reports that a large
number of records match. 1
Table 3.3 shows the proportion of searches in which the user did not display any records. It shows that a "high" result is moderately discouraging.
Table 3.3: Percent of non-failed searches in which no records were displayed

Search result"high""mid""low"
Fairly good match
Poor match
Overall

CAT1(4288 srches)
37.6
9.1
3.1
4.4
12.8
10.7

CAT2
(2246 srches)
24.6
9.2
3.3
1.1
12.0
9.1

INSP
(1601 srches)
35.0
5.0
3.3
3.5
15.7
10.0

The figures are percentages of the number of searches in each category.
(The actual numbers of searches are implicit in Table 3.2.)

There is little difference between the datasets. The system itself does notdiscourage users from looking at what might be thought excessive results.
The Okapi system used over the local ethernet usually gave very rapid updating of screen displays and it did not seem tedious to the experimenters
to look at a great many screens of brief records. It appears that experienced
users are more likely than inexperienced to display records from large sets:
for the CAT1 dataset 56.1% of "high" searches by first time users led to adisplay, but this increased to 70% of ''high" searches by users with eight ormore sessions (the overall CAT1 figure is 62.1%).
Further analysis shows that the proportion of searches in which norecords are displayed increases smoothly to reach about 43% when the system shows that more than 1000 records match the search.
!

O n the other hand it was not unusual to see a case where a user seemed content tobrowse 40 or more screens of brief records. In particular this was observed several times
in catalogue searches for "accounting" or "accountancy"; the user gave the impression oftrying to get an overall picture of the library's holdings.

30

CHAPTER 3. TRANSACTIONLOG

ANALYSIS

Action following a search where the user displayed no records
A limited further investigation of these cases was carried out by visual examination of the log summaries in 118 cases taken successively from theCAT1 set. They were classified as shown in Table 3.4.
Table 3.4: Action following search with no records displayed (CAT1 set)
ActionNarrow
Abandon(l)Abandon(2)Different search
Rephrase
Broaden
Repeat
Total

Cases
61
(51.7%)
17(14.4%)
13(11.0%)
11(9.3%)
8(6.8%)
4(3.4%)
4(3.4%)
118 (100.0%)

Examples"PhD" - uHow to get a PhD"
"civil engineering" (579 recs)
V \ "ew", "ESC 0 P"
"endocrine disorders" —• "diabetes""aids" — "hiv aids""pediatric larynx" —• "larynx"

Abandon(l) means abandon following a "sensible" (but usually highly posted) search.
Abandon(2) means abandon following accidental or foolish input. This was sometimes
caused by program bugs or data communication problems.

3.3.3

Full records

From now on the datasets will be limited to those searches in which theuser actually looked at some brief records. This eliminates about 10% of
the searches (Table 3.3). Table 3.5 gives the distributions of the number offull records displayed, and the number of records chosen relevant, for each of
the reduced datasets. In about a third of these searches no full records weredisplayed. Table 3.6 gives the corresponding figures restricted to searches inwhich at least one full record was seen.
As Table 3.5 shows, in about a third of the searches no records at all
were displayed in full. In the case of the catalogue database the brief records
did not show the location of the item (Figure A.8) and for INSPEC they
did not show the source. Hence searches in which no full records are displayed cannot be called successful if the object of a bibliographic search is
to locate bibliographic items. However, examination of the sessions in whichthese searches occur, and sometimes of other neighbouring sessions by thesame user, reveals that in a large proportion of cases the search is either
preceded or followed by an identical or similar search in which full records
are displayed.
INSPEC users have a marked tendency to display more full records thancatalogue users. Part of this difference may accounted for by the fact thatan appreciable proportion of catalogue searches, but few INSPEC searches,3.3. RECORDS RETRIEVED,DISPLAYED AND CHOSEN

31

Table 3.5: Full records displayed and chosen: searches in which some brief
records were displayed

Records
[None
1

1 2-3
4-7
8Total searches
Mean records per search

CAT1
Disp Chos
34.2
49.7
28.0
23.7
16.9
13.2
11.9
8.7
9.0
5.0
3830
2.9
1.7

CAT2
Disp Chos
33.5
50.9
28.1
25.2
18.7
13.1
11.7
6.8
8.0
3.9
2042
2.5
1.5

INSP
Disp Chos
31.7
57.7
21.2
14.1
18.7
11.9
13.5
8.3
14.9
8.0
1441
4.1
2.2

The figures are percentages of all searches in which some brief records were displayed.

Table 3.6: Full records displayed and chosen: searches in which some fullrecords were displayed

Records
None
1
2-3
4-7
8Total searches
Mean records per search

CAT1
Disp Chos
0.0
23.5
42.5
36.0
25.7
20.0
18.2
13.1
13.7
7.4
2520
4.3
2.7

CAT2
Disp Chos
0.0
26.1
42.2
37.9
28.0
19.6
17.7
10.9
13.3
6.1
1358
3.7
2.2

INSP
Disp Chos
0.0
38.1
31.1
20.6
27.5
17.5
19.9
12.2
21.5
11.5
984
5.9
3.2

The figures are percentages of all searches in which some full records were displayed.

are attempts to locate a specific work, despite the notice on the libraryterminal emphasising that the system is for subject searching only and theinformation on the search input screen (Figure A.4).

3.3.4

Relevance judgements

On displaying a record in full the user is compelled to answer a form of
relevance question ("Is this the sort of thing you are looking for?"). The
object, for the system, is to obtain relevance information for use in query
expansion. The motivation, for the user, is less obvious. There is at thispoint in the interaction some information available on request which informs
users that if they choose some books the system will look for similar ones ifthey type MORE. We have not tried to find out what proportion of users
looked at this information — certainly some did. Many users will also havelearnt that the system allowed them to review or to print out (or have

32

CHAPTER 3. TRANSACTIONLOG

ANALYSIS

emailed to them) a list of the items they had chosen (Figure A.18). The
fact that in more than a quarter of the searches in which full records weredisplayed the user chose no records at all (Table 3.6) is not in itself anindication that the user did not find any useful records.

3.4

Query expansion

One of the objects of these experiments was to evaluate the automatic query
expansion feature of the Okapi system (2.1.1). Before query expansion canbe done it is necessary for the user to provide some feedback in the formof records chosen as relevant. The interaction is shown in Figures A. 11
and A. 12. Query expansion is not available unless at least one record has
been displayed in full, and the user has given an affirmative answer to therelevance question.2
From here on searches in which no full records are displayed are excluded
from the analysis.

3.4.1

T a k e u p of q u e r y e x p a n s i o n o p t i o nTable 3.7 shows the proportion of searches in which query expansion wasused, broken down by the number of records reported as retrieved (cf Table3.2), given that at least one record had been chosen relevant.
The table shows, perhaps not surprisingly, that query expansion is much
more likely to be used in searches where few records were retrieved or where
none of the retrieved records contained all the terms of the search.
The overall figures for the takeup of query expansion are not significantly
different between CAT1 and INSPEC, but CAT2 shows a very significantly
lower takeup than the other two. We have been unable to account for thisdifference. There is further discussion of this point in 3.6.
Table 3.8 shows the proportion of searches in which query expansion wasused by the number of records already chosen relevant by the user.
Here, we find that query expansion is less likely to be used when theuser has only chosen one record. Many of these cases will be those where
the user is not seeking an exhaustive search. It is also less likely to be used
when the user has already chosen a large number of records.
Query expansion was not always offered unless two or three records had been chosen
relevant, although it was almost always available if the user knew the command. Whether
it was offered depended on the number of records which matched the search well and on
how many had been chosen.

3.4. QUERY EXPANSION

33

Table 3.7: Takeup of query expansion by search result
CAT1 CAT2
INSP(% of srches using QE)[ Search result
Exact match:"high" (more than 180 recs)"mid" (10-180 records)"low" (1-9 records)
Fairly good match
Poor match
Overall
Number of searches

20.0
26.2
34.8
50.7
53.7
33.5
1925

22.7
20.9
26.6 |
48.7
48.0
27.5
1002

22.6
30.5
40.8
44.7
71.4
35.3
609

The figures are percentages of the number of searches with the given result.

Table 3.8: Takeup of query expansion by number of records chosen
Number of recs chosen
1
2-3
4-7
8OverallNumber of searches

CAT1 CAT2
INSP(% of srches using QE)
21.8
16.3
22.8
46.2
39.6
41.1
49.7
46.0
50.5
30.0
34.5
30.1
33.5
27.5
35.3
1003
609
1928

The figures are percentages of the number of searches in which the given number of records
had been chosen relevant before query expansion.

3.4.2

R e s u l t s of q u e r y e x p a n s i o n s e a r c h e s

In some of the searches in which query expansion was used it was used morethan once, with a mean of about 1.3 expansion searches per original search(original search: search typed in by a user). To arrive at the figures in Table
3.9 all the expansion searches in an original search have been aggregated.
About 95% of query expansion searches were successful in the sense thatthey led to the system finding some records. There does not seem to be
an obvious way of measuring the "nearness" of records retrieved by query
expansion to the records used for relevance feedback. Hence the system
was given a fairly arbitrary, and low, cutoff for the minimum weight to beachieved by records in an expansion search. However, as Table 3.9 shows,in about 40% of cases no full records were displayed and in about 55% norecords were chosen relevant.
Table 3.9 gives the distributions of the number of full records displayed
and the number of records chosen from record sets retrieved by query ex-

34

CHAPTER 3. TRANSACTIONLOG

ANALYSIS

pansion. This should be compared with Table 3.5, which gives the same
distributions for the totality of records retrieved in all the searches in which
the user displayed some brief records.
Table 3.9: Records displayed and chosen from query expansion
Number of records

0
1
2-3
4-7
8Total searches
Mean records per search

CAT1
CAT2
INSP
Disp Chos Disp Chos Disp Chos](Percent of searches)
42.6 58.4 1 40.9
54.3 | 32.6
50.2
13.4 17.7
17.7
17.6
15.5 16.3
13.4 20.0
13.5
13.9
11.3 15.6
13.0 14.0
12.6
13.2
9.1 14.1
5.8 15.8
12.7
5.7 13.0
6.0
646
276
215
3.3
2.1
3.8
2.2
3.3
1.8

The proportions of searches in which no records were displayed in full,and in which no records were chosen are somewhat higher than the overall
figures shown in Table 3.5. However, the means are also higher. This suggests that query expansion searches are more likely than original searches to
produce no useful records, but that in the 40%-50% of cases where they do
produce useful records the query expansion record lists are a richer source
than the original lists.

3.4.3

Source of records displayed and chosen

Tables 3.10 and 3.11 compare searches with and without query expansion
with respect to the sources of displayed and chosen records and the proportion of records chosen. Bands 1, 2 and 3 refer to the source of chosen
records. For an original search, a record is from Band 1 if it contains all
the terms of the search, from Band 2 if it matches fairly well and Band 3otherwise. For query expansion searches Bands 1 and 2 refer to the first andsecond screens of brief records and Band 3 to any subsequent screen.

3.4.4

H o w useful was q u e r y e x p a n s i o n ?

(In this discussion the datasets are restricted to searches in which at least
one record was chosen relevant.) Query expansion made a substantial contribution to the choice of relevant records. It was used in 33.5% of CAT1
searches and in 35.3% of INSPEC searches Table 3.7. For reasons unknown,
it was less frequently used (27.5% of searches) in CAT2. Its use was more
frequent in searches where few or no records matched the original search
exactly. In CAT1 QE was responsible for 37.5% of the records chosen as3.4. QUERY EXPANSION

35

Table 3.10: Sources of displayed and chosen records: CAT1
QE not used

Brief recs
Full recs
Chosen: (total)-band 1-band 2-band 3% of bfs chosen% of fulls chosen
Searches

34.8
3.8
2.8
2.2
0.3
0.3
8.0
74.3
1270

QE usedOverallOrig
QE Tot Orig
QE
Tot(means per searc h)
30.6 58.6 89.2 33.4 19.8 53.2
4.3
3.3
7.6
3.9
1.1
5.1
3.0
1.8
4.8
2.9
0.6
3.5
2.1
0.8
2.9
2.2
0.3
2.5
0.3
0.1
0.4
0.3
0.3
0.6
0.4
0.2
0.6
0.6
0.7
1.3
9.8
3.0
5.4
8.6
3.1
6.5
70.9 53.4 63.2 73.1 53.5 68.7
637
1917

The searches are the subset of CAT1 consisting of all those where at least one record waschosen.

relevant in searches in which it was used, and for 17.1% over all searches inwhich any records were chosen (Table 3.10). The corresponding figures for
INSPEC are 31.9% and 15.4% (Table 3.11).
QE was in some sense less efficient that the user searches in finding
relevant records. In QE searches more than three times as many brief records
were scanned for each record chosen than in original (user-entered) searches(Tables 3.10 and 3.11). A portion of this difference must be put down to
the fact that by the time searchers use query expansion they have alreadychosen some records, and hence their need for more records must in general
be less than it was in the early stages of the search — they can afford to be
more choosy. There is some rather tenuous evidence for this in the fact thatin QE searches users also chose a somewhat lower proportion of displayed
records than in original searches, suggesting that they were less ready toanswer u yes" to the relevance question. 3

N o t e on r a n k i n g
Records displayed following query expansion come out in decreasing weightorder, where the weight of a record is the sum of the weights of the termsby which it has been retrieved. It is worth noting that for the CAT1 set
44% of the records chosen relevant are from the first screen (nine records),
17% from the second screen and remaining 39% from subsequent screens3

But another hypothesis might be that these people were so desperate for records thatthey exercised less discrimination in selecting brief records for display, and so were more
likely to reject the full record. Or they might just have been uncertain about what they
were looking for.

36

CHAPTER 3. TRANSACTIONLOG

ANALYSIS

Table 3.11: Sources of displayed and chosen records: INSPECQE not used

Brief recs
Full recs
Chosen: (total)-band 1-band 2-band 3% of bfs chosen% of fulls chosen
Searches

51.5
6.5
4.3
3.6
0.4
0.3
8.3
65.6
394

QE usedOverall
Orig QE
Tot OrigQE
Tot(means per search)
47.3 80.3 127.6 1 50.0 28.3 78.3
7.6
3.8
11.4
6.9
1.3
8.2
4.7
2.2
6.9
4.4
0.8
5.2
3.8
1.0
4.8
3.7
0.3
4.0
0.5
0.3
0.8
0.4
0.1
0.6
0.4
0.9
1.3 1 0.3
0.3
0.6
10.0
2.7
5.4
8.9
2.7
6.6
62.2 56.9
60.4 64.3 56.9 63.1
1214
608

The searches are the subset of INSP consisting of all searches where at least one record
was chosen.

(Table 3.10). The corresponding figures for INSPEC are 45%, 14% and 41%
respectively.

3.5

System use patterns

Tables 3.12 and 3.13 give the distribution of the number of sessions by identified users over the four month period Jan-Apr 1991. Table 3.12 includes
everyone who searched the given databases during the period and Table
3.13 limits the figures to those who had searched the databases at least once
prior to Jan 1 1991 and therefore excludes new users who arrived during theperiod under observation. The latter table also gives the number of users
who did not use the system again during the period. The tables show thatmost catalogue users only used the system once or twice, but there was asmall number of people who used it regularly, or at least intensively. Among
INSPEC users, who had all been to the trouble to register for use of the system, there are fewer occasional users, and a nucleus of regular users who
provided the data for the work described in Chapter 5.

3.6

Differences between classes of users

In previous sections some significant differences show up between catalogue
users who chose to remain anonymous and other users. In particular, theformer made fewer searches per session and were less likely to use query
expansion. One would expect to find differences between people using a

3.7. BEST MATCH SEARCHING

37

Table 3.12: System use Jan-Apr 1991: all identified users
Catalogue
INSPEC
Sessions Users Sessions Users
14
1
137 1
18
2-3
68 2-34-7
34 4-7
98-15
12 8-155
166 163717
257 220
81 1

Table 3.13: System use Jan-Apr 1991: people who had used system before
Jan 1 1991
Catalogue
INSPEC
Sessions Users Sessions Users
32
0
400 0
2
1
33 1
11
2-3
19 2-34-7
16 4-78
8-15
7 8-154
165 163373
490 17660

system from an open terminal and those using it from their own desks. In
the rather small open access library of City University it is easy to get up
from the terminal and go and look at the shelves, and this alone might
be expected to lead to shorter and less exhaustive sessions. However, acomparison between catalogue use by identified users at the terminal in thelibrary and at their own terminals showed no significant difference. One
would also expect considerable differences between INSPEC and catalogue
searching. However, the latter is less pronounced than the difference between
identified and unidentified users at the library terminal. There does seemto be some "real" difference between anonymous and identified catalogue
users, whether the latter are using the system from their own terminals orin the library.

3.7

Best match searching

These experiments were not intended to evaluate best match, ranked output
searching. This was done several years ago in the experiments described in

38

CHAPTER 3. TRANSACTIONLOG

ANALYSIS[16]. It is, however, an essential feature in any system which has an automatic query expansion facility. AQE can only be done on such a system 4 .
Table 3.2 shows that of searches which retrieved some records, 23% ofcatalogue searches and 14.6% of INSPEC searches would have failed completely on a system which required all the search terms to be present. Further, Table 3.10 shows that about 24% of the total records chosen fromoriginal searches (i.e. prior to query expansion) were retrieved by only someof the terms in the search (CAT1 set). The corresponding figure for INSPEC, where searches tended to retrieve more records (Table 3.2), is 16%.
The proportions when highly posted searches are excluded are very muchhigher.

4

In theory, although not always in practice, "best match" can be provided in a frontend to a conventional boolean system — see, for example, [12]