Chapter 5
Results

5.1

Experiment 1—holistic approach to searchingin the library

The searching activity of library users was observed in its entirety in two d a t a gathering
exercises: in the first the searches were initiated at the online catalogue and in thesecond the searches were initiated either at the catalogue or at the shelves.
Although microfiche catalogues were still available in parallel with the online
catalogue, none of the subjects used them. The OPAC1 sample was collected in thethird month after the installation of the online catalogue in the middle of the summer
term, whereas the second sample was collected in the middle of the first term fivemonths later. The great majority of searchers had used the OPAC fewer than fivetimes and a few between five and ten times in the first sample and most between fiveand ten times in the second sample. All had a very positive attitude towards theonline catalogue with most volunteering how much they preferred it to the microfiche
catalogue.
The observation of the entire search process revealed that users looked for bookmaterials in a variety of ways. They differed not only in the use or non-use of theonline catalogue, but also in the number of steps they followed and the sequence or
combination of those steps.
T h e analysis of the trials concentrated on:• the identification of types of searches and their relation to search strategy, i.e.
which search option (subject headings, keyword, author, title) searchers used toaccess the catalogue for what type of search• the search formulation in terms of expressed topic in relation to the initial accessto the catalogue• the search outcome, i.e. success and failure both at the catalogue and at the
shelves as well as the information extracted from the catalogue, (eg. specificreference or a class number(s) only).
35

5.1.1

Searching activity

Table 5.1 categorizes the different approaches in the use of the online catalogue andthe shelves that were observed, and Tables 5.2 and 5.3 map out the initial approachand progress of all the searches in the 0PAC1 and OPAC2 samples respectively.

Table 5.1: Steps in searching the online catalogue and the shelves, grouped by category
Step
AA
B
C
DD

Description
Online catalogue, author/title
specific item search
Shelves, specific item search
Shelves, subject search
Online catalogue, subject search

Table 5.2: Analysis of searches in the online catalogue followed at the shelves, showing
the order of steps—oPACI SAMPLE.
No . of searches

1 Steps used in search, in o r d e r
AA
fAA
AA
tAA
fAA
tAA
AA
fAA
tAA
fAA

B
B

C
C
DD
DD
DD
DD
DD
JDD
DD
DD
JDD
DD
DD
DD
DD
DD

AA
AA
AA
AA

DD

B
B
B

C
C

B
B
B
B

c
c B
c
B
c
B c

AA
AA DD AA
Total number of searches

24
3
20
7
4
3
1
1
1
1
4
3
8
2
11
2
3
1
1
100

fSearches initiated as specific item searches and then developing into subject searches.
^Apparent subject searches which were specific item searches.

36

Table 5.3: Analysis of searches in the online catalogue and at the shelves, showing theorder of steps—OPAC2 SAMPLE. For f and } see table 5.2.

| Steps; used in search, inL orde:r
AA
B
AA
B
tAA
tAA
B C
B C
tAA
tAA
C
tAA
c AA
DD AA
AA
DD AA DD
tAA
DD
tAA
DD AA
tAA
B
B
AA

tB
tB

c
c AA
c
c AA
c
c
c
c

No». of searches
12
12

AA

DD

B

C

B

C

B
B
B

C
C
11

B

19
DD
DD
DD
DD
DD
DD
fDD
DD
DD
DD
JDD
DD
DD

AA

B
B
B
B

B
B

C

C
C

5
C
4
3
C
1
AA
B C
1
AA DD AA
C
1
AA
Total number of searches 104

5.1.2

Defining subject searches

T h e true identity of the different types of searches was not always evident at theoutset of a search. To establish the type of search, the initial approach through tothe final outcome had to be taken into account. It was found that out of a total of
128 cases initiated as specific item searches, (Tables 5.2, 5.3, initial categories AA orB), 43 (34%) developed into subject searches (including any subsequent category C,
DD). Since these cases subsequently contained subject search characteristics, they wereregarded as subject searches. Cases thus classified as subject searches are marked witha dagger in Tables 5.2, 5.3. For example, after initiating searches for known items,
searchers may have:• looked for alternatives at the catalogue or at the shelves if they failed to find theintended item;• having found the intended item, looked for additional works either in the
catalogue or at the shelves;• intentionally used a known item as a means of locating a relevant area at the
shelves to carry out a subject search.
Such searches, in spite of their initial objective or tactic, taken as a whole, have been
deemed to be essentially subject searches and thus have been classed as such.
The reverse, i.e. specific item searches disguised as subject searches occurred only
eight times in the two samples (Category JDD).
Table 5.4 compares the occurrence of hybrid and pure subject searches. Puresubject searches contain categories C, or DD (Table 5.2, 29 cases, Table 5.3, 39 cases).
Those hybrid subject searches initiated as specific item searches at the online catalogue
were further analysed and divided, as shown in Table 5.5, into those where the subject
searching element occurred at the bibliographic tool (initial category f AA followed bycategory B only or DD, 9 cases Table 5.2, 6 cases Table 5.3, ) and those where thesubject searching was introduced at the shelves, (initial category fAA followed by B
and C or C, 11 cases Table 5.2, 9 cases Table 5.3). It appears that the subject searchingelement tends to be introduced at the shelves.
The use and non-use of the online catalogue for the OPAC2 sample is shown in
Table 5.6 which compares the subject and specific item searches and the type of search
ctpproach adopted by the user in initiating the searches. The data relates to Table 5.3
as follows:• Subject searches at the online catalogue initial categories fAA and DD, 30 cases• Subject searches at the shelves initial categories fB and C, 32 cases• Specific item searches at the online catalogue initial categories A A and JDD, 27
cases• Specific item searches at the shelves: initial category B, 15 cases
We find a predominance of subject searching, 60% (29% + 31%) and the use of theonline catalogue stands at 55% (29% + 26%)
38

Table 5.4: Hybrid and pure subject searches

OPACl

Hybridsubject
searches
20 (41%)Pure
subject
searches
29 (59%)

Total
no of
cases |
49

OPAC2

23 (37%)39 (63%)

62

Total

43 (39%)68 (61%)111

SampleTable 5.5: Hybrid subject searches initiated at the online catalogue
SampleOPAClsampleOPAC2
sampleTotal

Hybrid searches
subject searchingat the online
catalogue
9 (45%)

Hybrid searches
subject searchingat the shelves

Total
no of
cases

11 (55%)20

6 (40%)

9 (60%)

15

15 (43%)20 (57%)35

Table 5.6: Subject and specific item searches initiated at the online catalogue and at
the shelves
Type of search

Subject searches
at a bibliographic tool
Subject searches
at the shelves
Specific item searches
at a bibliographic toolSpecific item searches
at the shelves
Total no. of cases

39

OPAC2
sampleno. ofsearches
30

29

32

31

27

26

15

14

104

100% |%

5.1.3

Search formulation

Having identified subject searches in the initial quantitative analysis of the data sets,
the next stage was to carry out a qualitative analysis on the search formulation of
those cases.
For the online catalogue, all searches which included a subject searching element
at the catalogue in both samples, O P A C l (Category DD, 36 cases, Table 5.2), andOPAC2 (Category DD, 26 cases, Table 5.3), were combined into one data set totalling
62 cases.
T h e analysis of search formulations at the online catalogue examined how a user's
query or expressed topic is related to the initial access to the bibliographic tool, i.e.
whether or not it is modified and if so how it is modified, whether it is broadenedor narrowed. This corresponds to Taylor's (1968) formalized need (Q3) and thecompromised need (Q4) and is referred here as 'the initial search formulation'. Unlike
the case of a reference enquiry, the query here is presented directly to the system
without the benefit of an intermediary to allow for negotiation.
Each transition was classified as exact, narrow to broad, or broad to narrow,according to the contextual relationship between the terms of the expressed topic, andthe initial access point, i.e. what the user typed in first. In some cases the contextual
relationships could have been described by categories other than exact, broad, andnarrow, however it was found that the hierarchical interpretation suited the majorityof cases.
Subject interests were expressed in a very concise manner, on average two terms
only. Access to the online catalogue revealed an interplay of two approaches: contextual
and matching. In the former searchers altered the expressed topic in order to findrelevant terms in the tool and in the latter they attempted to match the expressed
topic directly with the tool. Initial access terms which were partially different from theexpressed topic may have encompassed either or both approaches, straight matchinga n d / o r contextual. It was not possible to ascertain in which circumstances a user
would adopt a matching approach as opposed to a contextual one or vice-versa. For
example in the case of 'short term and long term memory' being given as an expressed
topic, it is unclear whether the choice of the term 'memory' only as an access term
instead of 'short term memory' was an attempt to find the most appropriate term tomatch the language of the index or whether there was a attempt to place the queryin a broader context to include both specific aspects, i.e. short term and long term
memory.
Table 5.7 shows the contextual relationship between the expressed topic and theterms chosen for the initial access to the online catalogue. Having chosen their initialaccess points, 12 out of 62 cases (19%) didn't succeed in their initial access, whilstanother 15 cases partially matched their initial access points, either through the system
displaying a closest match or the searcher browsing subject headings.
Table 5.8 indicates the systems options users selected in carrying out subject
searches on the online catalogue. These include the first as well as subsequent choices.
The order of the options presented in the opening menu may have influenced theirapproach.

40

Table 5.7: Expressed topic and initial access to the online catalogue
T y p e of approach
Exact
Narrow to broad
Broad to narrow

No. of cases
36
17
9

~%]

62

100 j

Total

58
27

15 1

Table 5.8: Search options in the online catalogue
OptionNo. of cases

%

34
19
9
62

55
31
14
100

Searched by subject headings
Searched by keyword accessSearched by both
Total

Table 5.9: Orientation tactics in the online catalogue
Tactic| Consulted initial access point only
| Browsed subject headings
Browsed subject headings andgenerated new terms
Browsed subject headings, generated new
1 search terms and changed search strategy
Browsed subject headings andchanged search strategy
Generated new search terms only
Generated new search terms andchanged search strategy
Changed search strategy only
Total

41

No. of cases

%

11
12
11

18
19
19

6

10

3

5

9
9

14
14

1
62

2

100 1

Once searchers had accessed the online catalogue, they then followed various tactics
as shown in Table 5.9. A change in strategy here refers to switching from a strict subject
approach to include a specific item approach or vice versa, i.e. a hybrid search.
Searchers thus tended to refine their searches by browsing subject headings (52%),
generating new terms (56%) or changing search strategy (31%).

5.1.4

Search outcomesA third aspect of the analysis was concerned with determining success/failure bothat the catalogue and at the shelves for all subject searches undertaken in both
environments.
They included the following 83 cases:• Subject searches initiated at the online catalogue, (initial category DD OPAC1
sample Table 5.2, 29 cases, and OPAC2 sample Table 5.3, 15 cases)• Subject searches initiated as specific item searches at the catalogue but
developing into subject searches either at the catalogue or at the shelves, (initialcategory fAA O P A C l sample Table 5.2, 20 cases, and OPAC2 sample Table 5.3,
15 cases)• Subject searches which were initiated at the shelves but were continued at theonline catalogue, (initial category C + DD, OPAC2 sample Table 5.3, 4 cases)
Comparisons were drawn to find out:a) Success/failure rates at the online catalogueb) Subsequent success/failure rates at the shelves for cases followed up from theonline cataloguec) Final success/failure rates of cases initiated at the online catalogue
The results of the consultations of the online catalogue are given in Table 5.10 .
Success and failure can be defined in different ways. In particular cases with references
found but not available for loan could be considered as failed searches. Others howeverchose a class number as an alternative and were thus considered as successful searches.
In Table 5.11 the final result at the shelves for subject searches initiated at the online
catalogue are presented. Of the original 83 cases, 10 cases failed at the catalogue, 6 ofwhich found references not available for loan and did not pursue their search further.
T h e final success/failure rate at the online catalogue is 82%/18%. A high proportionof searches were satisfied with the titles they followed up from the online catalogue anddid not search any further, (23%). The selection of titles found in the online catalogue
also promoted the selection of other titles at the shelves, (30%).
The retrieval of titles from the shelves via class numbers extracted from thebibliographic tool appears less effective For class number searches from the online
catalogue the rate is 6 6 % / 3 3 % (10 and 5 cases respectively).
42

Table 5.10: Outcome of subject searches at the online catalogue
Result%]No. of cases

Found reference(s)Found class number(s)Found reference(s) but
not available for loan
No references found
Total

58
15
6

70
18
7

4
83

5
100

Table 5.11: Final result at the shelves for subject
Searches initiated at the
1 Result

online catalogue
No. of cases

Selected only title(s)I found at the catalogue
Selected title(s) foundat the catalogue &; others
Did not select any
title(s) found at thecatalogue but selected
othersDid not select any
title(s) at all
Selected title(s) froma class number(s)Did not select any
title(s) from a classnumber(s)
Total

17

%
23

22

30

11

15

8

11

10

14

5

7

73

100 j

Table 5.12: Success/failure rates at the online catalogue and at the shelves

Onlinecatalogue
Onlinecatalogue/
shelves

Success
no. of cases
73
60

%

Total

88

Failureno. of cases
10

12

83

82

13

18

73

%

43

Table 5.12 compares success/failure rates for searches both at the online catalogue
and at the shelves. The success rate was well maintained at the shelves, dropping only
6%. In the final outcome however searchers selected on average only 1.9 items fromthe shelves.

44

5.2

Experiment 2—subject searching strategies

The holistic approach of Experiment 1 showed that 39% (Table 5.4) of subject searches
were identified as hybrid searches initiated as specific item searches, 43% of which were
undertaken at the catalogue and 57% developed into subject searches at the shelves(Table 5.5). The second experiment sought to further explore the holistic approachwithout following the searcher to the shelves. The aim was to attempt to elicit user's
intention with respect to subject searching strategies.
The objectives were twofold:• To ascertain whether users would reveal their intent to undertake a subject search
via a specific item search if questioned either before accessing the catalogue, orafter the catalogue consultation but prior to searching at the shelves.
• To determine to what extent the screen logging facility represented an accurate
record of the searching activity and confirmed the searcher's declared intention.

5.2.1

Methodology and validation of data gathering m e t h o d s

In order to address the research question pertaining to the identification of subject
strategies, the following methods of eliciting information from catalogue users were
tested:a) a self administered offline pre-search questionnaire (Appendix B l )b) an online pre-search questionnaire (Appendix B2)c) a self administered offline post-search questionnaire (Appendix B3)d) an online post-search questionnaire (Appendix B4)e) a screen log of search sessions
A basic multiple choice format was adopted. This was then adapted dependingon the online or offline environment and at which stage the questionnaire would bepresented to the searcher. Separate data sets were collected for each questionnaire anda screen log was recorded for each case. Searchers were given the opportunity to declaretheir strategy or intended strategy at two stages in each of the questionnaires. Firstly
in a direct form at the outset when asked what they wanted and secondly indirectly
following an anticipated or actual outcome, when asked how they would pursue theirsearch as a result of the outcome.
The different methods adopted for elicitating this verbal d a t a were compared andvalidated in a number of ways:• The verbal d a t a obtained from each type of questionnaire was compared forreliability with the non-verbal data provided by the screen logs.
45• Pre-search and post-search questionnaires were compared to ascertain users'
perception of the search before and after the event.
• Online and offline questionnaires both before and after searches were compared
to assess the influence of the different medium.

5.2,2

Results of Experiment 2

Three types of searches were identified and classified as follows:a) Pure subject searches were subject searches declared in the questionnaire andconfirmed by the log.
b) Specific item searches were searches declared as such in the questionnaire andconfirmed by the log.
c) A hybrid search was identified if the search met one of three conditions. Firstly
if the searcher declared in the questionnaire that the search was or would be a
combination of a specific item and subject search. Secondly a search could be
identified as a hybrid by default if the searcher was unable to declare whether
the search was one of the two types. Thirdly searches which were declared in thequestionnaire as either a specific item search or a subject search but where thelog indicated otherwise, were also classed as hybrid searches.
We thus differentiate between a declared and undeclared hybrid search (searches
assumed hybrid by default are included with those declared hybrid. In the former
case, the searcher declares in the questionnaire the intention to adopt or the fact of
having adopted both a specific item and subject approach in pursuing a search. In thelatter case, there is indication in the questionnaire that the searcher does not intend
to undertake, or has not undertaken, a hybrid search but the log reveals that a hybrid
search has been undertaken. Alternatively, an undeclared hybrid search could also be
a search identified either as a specific item or subject search in the questionnaire but
which turns out to be the opposite in the log.
Table 5.13 presents the different types of searches for each of the samples for whicha different d a t a gathering method was adopted. The combination of both subject
searching strategies, i.e. pure and hybrid subject searches, reveals an overall dominanceof 72%. Hybrid searches account for the highest proportion (45%) of all types ofsearches. P u r e subject searches ranged between 16% and 35% of the different typesof searches and 37% of all subject searches overall, whereas hybrid searches ranged
between 31.5% and 63% of the total number of subject searches.
In Table 5.14 the 82 hybrid searches are divided between those which were declaredin the questionnaires and those which were not. Each data set is further divided
to indicate the identity of each search if the information derived solely from thelog is taken into account. In the case of the undeclared hybrid searches, both theidentity derived from the questionnaire and from the logs are given, thus indicating
the discrepancy between the verbal and non verbal data which led to the individual
search being classified as a hybrid search.
46

Table 5.13: Experiment 2 : Types of Searches
1 T y p e of QuestionnairePre-search offline
Pre-search online
Post-search offline
Post-search online
Total no. of cases

Specificitem
19 (38%)11 (23%)8 (16%)
13 (37%)51 (28%)T y p e of S e a r c hHybridPure
Subject
8 (16%)
23 (46%)
12 (26%)24 (51%)17 (35%)24 (49%)11 (31.5%) 11 (31.5%)
48 (27%)
82 (45%)

Total No.
of cases
50
47
49
35
181

Table 5.14: Experiment 2: Declared and Undeclared Hybrid Searches
Declared
hybridsearches
16

T y p e of Q u e s t i o n n a i r e
Pre-search offline
(54%)

12

Pre-search online

18

Post-search offline(46%)
Post-search online

6

T o t a l n o . of c a s e s

52 (63%)

47

logs

9 SI
2SU
5 H
7 SI
4SU
1 H
13 SI
2SU
3H
4 SI
2 H

Undeclaredhybridsearches
7 4 SI
3 SU
12 4 SI
8SU
6 1 SI
5SU
5 2 SI
3SU
30 (37%)

logs

3 SU
1 H
1 SI
2SU
2H
6 SI
1 SU
4 SI
1 H
2SU
2 SI

It would appear that irrespective of the timing of the questionnaire before orafter the catalogue consultation, or the medium, searchers declared their intention
to undertake a hybrid search as a subject searching strategy, 54% for the presearchquestionnaires and 46% for the post-search questionnaires.
A closer analysis of the logs of the declared hybrid searches, indicates that 11 outof the 52 carried out a hybrid search at the catalogue, but 33 carried out specific item
searches with the declared intention in the questionnaire of pursuing a subject searchat the shelves. The remaining 8 cases who carried out apparently pure subject searches
at the catalogue would appear to have used the subject searching option as a means ofsearching for both known items and unknown items on the same topic. In spite of theuse of the catalogue, users thus appear to be clearly stating their intention to adopt a
hybrid subject searching strategy and to pursue subject searching at the shelves.
Undeclared hybrid searches which account for 37% of hybrid searches reveal asimilarity of intent with 13 out of 30 logs indicating the use of specific item searches as ameans or strategy of undertaking a subject search with 6 other cases actually carrying
out hybrid searches at the catalogue. The remaining 11 cases however show the reverse
situation, that is a subject or hybrid search undertaken as a means of carrying out aspecific item search.
In comparing the verbal data elicited from the user through the questionnaires
and the non-verbal data derived from the logs, it is evident that the logs considered
independently are not a reliable source for identifying types of searches. Out of 181
searches, 110 (61% i.e. 51 specific item, 48 pure subject, and 11 hybrid searches) havelogs which match the information provided by the corresponding questionnaires. In
71 cases (39%) however, the logs did not confirm what the searchers declared in thequestionnaire.

48

5.3

Experiment 3—search articulation & formulation

Experiment 1 revealed that searchers on average expressed their search topics with
only two terms and also tended to use those same terms to access the online catalogue(Table 5.7). If Taylor's categories of information need were applied, it would appear
that there is very little difference, if any, between the formalised need (Q3) and thecompromised need (Q4), (Taylor, 1968). One explanation could be that there is alreadya strong element of compromise at the formalised stage. In view of the close relationshipbetween expressed topic and initial search formulation, the aim of the third experiment
was to attempt to establish a clearer distinction between the way searchers expressed
their topics in their own terms and the way they formulated their search in accessing
the catalogue.
The objectives were as follows:• To encourage searchers to articulate their search more freely.
• To establish how topics expressed more freely would relate to how a searcher
formulates a search in the online catalogue.
• To determine whether searchers could articulate their search differently before
or after searching the catalogue.

5.3.1

Methodology and analysis

D a t a was collected from four different groups of searchers using a version of acommon questionnaire adapted to suit the environment and timing of the particularquestionnaire. As with the previous experiment, each search was logged. The test
instruments for Experiment 3 included the following questionnaires:• self-administered offline pre-search (Appendix CI)• online pre-search (Appendix C2)• self-administered offline post-search (Appendix C3)• online post-search (Appendix C4)
The questionnaires consisted of a single question inviting searchers to declare what
they were looking for and to describe it as fully as possible. The example provided
suggested implicitly that they could express what they wanted in terms of a specificitem or a topic. In the initial analysis the questionnaires and the logs were used to
identify the type of searches being undertaken as described in the previous experiment,(Section 5.2.2). The pure subject searches and hybrid searches were then further
analysed:49a) To determine the relationship between the expressed topic as articulated in thequestionnaire by the searcher and the search terms used to formulate the searchin accesssing the catalogue.
b) To determine the influence on this relationship of the method of eliciting thearticulation.

5.3.2

Results of Experiment 3

Table 5.15 shows the distribution of the different types of searches across the different
d a t a sets. In comparison to Experiment 2 (Table 5.13) the low occurrence of hybrid
searches 16% as opposed to 45% could possibly relate to two factors. Firstly, the holistic
approach adopted in Experiment 2 took into account searchers' intention with regard
to pursuing their search after the catalogue consultation, i.e. at the shelves which
Experiment 3 did not. Secondly, the first multiple choice question in Experiment 2
included the possibility of a hybrid search whereas in Experiment 3 this approach was
not suggested.
In the case of items from a reading list, searches were classed as they were perceived
by the searcher. For example if searchers declared that they were looking for known
books a n d / o r listed them, the search was classed as a specific item search. If however,they went further and qualified or provided a subject description, those searches
were classed as hybrid searches. Although some searchers did declare that theywould undertake a hybrid search, it is likely that others remained undeclared andconsequently undetected. On the other hand, the number of pure subject searches inboth experiments were comparable (27% and 28% respectively).
Table 5.15: Experiment 3: Types of Searches
T y p e of Q u e s t i o n n a i r e

Pre-search offline
Pre-search online
Post-search offline
Post-search online
Total no. of cases

Specificitem
22 (65%)51 (53%)28 (61%)24 (50%)125 (56%)

T y p e of S e a r c hHybridPure
Subject
8 (23%)
4 (12%)
32 (33%) 14 (14%)
7 (15%) 11 (24%)
8 (17%)
16 (33%)63 (28%) 37 (16%)

Total no.
of cases
34
97
46
48
225

Each of the search statements articulated in the questionnaires was compared withthe different terms used by the searcher to access the catalogue. These included initial
as well as subsequent access terms. Each case was then classed according to what
extent the written search statements were reflected in the overall search formulations.
T h e cases were found to fall into three categories:a) The search articulations which were accurately or exactly represented in the
search formulations.
50b) Those cases who articulated their search in a more specific or qualified mannerthan that revealed by the terms used to search the catalogue.
c) The searches whose statements were couched more broadly than the actual search
formulations.

Table 5.16: Experiment 3—Search Articulations compared with Formulations
T y p e of Articulation
NarrowBroad

T y p e of Q u e s t i o n n a i r e
Exact
5 (42%)

Pre-search offline

34%

50%
Pre-search online
Post-search offline

24 (52%)10 (56%)
13 (54%)52 (52%)2 (16%)
7 (15%)
2 (11%)

46
18

6 (25%)
17 (17%)

24
100

16%
15 (33%)6 (33%)

26%

55%
Post-search online
Total n o . of c a s e s

5 (42%)

Total no.
of Cases12

19%
5 (21%)
31 (31%)

Although 52% of search articulations did match the search formulations, as seenin Table 5.16, 48% did not. The proportion of exact, narrow and broad search
articulations for the combined types of pre-search and post-search questionnaires wasof the same order with the exact articulations having the highest occurrence followed by
the narrow and broad articulations. It would appear that there is nothing to distinguishthe pre-search or post-search articulations and that the catalogue consultations had
no apparent effect on these articulations.
The average number of significant terms which were included in the search
articulations are presented in Table 5.17. Although searchers were encouraged to be
discursive the overall average for all types of articulations and all types of questionnaires
was 2.9 terms compared with 2 terms in Experiment 1. Narrow articulations tended tobe less concise with 4.3 terms on average overall followed by broad articulations with
2.5 terms and exact ones at 2.3 terms. The online statements tended to be slightly
more concise than the written offline versions but not significantly so.

51

Table 5.17: 15 Experiment 3—Number of terms in search articulations
Average no.
Type of Questionnaire Type of articulationof Terms3
ExactPre-search offline
Narrow
3.8
1.5
Broad
Overall 3
Exact
2.1
Pre-search online
Narrow
3
2.6
Broad
Overall 2.4
Exact
2.3
Post-search offline
Narrow
7.1
4
Broad
Overall 4.1
Exact
2.3
Post-search online
Narrow
5.3
Broad
2.2
Overall 3

52

5.4

Experiment
naires4—Oliveinteractive

question-

In Experiment 4 the aim was to explore the use of interactive online questionnaires asa more direct means of eliciting information from the user as opposed to the pre-searchand post- search, online and offline, paired tests carried out in Experiments 2 and 3.
The objectives were:• To determine convenient and appropriate points whereby a search could beintercepted.
• To assess the reliability of the method in relation to other methods being tested.
• To explore to what extent the method could provide more information on the
search process.

5.4.1

Methodology

Two questionnaires were tested (Appendix D l , D2) to address research questions
comparable to those of Experiments 2 and 3 relating to subject searching strategies
and search articulation respectively.
In all cases the 'Olive' interception was triggered by a restart after the display of abibliographic record containing circulation data (i.e. copy availability screen, Section
4.2.2). Different conditions were set for each of the questionnaires (Section 4.2.3) andthe sequence of the questions also depended on replies given to previous questions.
Experiment 4A in essence replicated Experiment 2 as described in Section 5.2.1.
The main difference was that searchers were asked what further action they intended topursue at the catalogue a n d / o r at the shelves with regard to particular record(s) they
had viewed. In addition there was the possibility of questions being repeated several
times during a particular search session, ie. each time the user pressed the restart key
following the viewing of a 'copy availability' screen in the previous search sequence.
Experiment 4B differed slightly from Experiment 3 (Section 5.3.1) in that searchers
were preselected in so far as those choosing a subject search option from the opening
menu, met part of the condition for the interception. Searchers were not only asked
to express their search requirements as fully as possible in a free text question but
were also asked to specify whether the item(s) displayed, satisfied or partly satisfied
their requirement. If an outcome partly satisfied their requirement they were asked toqualify whether the item(s) were related, narrower, or broader.

5.4.2

Results of Experiments 4A and 4B

Due to the technical constraints and complexities as discussed in Section 4.4 which
led to the belated implementation of Olive, the small samples collected can only beconsidered as an indication of the potential of interactive questionnaires.
The data on the subject searching strategies collected by this method as shownin Table 5.18, would seem to be comparable to that collected in Experiment 2, with53

hybrid searches being declared through the questionnaire and not necessarily being
evident in the log.
Table 5.18: Olive—Experiment 4A Type of Searches
Type of SearchSpecific item
Pure Subject

No. of C.ases

logs

15
2

Hybrid7

Total no. of cases

24

Declared6

Undeclared

1

SU

SI
H
SI

3
3
1

In Table 5.19 search articulations are classed according to how they comparedwith the search terms chosen by the searcher to access the catalogue. Three cases
whose expressed topics corresponded exactly with search terms found items which met
their requirements, whereas two cases found items which were related. Narrow search
articulations would appear to be less successful.
Table 5.19: Olive—Experiment 4B Search Articulations compared with search
formulations and search outcomes
No. of Search
Type of search
Cases Outcome
articulation
5
3 exact
Exact
2 related
4
1 exact
Narrow
2 related
1 narrow
9
Total no. of cases
Exact search articulations averaged 2.6 terms whereas narrow articulations averaged
4.2 terms.

54

5.5

Experiment 5: search replays

Although the screen logging facility provided an unobtrusive method of recording asearch in its entirety, the recording of the searching activity at the terminal was
predominantly from the system's perspective. The replay facility was developed inconjunction with the logging facility as a means of considering the retrieval task fromthe user's perspective without the risk of influencing or interferring with the actual
search process. The method was thus tested:• To assess users' perception of the search and retrieval task.
• To allow users to identify decision points and to offer explanations for decisions
taken including relevance judgements.
• To indicate users' expectations and satisfaction of search outcomes.

5.5.1

Methodology

A small sample of nine searchers who were observed to be undertaking a subject searchon the online catalogue whilst participating in Experiment 3, were invited to havetheir search replayed immediately after they had completed their search and declared
their topic in the post-search offline questionnaire. They were asked to 'talk through'their search and provide explanations on decisions and comments on results as it was
being replayed to them. This was done in real time providing an exact replication or
simulation of their search. Although a pause facility was available this was not used.
During the course of the replay the experimenter on occasion prompted searchers toindicate what they were looking for and to explain choices made. For the most parthowever searchers were most forthcoming in providing explanations. A summary ofusers' comments were recorded in writing by the experimenter together with some
additional notes.

5.5.2

Results of Experiment 5

Due to the exploratory nature of the methodology and limited number of cases, thesomewhat anecdotal information gathered in this experiment cannot be quantified.
Nevertheless the small sample did produce some useful qualitative data from whichindicative characteristics of subject searching behaviour could be derived. These fallunder several general headings.
I n f o r m a t i o n 4 neecT
Although all the searchers clearly expressed a formalized need prior to the replay, in 5
out of 9 cases it emerged in the course of the replay that what they were seeking was
far more nebulous than what they had indicated either in the search terms or expressed
topic. Some of the searchers declared that they were searching the catalogue in orderto define what they were looking for. For example, the searcher who expressed his55formalized need as 'corporate finance', searched under ther terms 'funding' followedby 'banking'. Although he found a title which included the terms 'corporate funding',
which he considered relevant, he also explained that he wanted different aspects ofcorporate funding but he wasn't sure what these might be. By using the search term'banking' he looked for clues in the titles and found that terms like 'equity', 'stock
exchange' and 'borrowed funding' might be of use. He did not however use these terms
to further his search at the catalogue, but noted the items as possible items to consult
on the shelves.
It would seem that Taylor's categories of information need in such circumstances,are perhaps too rigid and that Belkin's model of anomalous state of knowledge (ASK)
may be more appropriate. (Taylor, 1968; Belkin et al, 1982). There is further evidence
to support this. For instance two searchers also divulged that they would adapt theirrequirement according to what they found in the catalogue. T h a t is, they would alteror adjust their project topic according to, or as a result of the documents they could
find in the catalogue as in the following example.
A searcher expressed her requirement as looking for:information on international law of migration, the United Nations and 'race'or 'refugees', immigration: Britain, restrictions and practices, statelesspersons and their movements.
She commented after searching under the terms 'United Nations', 'race','international migration', 'refugees', 'race and colonisation' and 'immigration':
I have looked for things before, I am really looking for the theory behindthe policies but all of these (references) tend to deal with the law so I'll
have to tackle it that way.
Other searchers also indicated that the catalogue consultation did not represent
their first attempt at finding information on their topic. One case expressed that it
was her third attempt at looking for material on a topic, having consulted the catalogue
and shelves twice previously. Searchers clearly considered the catalogue consultationas part of an ongoing process and not as a single independent event.
Matching the s y s t e m language
In eight of the nine cases, users' formalised need was declared as a qualified statementfrom which terms were selected to access the catalogue. Users selected those which
they thought would most likely succeed in providing a posting even if those terms werebroader than their requirement. Searchers thus consciously took a broader strategy.
This proved to be a successful approach in so far as for all cases the selected search terms
resulted in a posting. Nevertheless one searcher objected that the option to search by
subject headings had been withdrawn by the library. She had found it a useful source
for suggesting possible search terms. "Better than nothing" she commented.
56

Refining s e a r c h e s
The broad approach to search formulation led to high postings rather than the reverse.
Users responded to high postings in one of two ways. They restarted their search
immediately by using new search terms to limit the search, or browsed through somethe references until they found something of interest. Postings considered to be high by
the searchers ranged from 84 to 1000, although in these particular two cases the userbrowsed through several screens. On the other hand postings of 93 and 241 prompted
an immediate restart of the search.
The different response to high postings did not apparently bear any relationshipto the type of search i.e. whether or not the searcher had a clear idea in mind or
was experimenting. For example one case searching for 'health statistics', entered'health' as a search term and browsed 14 out of 159 screens until he found a title
containing both terms 'health' and 'statistics'. On the other hand another case withthe expressed topic 'history of drama', entered the term ' d r a m a ' which resulted in a
posting of 241 references. The searcher restarted and entered the terms 'history ofd r a m a ' which resulted in 21 references. It then emerged during the replay in browsing
those references, that the searcher was unclear about what he was looking for and thatperhaps something on English theatre and drama might be useful.
Refining searches either by browsing references or by using new search terms
appears to be based on ad hoc decisions. Although in the example of health statistics
a Boolean operation may have been more fruitful, in the case of the history of drama
it is unclear what help could have been provided with the search mechanisms available
in present day systems.
S e a r c h o u t c o m e s and satisfaction
Searchers noted both specific references and / or class numbers as a result of thecatalogue consultation. Four out of the nine cases recorded class numbers only to
pursue their search at the shelves. Three out of those four had not viewed any
individual records but only browsed screens of brief references. The remaining fivecases noted individual references which they would look at in the first instance at the
shelves, but also indicated that they would need to look for other items at the shelves,for they did not consider the items retrieved from the catalogue alone could adequately
meet their requirements.
It could be said that for all of the cases the catalogue consultation was regarded
by the users as tentative and provided limited satisfaction. Judgements of success or
failure at t h a t point were considered premature.

57