-16CHAPTER 3
METHODS
3-1

General Experimental Outline

In order to achieve the aims of the design study, we decided upon thefollowing general methodology:a)Tape recording interviews with users of actual information systems(libraries) at reference points.

Asking users to discuss the problem

with which they are faced, and only afterwards asking them to present
a request to the system.
b)

Finding 30 abstracts in information science of 200-300 words each ofarticles written by authors in the U.K.

c)

Adapting the text analysis program from Belkin (1977 a) to thecircumstances of this situation.

d)Implementing the analysis algorithm as a computer program.

e)

Mailing the structures to authors and users with questionnaire, orinterviewing where appropriate.

f)Analyzing the problem statements according to various structuralcharacteristics (such as strength of association, number of nodes,
degree of coherence), and classifying them.

g)Developing preliminary retrieval strategies based on the classification and structural features of the representation.

3.2

Problem Statements*

The first stage of our experiment was to try to collect suitable problem
statements for analysis and subsequent evaluation.

Our basic ideas

about what constitutes suitability were that the statements should be:1.

directed toward the users1 research or work situations rather thanto what specifically they hoped to find;2.

as uninfluenced as possible by knowledge of the operation of IR
systems;3.

obtained with minimum intervention; and4.

representative of real information needs, expressed by the personwith that need.

Our subjects were drawn from users of Central Information Services Library Resources Coordinating Committee, in the University of London,
which carries out on-line searches for students and staff of all of thecolleges and institutes of the University of London.
*

Mrs. A. Vickery,

The data presented here, and in subsequent sections dealing with problem
statement analysis and classification are more fully reported in Brooks,
1978.

-17-

the senior Information Systems Officer, kindly gave us permission toapproach CIS users to ask if they would take part in the experiment.
The purpose of the research was explained to each user, and they weretold that they would be asked to evaluate the analysis of the statement.
The interviews with the subjects were short, informal, open-ended andunstructured, in order to elicit maximum information with minimum
intervention (Kahn & Cannell, 1957; Payne, 1951).

One question only

was posed:"Would you like to talk informally for a few minutes about
the research you are doing at the moment, the problem that has
led you to have a search carried out, and the sort of information you would like to have as a result of the search?11
We maintained some flexibility, however, in order to take account of
individual differences among subjects, such as tendencies to strayfrom the point.

The basic question always remained the same, but there

was also some interviewer intervention with some subjects.
The interviews were carried out before the normal pre-search interview
or search itself, and were tape recorded.

In all, there were 27 inter-views, 23 from CIS users and 4 from M.Sc. students at The City University
who were just beginning work on their theses.

Subjects were not selected -everyone arriving at CIS for a search during the period of data collection(one month) was approached, and all agreed to participate.

Both thetopics of searches and the background of the subjects were therefore
widely dispersed - topics ranged from medicine to linguistics to education,
status from beginning M.Phil student to clinician to professor (see Table 5 ) .
Eight written queries were also chosen from the CIS files and analyzed
in order to see if there were systematic differences between them andthe oral problem statements.

Those chosen were selected on the basisthat they contained details of the user's research as well as of whats/he wanted to know.
3.3

Abstracts

The next stage was to collect abstracts for analysis and subsequent evaluation.
We chose 31 abstracts of recent publications (journal articles or publishedconference proceedings) from the field
restricting them to U.K. authors.

of library and information science,

We chose the abstracts from Library

and Information Science Abstracts, and from our personal journal files,
attempting to include mostly author abstracts and to get a representationof lengths, ranging from 90 tokens to over 2000.

Restrictions as to date

and author location were for ease of evaluation.

(See Appendix F for

-18-

the full text of each abstract).
3-4

Evaluation ProceduresUpon analysis, both problem statements and abstracts were evaluated by
the IR users and authors respectively.

The procedures for the two groups

were slightly different, so we will consider them separately.
The IR system users were sent a package consisting of:a cover letter reminding them of the experiment and explaining what theyshould do with the other materials;
a transcript of the interview;a copy of each of two different representations of the problem statement;a questionnaire for each of the two representations, on each of which
the premises of the representation were explained;a third questionnaire for comparing the two representations;a stamped addressed return envelope; anda cheque for £2.00.
See Appendix A for the cover letter and questionnaires.
package was stapled together in a set order:

The entire

cover letter; transcript;

first representation, followed by its questionnaire; second representation
followed by its questionnaire; comparison questionnaire.

The order ofrepresentations was reversed for alternate subjects to guard against
possible interactive effects of one format on the other.
The evaluation package for the authors of analyzed abstracts was rather
simpler.

This evaluation was done after results for the interview

evaluations had been obtained, and on their basis we decided not to use
the association cluster representation.

This package thus consisted of:a cover letter, explaining the project;a copy of the abstract;a copy of the representation;a questionnaire including an explanation of the representation;a stamped, addressed return envelope;a cheque for £5.00.
See Appendix B for the cover letter and questionnaire for authors.
3*5

Text Analysis - Principles

In our work, we have assumed that a state of knowledge - anomalous in
the case of a problem statement - can be represented by a network ofweighted associations between words, standing for concepts.

We nowturn to the method used in our exploratory project to derive associative
structures from the texts obtained by interviewing and from published-19documents.

Figure 3 shows the text of a problem statement (one of theshorter ones), and a graphical representation of the derived structure
is displayed in Fig. 7.

Summaries of all of the problem statements

are in Appendix C, and their representations are displayed in Appendices
D and E.
The technique that we have used to work out associations between words
is linguistically very unsophisticated.

It computes a strength of associa1

tion between every pair of 'significant

word-types in the text under

examination, based upon their lexical separation.

There is no syntactic

analysis, and no attempt to represent the nature of an association.
Our view is that it should be possible to investigate the ASK hypothesis
through the construction and evaluation of a new retrieval system, evenwith a statistical, rather than linguistic text analysis mechanism.
The assumptions behind the measure of associative strength used are that
if two words are closely associated in the searcher's knowledge structure,then, firstly, they will be close in the textual expression of that structure, and secondly, the association will tend to find multiple expression
in the text.

The actual formula for the measure is given below, and issimilar to that used by Belkin (1977 a) in a previous, related experiment.
An important step in the analysis of a text is to identify the word-types
which are to stand for concepts in the structure.

This involves eliminatingwords which are non-significant for our purposes, and conflating thevarious forms of the significant ones.

Thus, the nodes of the derived

structures are labelled with truncated words - we take the liberty ofcalling them stems.

We have made no attempt to enable our analysis pro-grams to recognize synonyms.

As an example, consider the following sentencefrom a (tape-recorded) problem statement:"There is a problem in agriculture that if the grains germinate,
then there could be a substantial loss in yield, and this I would
presume has been looked at

in great detail."

The letter-groups underlined constitute the tokens selected from thissentence for inclusion in the association network.

It is usual, in

automatic indexing programs, to use a stop-list to eliminate words which
are regarded as poor indexing terms.

We have tended to include words,

such as "how", "why", "not", "effect", "presume", which haveimportance in the semantic structure of the statement, and which may
therefore partake in important associations.

The words which are notunderlined in the example above are a sample of the words we regarded
as non-significant in the problem statement corpus.

-20-

MY PROJECT IS LOOKING INTO THE INFORMATION SERVICESOF PROFESSIONAL INSTITUTES IN BUSINESS MANAGEMENT.
BY INFORMATION SERVICES I DON'T MEAN JUST THE
LIBRARY.

I AM TRYING TO COVER PUBLICATIONS,CONFERENCES, SEMINARS, EXHIBITIONS, MEETINGS, ENQUIRYSEARCHES AND THAT KIND OF THING.

I HOPE TO

DO THIS BY GOING AROUND TO EACH INSTITUTE AND
TALKING TO SOMEONE, USUALLY THE LIBRARIAN, AND
GOING THROUGH A QUESTIONNAIRE, COLLECTING DATAON THIS PARTICULAR INSTITUTE.

THERE HASN'T

BEEN MUCH EVALUATION DONE ON PROFESSIONAL ASSOCIATIONS SO FAR.

I ALSO WANT TO FIND OUT WHAT

LITERATURE THEY ARE EXPECTED TO KEEP AND TYPESOF SERVICES THEY WANT TO OFFER, TYPES OF BUSINESSINFORMATION THAT ARE NECESSARY FOR PEOPLE IN THESE
INSTITUTES.

I ALSO WANT TO KNOW OF ANY EVALUA-TIONS IN INSTITUTES OR LIBRARIES IN THAT FIELDTHAT HAVE ALREADY BEEN CARRIED OUT.
FIGURE 3: THE TRANSCRIPT OF A PROBLEM STATEMENT

-21Our immediate purpose in analysing the texts was to perform a check on
the appropriateness of our method by asking the interviewees (in thecase of problem statements) and authors (as in the case of abstracts)
to comment upon the structures produced.

Thus it was necessary to devise

ways of displaying the associative structure.

The straight-forward

method is to draw a network (see figure 7 ) , in which the nodes, labelledwith "stems11, represent concepts and the lines represent associations.
For clarity, it is necessary to omit the large mass of weak associations(the number of associations varies as the square of the number types ina text), and to simplify the representation of association strength.
The associations derived from an individual text were divided, subjectively,
into three groups:

strong, medium and weak.

In the network

the corresponding lines were drawn, thick, thin and broken.

diagrams,
Some indica-tion of strength was also given by length of line, although in a twodimensional picture, this cannot, in general, be done very accurately,or consistently.
Another method of displaying the associative structure of a text is tocluster the word-types, so that a person examining the structure cansee which concepts are closely linked, without being confused by theprecise pattern of interconnections.

We have generated a hierarchy ofsingle-link clusters of types for each text using an algorithm which
was originally developed for document clustering by van Rijsbergen (1971)
and Croft and van Rijsbergen (unpublished). Figure 14 shows such a hierarchy in tree form.

The horizontal distances from the branch-points tothe words on the left of the picture are directly related to the association strengths with which the cluster members are bound together.

See

Appendix E for this representation for all of the problem statements.
3*6

Text Analysis - procedure(i)

Data preparation.

Tsoe recorded problem statements were first trans-

cribed by a typist.

These and the written problem statements andabstracts were entered into computer files, care being taken to
separate sentences with full-stops, and to insert a special symbolat the beginning of each paragraph.

Each abstract begins with the

document's title, marked as a separate paragraph.
(ii) Dictionary construction.

In order to conflate word forms, and at

the same time eliminate non-significant words, each collection of
texts (problem statements and abstracts) was scanned to compile a
dictionary of stems (truncated words).
cally:
editing.

This was done semi-automati-a suffix stripping algorithm was used, followed by manual
Non-significant words were removed from this list.

-22(iii) Identification of tokens.

The stem dictionary was searched foreach word in the text, looking for the longest stem which matched
the word.
tor".

For example, the dictionary contained "fact" and "fac-The word "factorize" is matched by both stems, so "factor"

being the longer is used as the token in place of "factorize".
If there is no match in the dictionary, the word is taken to be
non-significant.

For each text, a list of stems is constructedgiving the position of each occurrence as a numerical triple:(paragraph number, sentence number, word number).

Occurrences ofrejected words are not included in the list, although theirpresence in theoriginal text is taken into account when working

out the positions of the significant words.

For example, the first

sentence in the problem statement of figure 3 will be processedas shown in Fig. 4.
(iv)

Calculation of association strengths.

The strength of association

between word-types in a text can be computed from the list ofoccurrences, as follows:Firstly, for any particular pair of occurrences of two types,
A and B, a score reflecting the distance between those occurrences
is calculated:
Score - -r-4—
1 + r

x

100

where r = 1 if A and B are adjacent within the same sentence.
r = 2 if A and B are within the same sentence, but notadjacentr = 3 if A and B are in adjacent sentences within the same
paragraph.
If the occurrences of A and B are any further apart, the score is
zero.

(The factor of 100 is present purely for computational

convenience).
Having calculated the scores for all pairs of occurrences of
A and B in the text, they are summed to give the strength ofassociation.
As an example, consider the stems BUSINESS and INFORM in the
text in Fig.3.

Their occurrences and association scores aretabulated in Fig.5.
The word associations for each text were listed by the computerin decreasing order of strength (e.g. see Fig. 6 ) .
(v)

Presentation.

Graphical representations of the associative struc-tures were prepared for examination by the originators of the

-23-

2S
CD

•~N

/*-\

•-N

eg

r^

oo

CO

1—

TEM

•—4

ISO

CO

a-

H

"S

^

H

"\

^N

/^S

^-N

^-N

CD
r-H

rH
rH

N^
rH

cr

rH

•>

•N

rH

rH

rH %

rH

rH

H

rH

rH

rH

rH

h-

CO
CO
LU

•—H

:s
—•
co

*s

rH

1—
CD

^0m-

LL.

1—H

i—^

CD

1—

^^

CJD

CO

1 ^~
1 LU
1—
CO

CD
LU
"O
CD
Q£
Q-

s:
a£
o
LL.
2:
H-l

•—•
>
c£

CO
CO
LU
LL.
CD

CO

Q-

CD

LU

ESSION

<=c

!

LLCD
ai

a.

\

\

LU
CO
r—
CD
LU

j

CD

CD

a-

J

•

1—
CO
Z2H

en

T TT T
i

ZD
1—

CD
Li-

\

\

1 \

<c
2:
^
S

=3
PQ

"1
>

CD

71
(

INSTITUTES Jff BUSINESS MANAG EMENT.

^<c

-24-

csj
CD

GO

en

^£
O

h-

OZ
ZD
CD

c^:

/—N

CXI

LL.

•>

21
LU
LU

CNJ

CD
LL.
Z

j»r

•—-«

CNJ
5?""

1—

^

LU
PQ

*-H

o
Q£

A*-^

CD

1

GO
GO
LU

<c

1—H

h-

*—«

s:

GO
ZD
PQ

Q

en
CD

zc
00
CD

GO
LU
CD
2 :
GO

CO

H

II
CD
LH

GO
LU

CO

LU
Q^

Cd

\—

CD
CD
GO

GO

I—
GO

m
CSI
m
ro

CD
GO

ZZ1

LU
a£

4-

+
..
zn

hCD

CD
CD
GO
GO

<c

GO
•—•
CD

PQ

<c
_J

GO

CD
GO
GO

t^\

CD
CD
GO

<C

GO

CD
_l

<c
CD

LU
CM
CD

GO
ZD
PQ

LU
_l
PQ

ZD
CD

-25-

INFORM SERVIC183

INSTITUT

LIBRARm

INSTITUT

133

EVALU
INSTITUT

PROFESSION125

INFORM

INSTITUT

116

INSTITUT

TYP

116

INSTITUTSERVIC116

BUSINESSINFORM

108

EVALU

100

TYP

BUSINESSINSTITUT

91

FIGURE 6^: THE STRONGEST ASSOCIATIONS FROMPROBLEM STATEMENT IN FIGURE 3,

-26texts.

Fig. 7 illustrates the network presentation (Association

Map Format), and Fig, 8 shows the corresponding hierarchical clustering of the word-types (Association Clusters Format),In our survey of problem statements, we sent both formats to thesubjects.
works.

The 40 strongest associations were used for the net-

The authors of the abstracts, however, only examined

the structures in Map Format, using about the top 40 associations
but with some flexibility in order to avoid splitting groups ofassociates with the same association strength.

Figure 9 shows

an example of an abstract, Figure 10 is its association map
representation.

-27-

*>*a

FIGURE ? :ASSOCIATIONIN

FIGURE

MAP FOR PROBLEM STATEMENT
9

-28-

I LIBEAE| 1WSTITUT

M| EVALU\

N* 3| PEOFESSION
1 SEEVIC
1 INFOEM

'

Wl

;
I

|
1 TYP
1 BUSINESS Lf

I PAETICUL

1OFFEB-\^I ASSOCI, \-r<r

1 FAR
1 AROUNDCOLLECTDATEXPECT
QUEST
TALK
USUAL
KEEP
LITEEA
NECESS\u

1PEOPL
ICAEE
FIELD
MANAG
MEAN
PEOJECT
IKNOW

Ptf

1 CONFEE
PUBL
SEMINABCOVEE
ENQUIEMEETSEAECH
1 EXHIBITFIGURE g t

to

ASSOCIATION
IN FIGURE 9

CLUSTERSFOR PROBLEM STATEMENT

-29-

Information Science and the Phenomenon of Information

This paper aims to deduce the fundamental phenomena of information
science, starting from two premises:that information science is aproblem-oriented discipline concerned with the effective transfer ofdesired information from human generator to human uaer, and that thesingle notion common to all concepts of information now extant is that°^ change of structure.

From these premises, a spectrum of information concepts is derived, anda partition of that spectrum particular to the purposes of information
science is described.

From this partition, the terms text and information(both in information science) are defined, and the fundamental phenomenaof information science are deduced:

the text and its structure, thestructure of the recipient and changes in that structure, and the structure of the sender and the structuring of the text.

These phenomena are seen as the basic components of the mechanisms ofthe channel, which have been the traditional area of interest to information
science.

Some implications of this approach for research in information

science are discussed in this paper.

And, finally, the question of the ethics of theoretical research in information science is raised, and a restrictive condition is proposed.

FIGURE

9:AN

EXAMPLE

ABSTRACT-30-

pea^»5

FLU\(D/W6NT
D6DUC

COtsiCBPTCWANJQ

PAPea.

PHfeNOfvieNJ
PAR-TIT

R60PI

S£M£>

xe^iKi

FIGURE

10: ASSOCIATION

MAP

FOR

ABSTRACTIN

FIGURE

9.