AUTOMATIC
1974

INDEXING

A
Stateofthe Art

SPARCK

JONES

Review

KAREN

April 1974

Computer Laboratory
University of Cambridge
Corn ExchangeCambridge

CB2Street3 0 6 , EnglandThis review was prepared under Grant No, SI/G/096 from theOffice for Scientific and Technical Information, Departmentof Education and Science (now the British Library Researchand Development Department).

The review was discussed at a Workshop held at Crawley, Sussex
on April 29 and 30 1974. This was sponsored by the British
Library Research and Development Division, and was organisedby the University of Kent, Canterbury. A Report on the Workshop
by Miss E. Wilson, Computing Laboratory, University of Kent, isin preparationo

C O N T E N T S

PrefaceI
cl,2.3.4

Introduction
The problem
Evaluation
Information retrieval:historical and general background
Related areas•1c, 2

Automatic Indexing Procedures
Semantics and syntax
Secondary system factors

clo2.3rAo5

SyntaxInput syntax
Description syntax
Index language syntax
Search syntax
Conclusion on syntactic indexing

IIIII

IV: terminology etc.

Semantics*1
Statistical semantics«2
Input semantics
3
Description semantics,4Index language semantics
c5
Search semantics,6Conclusion on semantic indexing

Vcl,2

Indirect Indexing
Citations
Document clustering.1o2Mechanised Systems
Standard systems
Non-standard systems

J02

Related Areas
Automatic abstracting and extracting
Question answering (fact retrieval)«1.2

Evaluation Experiments
Review of experiments
The SMART Project•1,2

Conclusion
Overview
Recommendations

VIVII

VIIIIX

P R E F A C E

Terminology

Information retrieval= document retrieval = reference retrieval= documentation (= retrieval).

Fact retrieval(alias question answering) direct extractionof facts from an information store (or obtaining of answers therefrom)

Indexing

provision of descriptions of documents for
retrieval purposes.

Extracting

provision of extracts of document texts forgeneral purposes.

Abstracting

provision of summaries of documents for general
purposesc

Automatic indexing

strictly, automatic provision of document
descriptions for retrieval. But in thisreview extended to cover all the linguistic
operations involved in analysing, describing
and searching for documents, and in creating
the index language required.

Analysisof documents to select information requiredfor indexing.

Description (1)of documents by characterising selected
information in an index language.

Searchingof file of document descriptions to finddescriptions meeting a request specification.

Natural language

the language of documents, their titles andabstracts, and of requests.

Index language

the language used to describe documents andrequests, which may be more or lessartificial.

Document

specifically the actual articles in a library,
but more generally the form of the document
input to indexing, whether full text, title
or abstract.

Surrogate(alias representative) the input form of thedocument when this is not its full text.

Description (2)the index language characterisation of adocument.

Unit

in analysis or indexing, any word or sequenceof words treated as a whole.

Worda word in natural language.
P.l

Keyword:a word or word stem from the input naturallanguage adopted as an index term.

Term:a word or word group in the index language used
to describe documents; a simple rather thancomplex item.

Subject heading:

also a word or word group in the index language;
but normally a complex or compound item.

Descriptor:

any unit in an index description, whether akeyword, term, subject heading or class label.

Precoordinate:

as ordinarily used.

Pos tcoord inate:

ditto.

Vocabulary:

the set of descriptors of an indexing language.

Classification:

any grouping of items, but ordinarily applied
to descriptors.

Clustering:classification applied to documents.

Syntax:(alias syntagmatic) refers to relations between
words or descriptors not constant in the language.

Semantics:(alias paradigmatic) refers to constantrelations between words or descriptors;
also simply to their own meaning.

Request:retrieval question,Collection:

in general, the set of documents in a retrieval
system; but specifically, the set of documents
and requests used for a retrieval experiment.

Recall:the ratio between relevant documents retrieved
and all the relevant documents in a collection,
for a request or set of requests.

Precision:

the ratio between relevant documents retrieved
and all the documents retrieved, for a requestor request set0

Pullout:

extent to which relevant documents are retrieved(in a general, not precisely measured way).

Selectivity:

extent to which relevant documents only areretrieved (ditto).

Performance:of a retrieval system, measured in appropriate
ways, for example by recall and precision.

Noticeable:interesting (as well as statistically significant) difference in performance,Material:

very interesting (ditto) difference inperformancea
P.2

Data references
A particular convention will be adopted for specifying testcollections. Thus an expression like "a 21x379 tropical foodstuffs
collection" refers to a set of 21 requests and 379 documents dealing
with tropical foodstuffs. These normally have associated relevancejudgements, i.e. sets of documents defined as relevant to therequests. Items like "the 42x200 Cranfield collection" refer towell-known test collections. As collection specifications are in
the form "mxn", the use of "m" or "n" refers to collections for whichfull details are not given, "m" representing an unknown number ofrequests and wn" of documents. The form "nK" is used where thenumber of documents is not given, but is manifestly large.

Literature referencesTo avoid overloading the text a slightly abbreviated form ofreference has been adopted. "Snooks 1969,1970", for example, refers
either to papers by the lone Snooks, or to papers of which the first
listed author is Snooks, or to papers of which he is deemed the leadauthor because he is project director or most consistent author
in a series of project papers. The relevant papers are all listed
under Snooks in the bibliography. One or two sets of references use
project or organisation names as leads.

Literature coverage
Sparck Jones 1973a attempted to cover much of the relevant
literature for the period 1965-1970. This survey concentrates on
the more recent period 1968-1973. It is based chiefly on papers
published in the Journal of Documentation, the Journal of the ASIS,
and Information Storage and Retrieval, but chapters like that dealing
with question answering use material from other sources. I have also
exploited other surveys, notably the relevant chapters of the Annual
Review of Information Science and Technology (Cuadra 1966-). General
acknowledgements must be made to Cleverdon 1966, Coyaud 1966,Lancaster 1968b, 1972a, Salton 1968a and Sharp 1965.

Organisation of the review
Section I defines automatic indexing for the purposes of thereview and provides background on information retrieval and linguistics.
Section II lists the linguistic components of an information retrieval
system, from the point of view of syntax and semantics, and considers
other system components which may be expected to influence indexing

P.3

performance. Sections III and IV deal with syntax and semantics
respectively, describing syntactic and semantic approaches todocument analysis, description and searching and to index languages.
Section V examines 'indirect1 indexing, represented by the useof citations and by document clustering. Section VI considers
automatic indexing in operational mechanised retrieval systems,
both standard and 'non-standard1, the latter covering interactive
retrieval and the use of printed indexes. Section VII makes abrief survey of work in the related areas of automatic abstracting
and automatic fact retrieval. Section VIII summarises the main
evaluation experiments involving automatic indexing, with specialreference to the Smart project. Section IX attempts an overall
conclusion and recommendations for the future.

P.4