SUMMARY

The present report is the ninth in a series covering research
conducted at the Computation Laboratory of Harvard University in automatic
information storage and retrievalo

An initial version of the fully-automatic SMART document retrieval system was described in Information
Storage and Retrieval, Report No« 7, dated June 196lu

Search results andretrieval evaluations obtained with this running system were included in
ISR-8, dated December l$6l\0

The present report, ISR-9, is divided intotwo main parts: a new "extended" SMART system is described in Sec0 I through
XVII, and search results and evaluation procedures developed in conjunction
with the original system are reported in the remaining Sections XVIII throughXXIV,IThe organization of the original SMART system reflects two principal

purposes: early computer implementation during the design stages, and fastdictionary look-up and search algorithms*. Because of the emphasis on
speed, many of the original programs make use of internal (core) processing
only, and eschew external (tape) procedures as far as possibleeThe

internal processing is responsible in large part for a variety of size
restrictions which limit the application of the system to relatively small,experimental document collections (£00 documents, 1000 words per document,
1000 thesaurus classes).

In order to ensure the applicability of the SMART

xviI

procedures and evaluation techniques to more realistic situations, adecision was made to reprogram the system, and thereby to remove theoriginal limitations.
The extended system described in the present report can. process
document collections of over 250,000 documents, and admit over 2^0,000
different concept classes as document identifiers0

The permissible lengthof each document is effectively unrestricted, except for the limitation on
the total number of concepts which can be produced.

The size of the various

dictionaries used as intellectual aids is also effectively unrestricted.
Theoretically, the extended SMART system should thus be applicable to almost
all search conditions which can be expected to be of interest in the foreseeable future; in practice, the system may not of course be operationallyuseful in every situation because of the inherent time limitations.
A short progress report of the SMART project is presented in Section
I of this report by G. Salton. Also included in this section, is a list of
possible extensions of the system and future tasks which are presently underconsideration.

Sections II and III by Tom Evslin contain, respectively, ageneral, relatively nontechnical, description of the SMART system, and amore technical, detailed discussion of the extended SMART system.

SectionIV by Michael Lesk includes a detailed listing of all input specifications
which are applicable to the new systemG

xvii

The actual programming details of the system are described in
Sections V through XVII of the report*Sections V and VI by Mark Canecontain the efficient dictionary (thesaurus) look-up procedures incorporated
into the system, and the thesaurus set-up and updating proceduresG

SectionVII by George Shapiro covers the statistical phrase process which makes it
possible to group individual concepts assigned to documents into larger units.
Arthur Priver and Michael Lesk describe the statistical clustering proceduresin Section VIII; the clustering techniques are used to form concept, as wellas document, groups, based on statistical co-occurrence criteria^
Analysis procedures using syntactic relations between terms arecovered in Sections IX through XII*The formats and updating procedures ofthe syntactic "criterion" tree file are described in Section IX by Guy
Hochgesang.

The main syntactic procedures, including the methods used forthe recognition of syntactic phrases are examined in. Section X by Alan Lemmon.
In Section XI, James Prowse covers the procedures used for the analysis ofincomplete English sentences, such as titles of documents0Finally, Section

XII by Michael Razar describes the tree matching algorithm used to compare
syntactically analyzed English text with the pre-stored dictionary of"criterion" trees.
The methods used to correlate analyzed documents and search requests
are discussed in Section XIII by Tom Evslin and Guy Hochgesang0Section XIV

by Mark Cane and George Shapiro covers the term-term and document-document
correlations used as a part of many of the statistical association procedures•
The construction and utilization of the concept hierarchy, including the list
processing methods available to perform all hierarchical expansion, arexviii

described in Section XV by Michael Razar and George Shapiroe

Output formats

are similarly described in Section XVI by Guy Hochgesang9 and the automatic
evaluation routines which terminate a SMART run are detailed in Section XVIIby Michael Lesk,
The remaining sections of the report describe evaluation methods andresults obtained during the first half of 196£0

The programs used to evaluate

the iterative search process incorporated into SMART are covered in Section
XVIII by Michael Lesk.

Section XIX by Joseph Rocchio examines the results

obtain by performing a number of iterative searches^ using the so-called"merged" methods0

In the fall of 196k 9 a number of students registered in a

graduate course at Harvard were asked to submit search request to the SMART
system without knowledge of systems operations; the results from thisexperiment are described in Section XX by Claudine Harris0
A number of special problems concerning retrieval systems evaluation
are covered by Joseph Rocchio in Section XXI0

Sylvia Sillers in Section XXII

reports on work performed to determine a reasonable cut-off procedure which
could distinguish retrieved from nonretrieved documentsc

Finally, a varietyof iterative, user-controlled search procedures are described in Section. XXIII
by Joseph Rocchio, and in Section XXIV by Joseph Rocchio and Gerard Salton0
Preliminary results indicate that these new iterative search methods are
generally more powerful than simple one-shot procedures,are planned in this area,

Additional experminents