II-l

II. Operating Instructions for the SMART Text Processingand Document Retrieval System
M. E. Leak

1.

Introduction
SMART is a fully automatic programming system designed for the analysis

of English text and for the processing of information requests. The SMART
system can also be used to perform complete studies of information
retrieval systems, vocabulary and dictionary investigations, and manyother linguistic and classificatory studies. Among the features of thesystem which are available for purposes of content analysis are: thesaurus
lookup, phrase searching methods (including complete syntactic analysis),
hierarchical processing, statistical association methods, and title/abstract/
text differentiation.

Intermediate and final results are available inconvenient form for detailed study, or alternatively, a complete retrieval
technique may be evaluated automatically. A full description of the methods
and programs used may be found in [l]; shorter writeups are available in[2], [3], and[k].

The system described here supersedes the operating system describeda year and a half ago in [5] and [6].

The basic differences between new

and old systems are a widely expanded system capacity and faster processing
speeds. In future years, it is hoped to experiment with time-sharing
environments, and additional processing algorithms are being developed.
Since the programs have reached a state of semi-stability, however, newoperating instructions are being prepared.

n-2
It is assumed that the reader has a general understanding of the
purpose of the SHfcRT system.
background.

Reference to [2] or [5] provide thisThe main purpose of the present section is to replace theobsolete operating instructions in [6].
1.1* Processing Sumnary
SHIRT accepts as Input natural language English text in a foim
close to normal typing.

Input is basically of two forms: requests9 or

queries for information; and documents, or individual units of a collection of English articles, or abstracts, which are compared with therequests.
The search procedure makes use of a flexible data representation
system in which eabh document is analyzed into a "concept vector".

The

concept vector consists of a list of "concepts11 (each with a weight) thatare associated with the document.

If each possible concept is imagined asa dimension, or a direction in a very-high dimensional space, the "concept
vector" representation of a document is now equivalent to an n-dimensional
vector; a concept vector is thus readily compared with other, similar
representations by simple correlation procedures.
The meaning of a "concept" can differ greatly from run to run.

Aconcept may be simply an English work; or it msy be a set of English words;
or it may be a phrase or set of phrases; or it may be a node label in ahierarchical classification system; or any combination of the above.
Provision is also made for future inclusion of other types of concepts;
e.g., concepts derived trm

an author's name or from a journal citation.

In short, a concept may represent virtually any feature of the text of adocument which reflects the document content.

II-3
The analysis of English language text for the detection of concepts
is performed without human intervention*

Text is analyzed by a set ofprograms which operate in conjunction with two types of inputs:a) A set of dictionaries, grammars, and hierarchies specifying therelations between properties of the input English text and of
the concepts. The simplest possible dictionary, for example,
associates with each English word a distinct concept. A moresophisticated dictionary, consisting in fact of a word thesaurus,
will define a many-to-many mapping of English word stems into
the concept vector space in which seme words are isolated ascommon words, some are considered ambiguous and resolved into
their various possible meanings, and seme are combined withsynonymous terms.
b) A set of specifications which describe in what way the various
content analysis programs are to be applied and which dictionaries
are to be used.

Specifications are used by the program to decide

what documents are to be processed, what dictionaries should be
used, what algorithms for associating concepts with documents
should be employed, how much weight each contribution to the
document representation should be given, what procedures should
be utilized to compare requests with documents, and what output
is desired from the run.
Once the text analysis has been performed, and concept vectors areavailable for all documents and all requests that have entered into thesystem, the programs proceed by comparing the requests with the documents
to determine which documents are to be identified as "answers11 to therequests.

This list may then be compared with a "correctH list, obtained

as a result of a manual operation and supplied by the programmer, andvarious evaluation measures may be computed automatically.

II-^
1.2. Operating Programs
The programs available fail into three categories:a) Retrieval system programs consisting of a large, connected set ofprograms which perform all the steps described in part 1.1.
These programs use a set of dictionaries to operate on acollection of English text,

and produce answers to requests,

and evaluations of processing methods.
b)

Dictionary preparation programs consisting of a smaller, butalso connected set of programs which prepare the various tapes
required by the program (a) •c) Miscellaneous programs to perform various useful functions outside the main system.
A description of the instructions for running programs (a) is given
in parts 2, 3, and k of these instructions*

Programs (b) can be run using

the instructions in part 5, and programs (c) are discussed in part 6.

2.

Basic Operating Procedures
SMART operates under a normal FORTRAN II monitor system. As alreadymentioned, the programs require a set of instructions, a library tape, anda set of data. The Usual procedure is to supply the programs on one tape,
the library on a second tape, and the data on another private tape and/oron the monitor input tape. In addition, up to seven scratch tapes may beneeded. Any data for the monitor input tape follows the run specifications.
2.1* Run outline
A complete processing run proceeds as follows:a) Monitor control cards are supplied to sign on to the computer,II-5
to instruct the operator to mount necessary tapes, and to transfer
control to a small program called CLCHN which calls in the SMART
programs from the program tape.
b)

The instructions for the run are read from the monitor input tapeand decoded*c)

The English language input text is read and analyzed.

This includes

a lookup of the text in a thesaurus, and an optional lookup ina statistical phrase dictionary and a syntactic phrase dictionary.
Printouts of the original text and the words missing from thedictionaries may also be performed.
d)Partial concept vectors are formed for these documents.

These

partial vectors contain all concepts derived from individual
words and phrases within each document.

The concepts appear in

their original uncombined form.
e)Partial concept vectors from documents looked up in this run,and concept vectors from documents looked up in preceding runs
are collated, and the weighting schemes specified in the instructions are applied to produce concept vectors in the proper form
for correlation.

If desired, the detailed results of the weighting

may be printed.
f) The concept vectors may now be expanded by means of statisticalconcept-concept associations, or through a hierarchy associatedwith the thesaurus. Various options may control the type of
expansion, and may specify the documents to be expanded.
g) The request vectors are compared with the document vectors using
the correlation procedure chosen by the user. All documents inthe collection are ranked with respect to each request.
h)

The highest ranking documents are printed out for each request.
This is the basic output of the simulated retrieval system.

i) The document rank list and the highest ranking documents are
printed and compared with the list of documents previously designated as relevant to the request.

Measures are computed whichII-6

indicate the success or failure of the run, relative to other runs.
The instructions to be used to regulate these procedures are describedin part 3; the data needed by the programs are discussed in part k*
2.2.

Tape Setup

The tapes used by SMART are named AONE, ATWO, ATHREE, . •. AEIGHT, BONE,
BHWO, BTHREB, ..., BSDC. By means of two internal tables, these names aretranslated first into logical tape numbers and then into physical tape unit
addresses.

This last translation takes place using the operating system

table (IDU), and SMART is therefore adaptable to almost any tape assignmentof the individual monitor system.

To aid in running SMART under a given

monitor, the first table (which translates SMART tape names to logicalnumbers) is variable at object time through the normal specification system.
Unless altered in this nay, the SMART tapes correspond to the 709^ tapedrives with the same names under the (10U) table of the Harvard monitor(see Table 1 ) . Because of this convention, the distinction between SMART
tape AFIVE (for example) and the physical drive A5 will not be made in thiswriteup, although the correspondence is indirect.
SMART assumes that input will be on A2 (actually ATWO) in accordancewith the practice in use for most FMS operating systems. Output print isplaced on A3; punching is done through the offline punch tape B*t. The
SMART program tape is placed on B3; it is a normal FM5 chain tape (seepart 5*3)*

The library tape is mounted on unit B5; any input data tapegoes on A6. All remaining tapes (Ak, A5, A7, A8, Bl, B2, B 6 ) are used as
scratch tapes. AU is needed for any lookup, and is also used if ANSWER
MEDIUM or ANSWER LONG is requested.

A5 is needed for the lookup, and forII-7
any correlations or expansions* A7 is needed for correlations or expansions,or for syntactic processing. A8 is needed only for concept-concept orhierarchical expansions. Bl is needed for expansions sad for syntactic
processing*B2 is needed for correlations, syntactic processing, andexpansions* B6 is needed for correlations, phrase lookup, and expansions.
If a tape is not needed by the specified process, it need not be mounted.
2.3. Input Deck Setup
Since the SMART system runs under a normal FORTRAN H monitor system(FMB), the usual deck setup is employed.

The job card (system accountingdata) appears first; then come operator messages and monitor control cards.
The program, consisting usually of a short routine which merely calls in
the main programs tram the program tape, follows. After this shortloading deck (labeled CLCHN) comes the monitor * DATA, card*

The data

which follow consist of the instruction cards and the data for the processing runs. The instructions are on A2.

The data, which usually consistof requests, texts, and relevance judgments, follow and may be on either
A2 or A6. Several processing runs may be stacked in the same IMS job,
as explained in the next section*

Alternatively, each processing run

may be made a separate FMB job*

This is often preferable if new tapes

must be mounted, since the SMART system makes no provisions for pausing
to allow the operators sufficient time to change the tapes*

3*

Specifications for the SMART Retrieval System
The first date required by the SMART programs are the specifications

which direct the processing to be performed during the computer run. These
specifications must immediately follow the monitor * DATA card.

II-8

Specification input always appears on the monitor input tape (FORTRAN
logical tape 5), even if SMART tape A W O is reassigned to another unit*
This exception to the tape addressing system is made to permit the program
to get started.

One computerjob may involve several sets of specifi-

cations, each of which Initiates a processing sequence*In this case,each set of specifications is followed by the data required by that setof instructions.

The AGAIN specification (see part 3*8) causes the system

to expect a second set of instructions and data following the first set.
Each section of a SMART processing run begins with the specifications
for that section.

The specifications are punched in columns 1-72 ofas many cards as are needed.

A specification consists of an alphabetic

word which identifies the specification, followed by its "value" if it
is not an "on/off" specification.

The names and meanings of all specifi-

cations, and their allowed values, are given in part 3* Whenever a
specification differs from the normal value for the item specified (seeTable 2 ) , the user must punch the specification and the value (if any)on the specification cards.

Specifications which are given their normal

value may be omitted, or may be punched to improve the clarity of thelisting, if the user so desires.
Any specification may be punched anywhere on a card. However, no
specification may be broken between two cards, nor may a specificationand its value appear on different cards. The specifications are separated by one or more blanks; the value of a specification should follow
the specification itself separated by one or more blanks. If the valueof a specification is a number, care must be taken to punch itcorrectly.

If the value of a specification is restricted to integers,

II-9
the number given oust not have a decimal point (2, not 2.0) • If the valueof a specification is allowed to be a fraction, it must be punched as a
FORTRAN floating-point number, i.e. it most have a decimal point (or theletter E).

For example, STATWT 1 is incorrect; the specification must

read STAOTT 1. or STATWT 1.0. In Table 2, x indicates a floating point
number and n indicates an integer.
If a specification appears more than once, only the last occurrenceis used. Specifications may occur in any order, except that the lastspecification (and only the last) must be either STOP or X.
3*1. Specifications Affecting Lookup
The SMART lookup uses a stem dictionary together with a suffix list
for an accurate determination of both semantic and syntacic roles. The
dictionary is stored in a semi-tree format, as is the suffix list. Detailsof the lookup may be found in references [ U ] and [12].
It should be noted that the lookup is sufficiently accurate so thatif, for example, IDES and HOP are two stems in the dictionary, HOPPING
will be found as HDP + P + DIG while H3P1NG will be found as H3H5 - E +ING; also, if EASY and EASE are stems, EASIER is found from EASY while
EASING is found from EASE.
The specifications associated with the lookup are:
ENGTXT

causes the Baglish text being looked tip to be printed;

NOTPND

causes the words not found in the dictionary to be printed;

PUNCH

causes document vectors to be punched out (with ghrases, if
searched for) in binary form. This saves time in futureruns since the lookup need not be repeated.

11-10
3.2.

Specifications Affecting Phrase Searching

Two basic phrase searching methods are available.

These are called

the statistical and syntactic phrase searching procedures*

Both use

pre-designed dictionaries of phrases which are searched against sentencesin the texts. Statistical phrase searching is the more common of thesemethods. The phrase dictionary consists of a set of pre-assigned wordpairs or word n-tuples (where n can be 2,3,1*,5 or 6 ) .

Each word n-tuple

has a "phrase concept number11 associated with it, that is used as theconcept number representing the whole phrase (as distinguished from theindividual concept numbers which give the components of the phrase)•
Every sentence is scanned for occurrences of phrase components, and when
all of the n^ phrase components have been found to occur at least m times
in the sentence, the phrase is considered to have occurred m times, anda weight is entered accordingly fog. the "phrase concept number" • The
method of writing the phrase dictionary on the library tape is given inpart 5*1»2.
It is noted that this is a rather imprecise type of search, since
the words searched for may occur anywhere in the sentence in any order.
For example, the sentence "Despite a second, larger order of textbooks,
approximately twenty-five percent of the students are still without them"will be considered as containing the phrase "second order approximation".
A more precise type of search is available through the "syntactic phrase
searching" procedure.
5.h.h)

This uses the "criterion tree" dictionary (seein which a complete syntactic/semantic/structural specificationof each phrase is given.
1)

If a syntactic phrase is to be detected,

the components of the phrase must have the proper semantic

n-ii
value, i.e. they most be assigned the proper word stem or theproper thesaurus category;2)

the components of the phrase must have the proper syntacticrole (e.g. "automatically translating" would not match "automatictranslation");3)

the components of the phrase must exhibit the proper syntactic
dependency relations with each other (e.g. "blind Venetian" would
not match "Venetian blind").

This phrase searching procedure makes use of the Kuno multiple-path
syntactic analyzer [7,8,9] to determine the syntactic roles and dependency
relations of the words in the sentence. This makes possible very accurate
searching, in that, for example the phrase "solid state" would not berecognized as appearing in "Seme authors whose other work is solid statethat information retrieval can be performed without accurate syntactic
analysis."
It should be noted that the syntactic phrase searcher is vexy slow
and also very erratic in its timing.

It is hoped that the use of a newversion of the Kuno syntactic analyzer will alleviate these problems [10].
The specifications to be used in the phrase searching procedureare:
STATER

which causes a statistical phrase search to be performed;

SYNTAX

which causes a syntactic phrase search to be performed;

NODBCO

causes a table of the detailed correspondences between
tree nodes and sentence words to be printed;

SYNAflAcauses the syntactic analyses of the text searched to beprinted.

11-12

An additional phrase searching method can be envisioned in which
statistical properties of words are analyzed to determine which words
appear to be occurring as phrases. SMART has in the past contained suchprocedures, but due to the poor results obtained with the algorithms
tried, these have not been implemented in the present version.

New

algorithms are under consideration, however, and specification CLWPR
has been maintained in the supervisor, indicating a call to a lookup
routine using statistically detected phrases. However, this specification is currently inoperative.

3.3. Vector Expansions by Means of Concept-Concept Correlation. Concept vectors obtained from the words and phrases in one document may be expanded based on statistical data obtained from an entire
document collection. In this way, local variations in individual document vocabularies can be corrected.

Procedures for these statisticalexpansions are available in SMART through the form of concept-concept
correlation.

This option involves the formation of a complete concept-

document occurrence matrix from the concept vectors of all the text
submitted in the run.

Different concepts are now correlated, to find out

which concepts appear to exhibit a similar occurrence pattern in the
documents.

This comparison is made on the basis of the rows of thetransposed concept document matrix in which each concept is represented
by rows of elements consisting of the numeric occurrences of the concepts
in the successive documents.
The correlation algorithms are as follows. Let the concept-document matrix be called A, so that A*, represents the number of occurrencesof concept 1 in document ^.

Then, the correlation coefficient r

is a

11-13measure of relation between the rows A . (all i) and A . (all i).

It hasthe value 0 for totally dissimilar rows, and 1 for identical rows. Two
different numerical algorithms are available for the computation of r;these are the cosine algorithm and the overlap algorithm. The cosine
algorithm is defined as follows:rpq

The overlap algorithm is defined as follows:
sum A^> sum A^))
XL

K

K

where min(x,y) • the numerically smaller of x and y. Note that bothmeasures are symmetric; i.e., r _ •r_ .
pq
qp
These correlations can now be subjected to a cutoff process defining$.. as follows: s _ » 1 if r
is greater than the cutoff value; • • 0
-ij
P4>
pq
M
otherwise. The matrix S with both rows and columns labeled by concepts
now identifies the concepts which have similar document environments;that is, the pairs of concepts which occur in the same documents have a
one at the intersection of their row and column.
This process can now be used for the expansion of the document vector
by augmenting all concepts by the list of concepts with similar environments.
The similarity matrix S can also be used as the starting point for
further correlations, however. Some writers feel that the truly significant
question is not "which concepts have similar document environments" but"vhich concepts have similar concept environments". This question requires
an additional correlation, a correlation of the matrix S, to identify theEWAconcepts which co-occur with similar concepts*

For example, if one isseeking to identify that "aeroplane" and "aircraft" represent the same
concept, one might deduce this not from the fact that they co-occur inthe same document, but that they each separately co-occur with words
such as "fusilage", "propeller", "aileron", .and the like. The SMART system
makes provision for iterated concept-concept correlation to any depth.
The specifications affecting this procedure are as follows:
CONCON n

the number "n" is the number of iterations of theconcept-concept correlation process.

"CONCON 1"

specifies simple first-order correlation;
MDDECC a

a is either "COS" or "OVLAP" to specify cosine or overlap correlation for the first concept-concept correlation;

MDDE2C a

a is either COS or OVLAP to specify the oorrelation modefor concept-concept correlations after the first correlation, i.e., for the second and following iterations;

CUTCC xx is a number between 0 and 1 (0.6 is typical) to specify
the cutoff for the first concept-concept correlation;

CUT2C xx is a number between 0 and 1 specifying the cutoff forcorrelations after the first;

CONMIN nn specifies the lowest concept number which will becorrelated.

This specification is to be used primarily

with null dictionaries prepared by THES (see part 6.1)
in which the concepts are arranged by frequency, thus making
it possible to specify the lowest frequency word to becorrelated;
CONMAX nn specifies the highest concept number which will be correlated.

CONMIN and CONMAX are used because the statisticalprocedures are not accurate for words which occur either
very rarely or very frequently;

n-i5
EXPANDaa is either REQS, DOCS, or ALL and specifies whether
requests only, documents only, or all input is to be
expanded*3^*

Vector Expansion by Means of Concept Hierarchies

In addition to statistical means for expanding document vectors, one
may wish to expand vectors by means of a pre-assigned concept hierarchy*
Various types of relationships between concepts nay be specified and usedin expansion procedures.

The hierarchy consists of structured trees ofconcepts, in which each concept has a clearly defined "parent11, plus a
set of "brothers", and "sons". To designate looser connections, each texm
may be assigned a set of "cross-references".

Expansion involves the intro-

duction into the concept vector of the sons and/or brothers and/or father
and/or cross-references of each term in the original vector. The selectionof the method of expansion is by means of a weighting scheme, and isdiscussed in part 3*5*
To perform a hierarchical expansion the specification
given whereais either EXPAND or SHRINK.

HIER ais

EXPAND indicates that the

final vector is to contain the original vector plus the related concepts
from the hierarchy; SHRINK indicates that the initial vector is deleted
from the final vector, and only the expanded concepts are used.

For

example, if it is desired to generalize a vector, each concept could be
replaced by its parent using HIER SHRINK; HIER EXPAND would merely augmenteach concept with its parent.
EXPAND a

serves the same function as OONCON; ais either REQS,

DOCS, or ALL and indicates respectively that requests, documents, or all
input texts aire to be expanded.

II-163*5*

Vector Formation

The final vector is formed by combining all the concepts introduced
from all sources with appropriate weights. The weighting parameters arenumbers which are zero or greater (and nay be greater than 1), indicating
the weight associated with each unit occurrence of the given concept in
the specified class. The weighting parameters are:x

STEMWTx is the weight associated with word stem concepts*

Itshould be a FORTRAN floating point number (e.g. 1.0, not 1 ) ;x

STATWTx is the weight for statistical phrases*

STATPR must havebeen specified when lookup was done; otherwise thiscommand is ignored*

If STATPR was specified, phrases may

be ignored by giving STATWT 0.0;
SYNWT

x

00C0WT x

x is the weight associated with syntactic phrases;
x is the weight associated with concept-concept expansion
concepts.

The hierarchical weighting specifications are used to select themethod of hierarchical expansion*

Any number of expansions with arbitrary

weights may be performed simultaneously.

There are four weighting para-

meters, one for each possible mode of expansion*

If the weight for a

possible expansion mode is zero, that expansion is not performed*

If theweight for a possible expansion is not zero, the expansion is performed
and the concepts derived from it are given the indicated weight*
ROOTWT x

specifies the weight for expansion by parents;

BRANWT x

specifies the weight for expansion by brothers;

LSAFWT x

specifies the weight associated with expansion by sons;CROSWT x

specifies the weight associated with expansion by crossreferences*n-17
Title and body differentiation is also performed by manipulating theweights*

Usually, the weight associated with the body of the text isleft at 1*0 and the weight of the title adjusted.

However, to obtain

only title processing, the user specifies BODYWT 0.0 and TTTLWT 1.0.

BODYWTxx is the weight associated with conceptsftrcmthe bodyof the text.

TITLWTxx is the weight associated with concepts from the title.

These parameters apply to all sources of concepts (phrases, etc.),
including the expansion concepts (hierarchy and concept-concept).
LOGVBC

this specification causes all weights to be either zeroor one. Weighting proceeds normally, except that after
the computation of the weight, any nonzero weight is
set to 1.

PRNVEC

causes the vectors to be printed for each document in
the system.

If concon or hierarchy expansions were

specified, two printouts of vectors are made, one before
and one after the expansion. In each printout the sourceof each concept and weight is shewn in detail.
3*6. Request-Document Correlation
Once the final set of vectors is formed, the request vectors arecorrelated against the document vectors. The documents with high correlations are considered to be the computer-generated "answers" to the queries.
These may now be printed.

They can also be compared with a set of "relevant"documents, obtained from some other source. SMART can automatically compare
the relevant documents with its answers, and evaluate the performance ofthe computer system against the outside standard.

18
The specifications which control this process are:a

MDDERDthe correlation mode used for the request-document
correlation is taken as the cosine mode or overlap
mode, according as a is "COS" or "OVLAP" respectively*
Cosine is the normal mode;

CUTRD xthe cutoff for request-document correlation is taken
a s x , a number between 0.0 and 1.0.

Woznal setting is

0.35;
ANSWERa

SMART can print out the answers to requests in threeformats.

If a is "SHDRT" twelve-character identifiers

are used for both requests and documents.

If a is"MEDIUM", the full text of the request is used, and oneline is used for the document identifier*If a is "LONG"

the complete request and the complete document citations
are printed for each answer*

These formats assume, ofcourse, that the necessary information is supplied witheach document at read-in time (^.1);
SCORES

this specification causes the automatic evaluation procedure
to be performed. A list of relevant documents must be
provided, according to the format described in (^A).

The output provided by the SCORES specification includes the following:a)for each request, the fifteen documents with the highest
correlations, and the values of those correlations;b)for each request, the relevant documents, their ranks in thecorrelation list, and their correlations;c)for each request, the normalized and un-normalized recall andprecision measures;d)the averages of the recall and precision measures aver
the requests used in this computer run;all ofn-19
e)

a deck of the rank lists for each request, punched in thecorrect format for the MORVAL evaluation program (part 6.2).

3*7*

Document-Document Expansion

After performing the initial request-document correlation, it ispossible to augment the list of answers through a correlation of the
documents with each other. Documents which exhibit a high correlation
with documents already identified as "answers" can thus be retrieved.
The specifications used in this process are:
DOCDOC

this specification indicates that a document-document
correlation is to be performed;

MODEDDaselects the correlation mode for document-document
correlation as COS or OVLAP; COS is the normal mode;

CUTDD xsets the cutoff for document-document correlation tox, a number between 0.0 and 1.0.

The normal value is

0.5*
3*8. Other Specifications
DOCTAP

this specification indicates that looked-up documents
appear on tape P£, in the format prepared by the MACTAP
program (part 5*2). These documents will be read by the
SMART programs, and are used as if they were includedin the normal input tape;

AONE nthis specification sets the SMART tape AONB to FORTRAN
logical tape number n.

Normally n • 1;

ATWO, ..., BSIX
these specifications change the other tape specifications.
Normal settings are shown in Table 1;
AGAIN

this specification indicates that another set of
specifications follows this run on the input tape.

SMART11-20proceeds to the next set after finishing the present run,instead of returning control to the monitor system;
PAGE nthis specification sets the initial page number to n.
The normal setting is n « 1;

STOP
or X

these synonymous specifications both indicate the end of
the specification list*

One of the two should be the lastspecification on the input cards; neither should occur anywhere else in the specification list.
A summary of the specifications is given in Table 2.

k.

Data Input
After the specifications are submitted, the user must provide the data

for the retrieval processing*This consists of requests, documents, andrelevance judgments, in a typical run. They .may be introduced in either
English or in looked-up binary form, and documents may be introduced on
A2, A6 or both.
Documents are divided into three categories, the requests, the texts(documents to be searched), and requests which are also texts.

This last

category consists of documents to be used as search requests, but which amy
also appear as answers to other search requests (normally, requests may notappear as answers to other requests).

The last named document type mightoccur if a user has identified one document in his field of interest andwishes to see more documents on the same subject. All of these types may
occur in natural language; in binary form, or both.
U.l.

Natural Language Documents

Natural language input must be submitted before binary-form documents.

n-2i

SMART
Name

NormalLogicalUnit

Harvard JMB
fhysical Tape
Unit

AONE

1

Al

Fortran monitor system tape

ATWO

5

A2

' Monitor input tape - card input

ATHREE

6

A3

Monitor print tape - printed output

AFOUR

k

A4

Scratch tape - document identifications

AFIVE

9

A5

Scratch tape - document concept vectors

ASK

U

A6

Document input

ASEVEN

12

A7

Scratch tape - used for sorting

AEIGHT

13

A8

Scratch tape - used for sorting

BONE

8

Bl

Scratch tape - vectors for expansion

BTWO

2

B2

Scratch tape - correlations

BTHREE

3

B3

Program chain tape

BPOUR

7

B^

Monitor punch tape - card output

BJTVE

10

B5

Library - thesaurus, phrases,hierarchy

BSK

16

Bo-

Scratch tape - used for sortinguse

Smart Tape Assignments. Table 1

The only incompatibility with normal FMS operation is that B3 is notavailable as a scratch tape* This prevents compilations or assemblies
when the program tape is used*11-22

Specification

.

Value

Meaning

NonnalValue

ENGTXT

Print texts of English input

off

NOTFND

Print words not found during
the dictionary lookup

off

PUNCH

Punch out looked up documents
in binary deck format

off

STATPR

Look up text for statisticalphrases as veil as word stems

off

SYNTAX

Search for criterion trees in
text

off

NDDBCO

Print out details of criterion
trees foundoff

SYNANA

Print syntactic analysis ofsentences analyzed

off

CONCONn

Perform concept-concept
correlations,iterating n times

off

MDDECCCOS
OVLAP

Choose mode of concept-concept
correlation, cosine or overlap,(first iteration)cos

M0DE2CCOS
OVLAP

Choose mode of later conceptconcept correlations

COS

CUTCC

X

Set cutoff for first conceptconcept correlations to x

0.60

CUT2C

X

Set cutoff for further conceptconcept correlations to x

0.50

CONMENn

Do not include in the conceptconcept correlations any
concept numbers below n

0

CONMAXn

Do not correlate in conceptconcept correlations concept
numbers above n

32000

I

Summary of Specifications
Table 2

II
NormalValue

Specification

Value

Meaning

EXPAND

REQS
DOCS
ALL

In concept-concept orhierarchical expansions,
expand the requests only, or
the documents only, or expand
everything

ALL

HIER

EXPANDSHRINK

Perform a hierarchicalalteration of concept vectors,
either expanding them, or"shrinking" them (replacing
the original concepts insteadof augmenting them)off

PCOCOR

Print the concept-concept correlations

off

LOGVEC

Do not compute any weights;
non-zero weights are taken
as 1

off

STEMWTX

x is the weight of theconcepts derived from word
stems

1.0

STATWTX

x is the weight attached toconcepts derived from statistical phrases (note: allweights are relative; Usually
STEMWT is left at 1.0, andother weights varied)

0.0

SYNWT

X

x is the weight attached toconcepts derived fromsyntactic phrases

0.0

OOCOWTX

x is the weight attached toconcepts derived from conceptconcept expansions

0.0

ROOWT

X

x is the weight attached toconcepts derived from hierarchical expansion by parents

0.0

BRANWTX

x is the weight attached toconcepts derived from hierarchical expansion by brothers

0.0

Table 2 (continued)

n-aUSpecification

Value

Meaning

NormalValue

LEAFWTX

x is the weight attached toconcepts derived from hierarchical expansion by sons

0.0

CROSWTX

x is the weight attached toconcepts derived from hierarchical expansion by crossreferences

0.0

BODYWTX

x is the weight attached toconcepts derived from thebody of the text (usuallyleft at 1.0, unless only
titles are to be processed,
in which case set to 0.0)

1.0

TITLWTX

x is the weight attached toconcepts derived from thetitle of the text

1.0

Concept vectors are printed
for all requests and documents

off

FRNVECMDDERDCOS
OVLAP

Choose mode of request-document correlation

COS

CUTRD

X

The cutoff for request-document correlation is set to

0.35

X

ANSWERSHORTMEDIUMLONG

DOCDOCMODEDD

1

COS
OVLAP

1

The answers to the requests
are printed in a condensed
format, a moderately detailed
format, or an expanded format,
respectively

OffThe list of answers is
expanded through documentdocument correlationoff

The mode of document-document
correlation is set to cosine
or overlap respectively

COS
— —*

Table 2 (continued)

Specification

Value

Meaning

Normal
Value

CUTDD

X

The cutoff for documentdocument correlation is set
to x

0.50

AONE

n

SMART tape AONE is set tological tape n

1

ATWO
•••
BSEC

n

SMART tapes ATWO, ... AEIGHT,
BONE, ..., BSIX are set tological units as specified(see Table 1)n

DOCTAP

Documents are read from ASIX
in binary form, as well asfrom A2.

off

SCORES

An automatic evaluation is
performed

off

AGAIN

Another set of specifications
is read and processed after
this run

off

STOP
X

End of specification list;
same as STOP

Table 2 (continued)

11-26
Within the natural language documents pure requests mist be submitted
first, requests which are also texts next, and texts last.
Each document is preceded by an identifying card containing a*

in column 1, a four character control word In cols 2-5, a blank incolumn 6, and a twelve character identifier in cols 7-18. Comments
and identification are punched in 19-72, as desired. The control word
is FIND for a pure request, LIKE for a request that is also a text, andTEXT for a pure text*

The document follows the identifying card (thecard with the asterisk in column 1) and continues on as many cards as
are needed.
Basically, input text to SMART resembles typescript. For example,
text may be punched anywhere in columns 1-72 of any number of cards. Any
number of consecutive blanks are equivalent to one blank* A blank isassumed between column 72 of one card and column 1 of the next card*
The major differences from typescript are as follows:...
a) An asterisk (*) in column 1, or two consecutive dollar signs
anywhere ($$) are end-of-text signals* These indicators should
not be used unless an end-of-text occurs* Normally, the end ofa text is indicated only by the * in column 1 of the control
card beginning the next text*b)

Periods not preceded by blanks are taken to be parts ofabbreviations. Thus, a period meant to indicate the end of
a sentence should be preceded by a blank.

c) SMART provides for the inclusion of up to 355 characters ofidentification for each text. This is used in the ANSWER LONG
output option (see part 3.6).

This identification should bepunched at the beginning of the text on cards with a single $in column 1*

If a text begins with cards that contain a $11-27in column .1, SMART and all auxiliary programs ignore theircontents, except that the first five (or fewer) such cards
cure collected as BCD identification. Usually, this consistsof the title, author, and publication source of the document.
The text proper is assumed to begin on the first card withouta $ in column 1, and any further cards with $ in column 1 aretreated as normal text. The identification may be omitted;
in this case, the first card of the text must not have a $in column 1.
d) The convention used to hyphenate a word between two cards isto place a minus sign (ll-punch) followed by a blank at theend of the first part of the word. SMART ignores theremainder of the card, throws away the minus sign, andcontinues from column 1 of the next card. For example, hyphenate is properly hyphenated. Note that this rule makes thefollowing construction illegal: "sub- and super-scripts." Also,
note that "normally hyphenated words which are broken between
two lines at the hyphen must have a double minus sign tobe properly recognized by the input programs; thus, Runge—
Kutta. Hyphenated words in the middle of a line are typed
normally; thus, Runge-Kutta.
e) Hardware restrictions require that only upper-case letters be
used. Special characters are treated as follows:
?
I
II

•
;
:

.QUE (preceded by a space),.EP (preceded by a space),/ (for both open and closed quotes. There should be
no space before a close quote or after an open
quote; similarly, parentheses and commas axe spaced
normally),-DASH (ll-punch minus sign followed by the word DASH,
if a dash is meant; for hyphens see d) above),t (8-4 punch, as in IBM Scientific character set H ) ,,. (not preceded by space),.. (preceded by space, and ends a sentence).

Any other conventions could also be used, if the dictionaries were11-28constructed accordingly; but the foregoing are simple equivalents forthe special characters*
The natural language documents must be on SMART tape ATWD.

Bulk

lookups from tape can be performed, however, by giving the specification
ATWO 11 (defining ATWO as physical A 6 ) . In this case no binary documents
can be submitted.

Alternatively, ATOO can be defined as 12 (A7) or 13(A8) in which case binary documents may still be submitted on A6 with a
DOCTAP specification.

Tape ATWO should not be moved to A*f, A?, Bl, B2,

or B6; these tapes are used in the lookup*^.2.

Binary Documents

The binary documents are submitted on A2 and/or A6 following the English
documents.

If there are any binary documents on A2, they must be precededwith a card containing *ONLY in cols 1-5 and a blank in column 6.
As in the case of English documents, the pure requests are submitted
first, followed by requests which are also texts, and finally .concluding
with the pure texts. The documents are preceded by *FIND, *LIKE, and*TEXT cards exactly as in part 4.1.
them from English documents.

The *ONLY card suffices to distinguish

The binary cards which make up each document

contain the identification punched at the beginning of the text, and theconcept vectors obtained at lookup time.

These documents are punched by

the PUNCH specification, after the lookup.

Thus, when binary documents

only are being submitted, the library tape need not be mounted unlesshierarchical expansion is required.

The binary documents may be usedwith any title or phrase weighting (if STATPR was in effect at lookuptime), and with any expansion options.

If it is desired to change the11-29thesaurus or phrase list, documents must be looked up again. The punchedout decks include the appropriate •FIHD, *LIKE or •TEXT card.
4.3. DOCTAPS
If the DOCTAP specification is given a tape of binary documents
should be mounted on A6. This tape contains documents looked up in an
earlier run, as is true also of the documents which follow the *ONLY
card on A2.

The tape on A6 is in a different, more compact, format,

however, which is written by the program MACTAP (6.2) from the binary
cards. The tape may contain several sets of looked-up documents* To
select the set to be used, a *FILB card should be provided.

The •FILE

card contains •FILE in columns 1-?, a blank in column 6, and either an
integer or an alphabetical file name in columns 7-18. If a file name isprovided, the set of binary documents identified with that name on the•NAME card produced when the tape was written (part 6.2) is read from A6.
If the integer n (punched anywhere in the field) is provided, the nth fileis read.

If no •FILE card is provided, the physically first file on A£

is used.

The •FILE card is placed on A2 following the specifications, but

preceding the *STOP card and the *ONLY card (if there is one).

The •FILE

card must not appear in the middle of an English document, however, nor
between a •TEXT, •FIND, or •LUCE card and the following document.
k.k.

Relevance Judgment Data

After all documents are read, the next input card must have •STOP incols 1-5 and a blank in column 6.

This card must occur exactly once on

A2 for each set of instructions, following all English and binary documents(if any).

It must also occur at the end of each set of documents on A6.

n-30
Following the *STOP card, the relevant document data needed for the
automatic evaluation system must be submitted, if SCORES is specified.
If SCOPES is not specified, relevant documents are omitted.

The cards

containing the relevance data are preceded and followed by a card with
*RELS in columns 1-5 and a blank in column 6.

For each request, a cardis punched with the request identifier in cols 1-12, and the number of
relevant documents in cols 13-16, right adjusted.

This card is followed

by a set of cards specifying the relevant documents. Each relevant
document identifier is punched in cols 1-12 of a separate card.

Columns

13-15 should be left blank, as these are reserved for a future extensionof the evaluation program to include degrees of relevance.
Relevance judgment data may be placed on either A2 or A6.

If

relevance judgments are given, they must occur as one continuous deck ofcards, surrounded by *REL& cards, placed after the *ST0P card.

Exactly

one set of relevance judgments should be supplied for each request(i.e., documents preceded by *FIND or *LIKE cards).

These relevancejudgments may be on either tape, in any order. All document and request
identifiers, however, must be spelled in exactly the same way on therelevance judgment cards as on the *FIND, *LIKE, and *TEXT cards. Relevancejudgments may be placed on A 6 , whether or not SCORES is requested when thetape is read, but if SCORES is not specified and AGAIN is specified, there
should be no relevance judgments on A2.
*t«5*

Other Instruction Cards

To place comments on the print tape, the *N0TE card is used.

This

card contains a * N O T E in columns 1-5, blank in column 6, and comments
to be printed and ignored in columns 6-72.

Printing is done off-line.

n-3i
To determine the time taken "by a program run, a card with *TIME
in columns 1-5 and a blank in column 6 is placed on A2. When thiscard is read, the elapsed time is printed. The printer clock (IBM RPQ
78051*) is required if the timing printout is to operate.
A list of all the SMART instruction cards with a * in column 1 isgiven in Table 3.

5. Tape Preparation Programs
SMART retrieval runs make use of a set of pre-written tapes, toavoid inconveniently ljarge input decks. There is no loss of flexibility
or capacity in the use of tapes. Up to three special tapes are used: thelibrary tape, the document tape, and the program tape.
The program tape contains the binary programs used by the SMART system.
It is mounted on B3.

The document tape, on A6, contains looked-up

document collections to be processed. The library tape, which goes on B5,
contains the thesaurus, phrase dictionaries, granmar, and hierarchy.
5.1. Writing a New Library Tape
The library tape contains six files. These are the thesaurus, thestatistical phrase list, the syntactic suffix list, the English grammar,
the criterion trees (syntactic phrase list), and the hierarchy. A set ofprograms in two chain links (numbered 101 and 102) is used to write alibrary tape. The programs in link 101 update the thesaurus file; theprograms in link 102 write the remainder of the tape. Link 101 automatically calls link 102. The programmer may omit link 102, however, if
nothing but a thesaurus is desired on the library tape (this is usually
the case with null dictionaries).

32

Cols 1-6

Cols 7-18

Meaning*FIND

Request name

Precedes a pure request for thesystem•LIKE

Document name

Precedes a text which is also arequest; i.e., a document "which
is both searched for and searched
on

•TEXT

Document name

Precedes a pure text, i.e. a
document which is only searched
for

*STOP

Anything

End of documents submitted on this
tape

*RELS

Anything

Precedes and follows deck of
relevance judgments•NOTE

Anything

Comments*TIME

Anything

Finds out the time

*ONLY

Anything

Indicates end of English language
documents and beginning of binary
documents•PILE

Collection
name or
integer

Identifies which collection of
documents on A6 is to be read into the system

SMART Instruction Cards
Table 3

11-33The new library tape is written on B5. If an old tape is being
updated, it is placed on A6.

New data is submitted on cards, on A2,

except for a new grammar, which is presented as a tape on A5»
5.1.1. Thesaurus and Suffix List Formation
The thesaurus is the first file on B5 to be written*

It contains asuffix list used in the lookup, followed by the thesaurus records themselves*
The thesaurus and the suffix list may be introduced from cards (on A2), orfrom an older library tape on A6*
The first data card for the update programs specifies whether thethesaurus and suffix list are on A2 or A6*»

This card contains a word,left-adjusted in columns 1-6, and followed by trailing blanks until column
7, which identifies those data which are on tape A2. If this word is BOTH,
the thesaurus and suffix list are assumed to be on A2, and tape A6 is not
read*

If it is desired to skip over (without reading) a thesaurus file on

the old library tape, columns 7*12 of the BOTH card should contain theword SPACE followed by a blank*

If the suffix list is being submitted on

cards, while the thesaurus is to be copied from A6, columns 1-6 of thefirst card should contain the word SUFFIX.

The word THES identifies thethesaurus as being on cards, while the suffix list is taken from the oldtape. If both the thesaurus and the suffix list are on the old librarytape, COPY is given as the control word*
If the first data card contains either SUFFIX or BOTH, the next data
deck must be the suffix list*

Each suffix is punched on a separate card,

beginning in column 1, and followed by blanks. Each suffix should be
assigned a distinct sequence number, less than 256, which is punchedrigjit-adjusted in columns 13-15 • The suffix list ends with a card con-II-31*
taining ZZZZZZ in columns 1-6.

The suffix list may appear in any order,

provided that the first suffix begins with "e".
If the thesaurus is introduced from cards (control word BOTH or THES),
it follows the suffix list (if any) immediately.
list, it follows the first data card.

If there is no suffix

The thesaurus may contain any numberof words. Each word is punched on a separate card.

Since a suffixing

routine is available, only the word steins should be entered in the dictionary.

The suffixing routine will accept multiple suffixes, and also

recognizes that words may double their final consonant, change a final "y"to an "i", or drop a final "e" before adding a suffix [7]. This permits ahighly accurate lookup in which almost all derivatives of most English
words can be identified automatically [12]. Of course, the extremely
variant forms (such as "mice", "geese", etc.) must be entered in the dictionary as separate words. It is sometimes necessary to enter complete formsof words to preserve important distinctions. For example, "programmer"
is readily identified from "program" + "m" + "er", but if it is desired
to distinguish a programmer from a program, both forms should be entered.
The dictionary lookup procedure may always be used as a full paradigm
dictionary by entering every form of a word in the thesaurus, at theexpense of a great deal of keypunching. Words entered explicitly takeprecedence in the lookup over possible stem-and-suffix combinations.
Each stem to be entered in the dictionary is punched left-adjusted
in columns 1-2^ of a card, with trailing blanks.
With each word are associated between one and six concept numbers.
These are punched right-adjusted in five column

fields, starting incolumn 25 and extending to 5*t« Ike first concept number is punched in11-3525-29, the next in 3P-&,

then in 35-39, and so on. If there are lessthan six concept numbers, the remaining fields should be left blank*
No concept number should exceed 32767. Concept numbers above 32000(and the concept number o) are reserved for non-significant words. The
concept numbers 1-32000 are significant concepts, used in correlation
process.
Up to eight syntactic codes may also be punched for each stem. They
are punched in columns 55-78 in three-column numeric fields, right adjusted
within each field but using the leftmost columns first. Each syntacticcode is a number, less than 256. As currently implemented, these numbers
correspond to partial stem homographs for the Kuno syntactic analyzer.
The correspondence between syntactic code numbers and partial homographs
is shown in Table k.

The homograph is completed by combining the partial

stem homograph with the partial suffix homograph of the syntactic suffix
list (5*3*1) as explained in [13]*The word stems on the input cards should he in correct BCD alphabeticalorder. They must be correctly ordered by the first letter, as the programs
are currently arranged: the degree of order necessary depends on the sizeof the dictionary.

Bie thesaurus must end with a card containing a 0-5-8

multipunch in column 1, and *END* in cols 2-6.
5.1.2. Statistical Phrase Dictionary
The second file on the library tape contains the first of the phrase
dictionaries, the statistical phrase dictionary. The first card of thestatistical phrase deck (following the last card of the thesaurus update
section) is a control card specifying the use to be made of the old fileon tape A6.

Columns 1-6 of this card should be either KEFLAC, indicating

11-36
l
2
3
k
5
6
7

ADJ
ADK
ADL
ADM
ADN
AD0
ADP

8 ART
9 AUXC
10 AUXP
11 AUXS
12 AVI
13 AV2
Ik AV315 AVU
16 AV5
17 AV6
18 AV719 AV820 BBOP
100 V H P
101 III
102 VI1S
103 VI1C10U PI1
105 0I1S
106 RI1
107
108
109
160 VT4P

161 IT^
162 VT^S
163 VT4C

l6U PT^
165 GT^S
166 *T4
167
168
169

21 BEOS
22 BEOY
23 BGOSSk BID
25 BPO
26 BRO
27 CC028 en29 CMA
30 C01
31 C02
32 C0333 001*
3^ C0535 006
36 C07
37 C0838 CPR
39 001
ko 0110110 VT2P
111 112112 VI2S
113 VI2C
13* PI2
115 GI1S
116 RI1
117
118
119
170 VT5P
171 IT5
172 VT5S
173 VT5C
17^ PT5
175 GT5S
176 RT5
177
178
179

kl 0120
k2 0130
43 OT10
kk 0T20
k5 0T30
1*6 OT^O
k-J 0T50
U8 0T60
49 OT601
50 0T70
51 0T701
52 HAVC
53 HAVP
5k HAVS
55 HP156 HP357 HP^
58 HP559 HPP
60 HVGS
120 VI3P
121 113
122 VI3S
123 VI3C
12fc PI3
125 GI3S
126 RI1
127
128
129
180 VT6P

181 n6
182 VT6S

183 VT6c
i8h P T 6
185 GT6S
186 RT6

187
188
189

61 HVI62 HVP
63 IAD
6k IAV
65 IPN
66 IP067 NAD
68 N04C
69 N0^S
70 N0UO
71 N0VO
72 NUMO
73 PRD

Jk PRE75 PRNC
76 PRNP
77 PRNS
78 PR079 PRZC
80

81 QUE
82 RL1
83 RL2&k RL385 RlA86 RL587 RL6
88 TITS89 T0IS
90 XC091 YC0
92 N0US93 VT1P
Sk VI1P
95 N0UP
96 N0UC
97
98
99

PRZP

130 VT1P
131 IT1
132 VT1S
133 VT1C
13^ PT1
135 GT1S
136 RT1
137
138
139
190 VT6P1
191 IT6 1
192 VT6S1
193 VT6C1
19^ PT6 1
195 GT^Sl
196 RT6 1
197
198
199

lUo VT2P
1U1 n 2
1U2 VT2S
lk3 VT2Cl l ^ pi2
1^5 GT2S
146 RT2

11*7
lk6

1U9
200 VT7P
201 EP7
202 VT7S
203 VT7C
20^ PT7
205 GT7S
206 RT7
207
208
209

150 VT3P
151 IT3
152 VT3S
153 VT3C
15^ PT3
155 GT3S
156 RT3
157
158
159
210 VT7P1
211 IT7 1
212 VT7S1
213 VT7C1
214 PT7 1
215 GT7S1
216 RT7 1

Partial Stem Homographs(See Mathematical Linguistics and AutomaticTranslation, Report No. HSP-9, Vol. I)
Table kNote: Slashed 0 stands for letter "Oh";Normal 0 stands for numeral zero.

n-37
that new cards on A2 replace the old file on A6 which is skipped, or
IGNORE,

indicating that new cards are on A2 and tape A6 is to be ignored,or COPY, indicating that the old phrase dictionary on AS is to be copied
unchanged to B5If REPLAC or IGNORE are specified, the control card must be followedby the cards for the phrase dictionary. Each phrase is punched on a
separate card. The concept number of the entire phrase is punched rightadjusted in columns 1-5*

The component concept numbers are punched right-

adjusted in six-column fields, using the leftmost fields first. Thus, thefirst component goes in columns 6-U, the next in 12-17, the third component(if any) in 18-23, and so on.
Statistical phrases must have two components, and may have up to six..
Non-significant concept numbers may be used; for example, one could findthe phrase "exclusive or" by searching for the components "exclusive" and"or" even if "or" had a non-significant number (as it usually does).
The statistical phrase dictionary may be in any order, but the lastcard mast contain N0M3R in columns 1-5 and a blank in column 6.
5.1*3. Syntactic Suffix List
The second file on the library tape contains the partial homographcodes associated with the suffixes. One control card is required by thelibrary update programs; it contains one of four codes in column 1-5 •The possible codes and their meaning are:
(ORIG

A deck of suffix cards follows in the format described
below. Generate a file of these suffixes, ignoring tape A6.

(DUFL

The file on A6 is copied unchanged to B5* No other cards

11-38axe submitted for this file.
(MERG

Merge the new suffixes which follow with the old list*(SKIP

Same function as (ORIG, except that P£ is spaced over theold file.

The suffix cards required for the (ORIG, (DUEL, and (MERG options arepunched as follows. Each card contains one suffix identified by its number,and the list of partial homographs for this suffix. The suffix number(associated with the suffix by the cards described in part 5-1-1) is punchedanywhere in columns 1-6, and the various possible partial or complete homograph codes are punched in six-column alphabetical fields beginning in column
7, using the leftmost fields first.
The partial homographs are alphabetic, with the unneeded characters
replaced by zeros. For example, the Kuno homograph for a plural noun is
IWUP. A noun stem, in the SMART dictionary, is given the syntactic code
number 70, corresponding to N0UO (note as before that 0 is "zero11 while 0
is the fifteenth letter of the alphabet).

The suffix "s" has a partial

homograph OOOPO. When stem and stiffix are combined, the complete homograph
N0UP is formed.

The stem RECTI, to take another example, has the syntax

number 0^3, which corresponds to 0T10. When combined with the suffix PIED,
which has the suffix homograph VOOC (among others), a correct homograph
VT1C is obtained for RECTIFIED.

The remaining homographs are obtained ftom

the other suffix homographs; they are ADJ (obtained completely from thesuffix) and PT1 (0T10 + POO 0 = PT1) •The partial homographs of some sample suffixes are shown in Table 5«
5.1.^. The Condensed Grammar File
The condensed grammar file follows the syntactic suffix list on the11-39Suffix

Homographs

ED

VOOCO POO 0 ADJ

S

NOUP VOOSOESNOUP VOOSO

LY

ADJ

ING

ROO 0 GOOSO NOVS ADJYNOVS ADJ

ION

NOUS

IONS

NOUP

OUS

ADJ

ER

NOVS ADK

EST

ADJ

NOVC

ENT

ADJ

NOVC

ERS

NOUP

NOVC

AVI

NOVC
AV6

NOUP

ABILITIES

1

Some Partial Suffix Homographs
Table 5

Il-k)
library tape.

It is used by the Kuno multiple-path syntactic analyzer.

This analyzer is discussed in [9]. The grammar tape itself is not actually
written by any program in the SMART package; a tape containing the grammar
is prepared by the analyzer programs operating independently.

This tapeis then copied onto the SMAJtT library tape.
The card following the syntactic phrase list directs the tape copy.
If it begins with CDPYA5 in columns 1-6, a new grammar tape on A5 replaces
the old grammar.

If it begins with C0PYA6, the old grammar is used.

If

UFDTA5 appears in cols 1-6, a new grammar is copied from A5 and the oldgrammar is skipped on A6.
Tl}e SMART programs have been run with a grammar modified to accept
noun phrases and prepositional phrases,

nils permits the syntactic analysisof titles [l^]. However, as noted above, the syntactic analysis routines
are generally in disuse because of timing problems, since the new, faster
analyzer is not yet available.
5.1.5•The Criterion Tree FileBie fifth file of the library tape consists of the criterion trees, aphrase dictionary in which phrases are defined by a complete set of structural/semantic/syntactic specifications.

The overall control cards recognized by

the update editing criterion tree routine are:
/COPY" in columns 1-5 > with an optional integer beginning in column 7*
The indicated number of trees are copied ftom A6 to B5-

If nonumber is given, the whole criterion tree file is copied.
/SKIP in columns 1-5, with an optional integer beginning in column 7*
The indicated number of trees on A6 are skipped (if no number isgiven, the whole file is skipped).

IL-hl/EDIT in columns 1-5, with an optional integer beginning in column
7*

The indicated number of trees are copied from A6 to B5

with the trees specified by the deletion requests (which follow
this card) removed from the file. These deletion requests arecards with right parentheses in columns 1 and 2, and with aserial number in columns 7-12 and/or a BCD identifier incolumns 13-18. Up to 30 of these delete requests may be given.
Any criterion tree on the tape which matches the informationon the delete card (either the identifier or the serial number,
if only one is specified, or both the identifier and the serial
number if both are given on the delete card) is removed fromthe file.
/WEOF in columns 1-5*This terminates the processing of the criterion

trees. The file on B5 is terminated and A6 is spaced over any
remaining trees. This should be the last control card affecting
the criterion trees.
/ADDL This specification causes trees to be read from A2, encoded as
required by the search programs, and written on B5»

The format

required for the specification of criterion trees is given below.
5«1«5«1*

Criterion Tree Input Format

Each definition begins on a new card. Basically, six pieces of information are required to define a criterion tree. These are:a) the tree name and serial number,b) the output concept numbers to be assigned to the sentence if
the tree is found,c) the place in the sentence where these concepts are attached,d) the dependency relations between nodes (the structure of thetree),e) the syntactic values associated with each node that must be
matched,11-42f)

the semantic values associated with each node, which must be
matched.

Normally, the data under c), d) and e) are supplied by reference toa standard "tree type". A table of tree "types" exists in the program,
represented in Table 6.

New types may be defined, however*

Trees may also

be defined completely on the input cards. In fact, one may simultaneously
define both trees and types with an input card.

Usually, however, only

the output concept number, the semantic data, the tree type, and the nameand serial number are assigned by the user. The format of the criterion
tree input cards, going from left to right, is as follows:a)columns 1-6 contain six BCD characters.

If this is the firstcard of a tree, this field contains the name of the tree.
is the BCD identifier used in the /EDIT control card.

This

If thisfield contains five blanks followed by an asterisk, this card
is a continuation card frcm the preceding card.

Column 1 should

not contain a /, except for the last input card of the criterion
tree deck, which must contain ////// in cols 1-6.
If a type, rather than a tree, is being defined, the BCD namefield is ignored.

However, it must be non-blank, non-zero,and satisfy all the other requirements as usual.
b)

the output concept number field(s) follow.

This may be emitted

if a type only, and no trees, are being defined.

The contentsof this field are the concept numbers to be used as output, andthe positions in the sentence (the nodes) where they are to beinserted.

More than one output concept number field is allowed;

each field, including the first output concept number field,
must be preceded by an equals sign (=). Each field is defined
as follows:i)

if the output concept number is a specific integer, it must be

II

1?*

1.

8.

2-6

(V)

3Q

10

2

^cJ(0, s)

i,r ^

a

(v)

]b(0)
11.

KX(s)

, ^ o (c)'< v)

k\>(5)
\

26

i6w)
12

6.

2Q*

l6

:r

<0

13-

/°^ >

/a(0, 3)sbw)

Standard Tree Types
Table 6
Solid lines indicate direct dependence; dotted lines indicate indirect
dependence. The asterisk denotes "key" nodes; i.e» the nodes to which
the output concept number is attached when a match is founds

II-W^
first in the field, and only one such integer is allowed.
For example, to specify 285 as the output concept number,
one writes * 285 for the output concept number field;
ii)

if no definite integer is given, at least one subfield
consisting of the letter "N" followed by an integer must
appear.

Each such integer identifies a node in the tree

whose concept number is used as an output concept number.
No nodes numbered above 15 may be referred to in this way;iii) normally, a pre-assigned "key" node in the tree receives
the output concept number.

In Table 6 this key node ismarked with a * for each type. If the user wishes toassign the output concept number to another node or nodes
he may do so by punching a period followed by the numberof the node to which the concept number is assigned.

Anynumber of periods followed by integers (the integers mustbe smaller than 15) may be punched. A period not followedby an integer suppresses the weight assignment to the key
node.

For example, if it is desired to change the assignment

from the key node to node number 3, the user would punch .3.
in this subfield;
iv)

normally, the output concept numbers are given equal weights,
assigned in such a way that the total weights of each phrase
are equal. If SYNWT 1.0 is specified, the total weight ofeach phrase will be equal to the weight of one word occurrence (assuming the normal STEM/T 1.0).

However, the user

may punch a subfield consisting of an asterisk followed by an
integer.

The output concept number of this field will thenbe assigned a weight specified by this integer. The integer
may be zero; it must be less than seven. Normally, weightingwith one output concept nuriber per phrase corresponds to an
integer specification of k.

To halve the weights, specify 35

to double them, 6; and to triple them, 7eliminates this phrase*

A zero specification

n-^5
Thus, to define an output concept number of 285, and to attach it
to node number 3 instead of the keynode, and also to double its
weight, the user punches «285«3»*6 as an output concept number
field. If, in addition, the concept number on node 2 is to be
moved onto nodes h and 5 with half weight, another output concept
number field would be added, the total reading:*285.3**6«N2.4.5-*3c) it is also necessary to supply the conditions specified foreach node. For each node exactly one node condition field exists,
except that conditions need not be specified for the last node(and adjacent nodes) if a definition using a tree type is employed,
and the syntactic and structural criteria supplied by the type are
all that is needed for these trailing nodes.
The first node condition field begins immediately after the output
concept number fields* Each node condition field is followed bya slash, (/) except for the last node condition field which isfollowed by a dollar sign ($).

If the tree is being definedaccording to a standard type, only the semantic information need
- be supplied in the node condition fields. If a type is being
defined, however, or if a tree is being defined without the use ofa type, all information must be supplied for each node. In thiscase, there must be exactly as many node condition fields as there
are nodes in the final tree.
If structural data is being specified, all but exactly one nodecondition field should begin with either an I or a D followed byan integer. The node whose condition field does not begin withan I or D is taken as the root of the criterion tree. A node whose
condition field begins with "In" must match a node whidh isdescended from (direct or indirect ancestor) the correspondent of
node number n. A node whose condition field begins with "Dn,fmust match a node which is the direct (first-generation) descendantof the node which matches node n.

"Descend" is understood in the11-46sense of "being syntactically dependent upon, according to thesyntactic analysis used for this sentence/1
The semantic (and syntactic, if needed) data now follow; or, if
this tree is being defined using a standard tree type, these data
begin the field.

The node condition field, except for the In

or Dn subfield, is composed of a set of "relations", each "relation"
being a set of one or more "relation generators". Each relation
generator is either a concept number or a syntactic role code.
A concept number, indicating a semantic condition, is punched as an
explicit integer. A syntactic role code, indicating a syntactic
condition, is punched as a letter enclosed by apostrophes. Thus,
the relation generator for concept number 285 is simply 285;
the relation generator for a verb is 'V.

The syntactic role

codes and their meanings are shown in Table 7.

If a tree node

is to match a sentence node, at least one relation generator ineach relation must correspond to a property of the word.

Relation

generators are punched separately by commas, and all the relation
generators in each relation are enclosed with parentheses. For
example, the node condition field I3(8,l6)(t0',tS') can only matcha word which:i)depends directly or indirectly on the word matching node
number 3;ii)

corresponds to either concept number 8 or concept number
16 in the thesaurus;iii)

functions either as an object or a subject in thesentence.

On the other hand, the condition field <8)(l6) would only matcha word which would correspond to both concept 8 and concept 16 in
the thesaurus*
The node condition fields correspond to nodes in sequence counting
ffcm left to rigfct. Thus, the node condition fields(26,78)/(95)/(22,105) indicate that node 1 must match a word inconcept 26 or 78; node 2 must match a word in concept 95; and nodeMeaning

Letter1

Declarative sentence

2

Interrogative sentence3

Imperative sentencekSubject clause

5

Object clause

6

Complement clause

7

Adjective clause

8

Adverbial clauseA

Adjective

C

Complement

D

AdverbE

Adverbial noun phrase

G

GerundM

Participle

0

ObjectP

HiraseR

Phrase or clause introducer(preposition or conjunction)

S

Subject

V

Verb

X

Auxiliary Verb+

Conjunction (and/or/but)

9

Comma

0

Periods

Question markSyntactic Role Codes
Table 7

Footnote to Table: see section I of Reference 8, page 127 (Table 25)

3 oust match a word with either concept 22 or 105. The structuraland syntactic data are supplied, as usual, from the tree type tobe specified later.
following the dollar sign ($) ending the node conditions fields,a final, miscellaneous field appears. This field specifies
whether trees or types are being defined, what types if amy arebeing used in the tree definitions, and what the serial number(s)of the output tree(s) are.
If trees, rather than types, are being defined, the dollar sign($) is followed with a set of tree specifications, separated bycommas.

Each specification consists of a type number, identifying

the standard type which supplies the syntactic and structural data,
and a serial number code.

The type number is simply an integer,

identifying the tree in Table 6.

It is followed immediately by

the serial number code, which is one of the following:i) +(plus sign) indicating a serial number one greaterthan the last serial number assigned;ii) *(asterisk) indicating a serial number equal to the last
serial number assigned;

iii)

/

(slash) followed by an integer, which assigns this, integer as a serial number.
If the syntactic and structural data are supplied with the tree
definition, instead of from a type, the type number in the tree
specification should be explicitly set to zero. Any number of tree
specifications may be given, separated by commas, to define several
trees at once. For example, one can simultaneously define trees
to represent "information retrieval is ..." and "retrieving
information by ..." in this manner.
If a new type is defined, this last field should end with an
integer followed by either "T" or "P". This declares the structureand syntactic conditions defined on this code to be a standard
tree type, numbered with the integer provided.

This rule applies

n-49
even if the integer was previously in use to identify anothertree; the old tree is lost in this case. If the letter "P" isused, in addition to specifying a definition valid for this run,a deck of binary cards is also punched which may later be added tothe deck of the subprogram TRECND in link 102 to permanently alter
the standard tree types. The "T" or "P" should be followed bya set of integers, separated by commas, which identify the "key"nodes of the new tree type. Usually there is only one key node.
e) the following additional format requirements should be noted.
Blanks are completely forbidden except in the BCD name field.
The occurrence of a blank ends the definition. To continue adefinition from one card to the next, a minus sign (-) should bepunched as the last non-blank character of the first card, andthe next card should be started with five blanks and an asterisk
in columns 1-6.

Punching of the tree definition should beresumed in column 7.
Consider the following examples. Assume that "information" isrepresented by concept Ilk,"data" by 53> "retrieval" 26, and"information retrieval" is to be defined concept 301. Note thattree type

1 is : l Q

and type 3 is:26(v) i C r

jCt3

O2 (0)

To define tree type 3, one could write:
TTCE3 D3(,VI)/I3(,0,)//$3T1
and to define type 1:
TYEEL /I1$1T1This tree would match all the following sentences if the appropriate
concept numbers are assigned to the nodes (numbering the trees 10
and 11, and naming them INFKET):
INFRET«30l(26)/(53,ll^)$l/lO,3+Information retrieval is useful,
Data retrieval is useful,11-50

They retrieved data by computer techniques,
They retrieved information by computer techniques.
To match "They retrieved information by computer'1 but assign the
output concept 301 to "information" instead of to "retrieved" onewritesINPRET*301.2*(26)/(53A1^)$3/10
To define tree type 3 and the original tree simultaneously, one
writes
INFRET*301D3( 'V ) (26)/l3(f 0 f ) (53,l^)/$0/lO,3T1
5-1*6.

Hierarchy

The hierarchy is punched in a comparatively simple format. Each
concept number is punched on a separate card, and represents a single
node in the hierarchy.
hierarchy.

Indenting is used to indicate descent in theMore exactly, the "level" of a node is defined as the numberof the column on the IBM card in which the units digit of the concept
number is punched.

The parent of each node is defined as the last prece-ding node on a lower level, i.e. with its units digit punched further tothe left.

If a node is furnished with cross-references to other nodes,

these are punched on the same card, to the right of the node number. Theremay be any number of cross-references, punched anywhere on the card
(separated by blanks). All punching is restricted to columns 1-70.

If

additional space is needed, any character may be punched in column 72.
This causes the next card to be interpreted as a continuation card.
It is esthetically pleasing, although not necessary* to punch allbrothers on the same level (in the same columns). For example, thefollowing structure (with node number 2 cross referenced to node 7 and viceII-51versa):

could be punched either as:

or as:
1

1
2

2 7

7

k

k

5
3
6
8
7 2

5
3
6
8
7

2

If the hierarchy consists of several bushes, rather than one connected
tree, bushes are separated by single blank cards. Two consecutive blankcards end the hierarchy.
5>2. The Document Tape
To avoid the necessity of submitting document collections on cards
for each retrieval run, a prewritten tape of binary documents can be used.
These tapes are written by a program called MACTAP.
MACTAP

The control cards forcontain asterisks (*) in column 1, and control words left-adjustedwith trailing blanks in columns 2-6. Data cards are submitted on A2 andthe tape is written on A6.
The first data card may be the control card *STAHT. This card isused if additional data are being added to a previously written documenttape. If it is omitted, it is assumed that a blank tape is mounted on A6.

11-52
Any number of sets of documents may be submitted in one run. Each setof documents is preceded by a *NAME card, which contains a twelve character
identifier in columns 7-l8.

This identifier represents the collection

name which is used in the search initiated by the *FELE card (see part U.3).
The documents are submitted on cards, in looked-up form.
usual:The order is thefirst any pure requests (identified by *FIND cards), then any

combination request/texts (identified by *LIKE cards), and finally texts,
identified by *TEXT cards. A *STOP card follows each set of documents. A
deck of relevance judgments, as in part k-.k, may be placed between *KELS
cards after the *STOP card.
After each set of documents and relevance judgments, another *NAME
card and another document collection may be added.

The end of the run ismarked by a *FINISH card.
5-3*

The Program Tape

The SMART program tape is a normal FM3 chain tape. To write it, thelinks to be included are submitted on binary cards, with a blank tape on B3.
The FMS loader will write a chain tape automatically, which may be saved
and reused.
Currently, SMART has 1^ chain links, as shown in Table 8.

Tape-writing

programs operate separately. All links need not be included on the program
tape, but if any links are omitted some options will be unavailable.

It isnecessary to resubmit the programs on cards and have the loader write a new
chain tape to make any changes in any of the programs.

6.

Auxiliary Programs for Use with the SMART System
Several programs, although not directly a part of the SMART system, areII

Function

Link No.

1

Lookup in thesaurus and statistical phrase
dictionary.
Printing of English texts

OJ

Printing of words not found in thesaurus

3

Formation of complete homographs (lookup insyntactic suffix list)kSyntactic analyzer, link 1; formation of binary
sentence tape5

Syntactic analyzer, link 2; analysis, formationof binary analysis tape6

Syntactic analyzer, link 3; formation of edited
analysis tape7

Criterion tree matching routine; lookup ofsyntactic phrases

8

Vector formation, combining all sources of concept
numbers, weighting of vectors; correlation ofrequests against documents

9

Concept-concept correlation; formation of conceptconcept expansion matrix

10

Hierarchical and concept-concept expansions;formation of new document vectors

11

Document-document correlation, and expansion

12

Sorting of request-document correlations

13

Printing of answers to requests

Ik

Evaluation of run

SMART Chain Links
Table 8

II-51*
frequently used for auxiliary functions.

These are described in the present

section.
6.1.

THES

THES is a program to form null dictionaries from a collection ofEnglish text.
by-product.

It also prints out frequency counts and listings as a
It includes a suffixing routine.

THES requires an A2 one control card.
integers.

This card contains three

The first integer, punched right-adjusted in columns 1-5,

specifies the maximum number of concepts (words) to be included in the null
dictionary.

The second integer, punched right-adjusted in columns 6-10,

specifies the minimum number of occurrences in the collection that any wordin the dictionary may be expected to exhibit.

These two numbers pennit theuser to control the size of the null dictionary.

For a complete null dictio-

nary, the first number should be very large and the second number should be
1.

The third number is punched right-adjusted in columns 11-13 *&<* specifies

the tape on which the document collection is located.

If this field isblank, tape 5 (the input tape) is assumed.
The collection is placed on the specified tape in normal SMART format(4.1), with documents preceded by *TE3CT cards only.
cards should not be used.

*FIND cards, and *LIKE

Of course, since no searches are made during

thesaurus construction, the requests may be labeled *TEXT, without problems,
if it is desired to include them in the counts. A *ST0P card ends thecollection.
6.2.

MDRVALM3RVAL is a program to compute additional evaluation data for a set of

11-55
requests and methods*

MORVAL accepts as input a set of decks of rank

positions punched by the SCORES specification at run time. The program
produces for each method and each request the top fifteen documents, andthe ranks of the relevant documents*

All documents are abbreviated tothree-character identifiers to permit a compact output format*

Thus, touse MDRVAL properly the twelve-character identifiers used for the documents
should be chosen so that the first three of these characters are adequate
to uniquely specify a document*

MDRVAL also produces normalized andunnormalized evaluation measures, and recall-precision graphs of two types(averaged over requests, and cunmlated-type graphs)*

MDRVAL is also capableof evaluating "merged methods", where the results of several methods are
merged*MDRVAL is described in reference [15 3, where complete instructions
for using the program are given*
6*3*

SOCCER

SOCCER is a concordance generating program.

All occurrences of eachword in an input document collection can be listed with their context*
Such a concordance greatly simplifies many tasks of word use analysis*
SOCCER contains full facilities for suppressing concordances on unwanted
words; or restricting the concordance to selected words.

It also provides

a variety of statistics about the collection of text processed.
Because of the simpler nature of the processing, SOCCER will function
with virtually any input format for the text.

In particular, the SMART

formats are satisfactory, and are convenient for some of the format optionsof SOCCER.
SOCCER is a relatively fast program; a concordance prepared for a11-56collection of 110,000 words took about forty minutes. SOCCER is fullydescribed and complete instructions given for use in [16]. It has
been distributed by SHARE as distribution no. 3*1-07.

7.

A Sample Input Deck
Note:

Normal typing represents comments; UPPER CASE UNDERLINED

represents BCD input cards; lover case underlined represents binary input
cards.
*

JOB,630201,5MIN,5000,LESKThis is the Harvard FMS job card format.

*

XEO
Monitor control card.

»•

PLEASE MOUNT TAPES ...

*SALTON 25 ON B 3 RING OUT 800 BPI*SALTON hk ON A6 RING OUT 800 BPI*

PAUSE (THANK YOU)
These cards instruct the operator to mount the program tape, Salton

25, and the document tape, Salton hk.
clchn - two card binary programThis program is equivalent to the compiled version of the Fortranstatement

CALL CHAIN (1,3)-

program tape.

It brings in the SMART programs f*am the11-57*

DATA

Monitor control card.
DOCTAP ANSWER SHORT SCORES MODERD COS CUTRD 0.3$ PRNVECSTATWT 2,0 X
These instructions cause the programs toa) read input data from a document tape mounted on A6;b) to print only twelve-character identifiers for the request anddocuments when answers are obtained;c) prepare an automatic evaluation;d) define the correlation mode for request-document correlation
as the cosine mode;e) choose the cutoff for request-document correlation as 0.35;f) print out document and request vectors during the run;g) weight concepts derived from statistical phrases twice as heavily
as concepts derived from word stems;h)end the specification list with the X specification.

*FILE ABSTR THESThe file to be read from the document tape is named ABSTR THES. At
input time to MACTAP, this tape looks as follows:*NAME ABSTR THES
*FIND QA1INF0RM
binary looked up deck for query QA1INF0RM*FIND QA2

11-58binary deck for query QA2 ...
•••

«...

•••
*FIND QB16
a total of 35 questions are placed on the tape
*TEXT 01X33C* •..
binary deck for the first text, punched when looked up•••*TEXT 82X1106 ...
binary deck for the last text
*STOP
*RELS
QA1INP0RM
55XRELEVAOT
23XRELEVANT...

10

The relevant documents for the first question are listed
QA2
the request name for the second request and then its relevant
documents are submitted, and so forth.
...
*RELS
*NAME ABSTR NULL
the next set of documents for MACTAP is provided here, after all
relevant documents are introduced for the requests in the firstcollection.
•STOP
This card appears on A2. It indicates that no documents were submitted

n-59
on A2, only the set on PJS is to be processed*end of file
This marks the end of the job for the monitor system. Total cards
submitted: BCD 12, binary 2, total Ik.

Most of the production runs madewith SMART are of this size.

8.

Miscellaneous
SMART is frequently changed and revised; thus writeups may differ.

The current writeup supersedes the ones included in reports ISR-7 andISR-8 completely, and it supersedes report ISR-9 wherever the present
writeup differs from the one in ISR-9*

SMART is written in Fortran II

and in FAP. With all its subroutines, the programs include approximately
30,000 souroe cards and 5,000 binary cards.
8.1.

Size Limits

The thesaurus may consist of any number of English stems; however,
only 32,000 significant concepts are allowed.
The program can address up to 250,000 documents; however, the capacityof the intermediate tapes will be exceeded before this point. Assumingthat MACTAP is used to prepare the input document tape, about 25,000-100,000
documents (depending on their length) is a probable practical limit. If
one insists on submitting documents on cards, about 1500-2000 documents is
fl.il will fit. The only truly significant limitation may be expected to be
the fact that only fifty requests can be processed in one computer run.
8.2. Timing
The following timing estimates are approximations derived from our11-60experience on a batch-processing 709^ !•Since SMART is an experimental

system, no great effort was spent on optimizing the object code for speed,and considerable improvements could be made in many of the programs.
Starting:

Mounting two tapes, signing on, reading the specifications,etc. requires about two minutes. This represents mostly tape mounting time.
To look up £ words in a dictionary of g stems takes

Lookup:roughly pq.10-7minutes. Statistical phrase searching of £ words in a listof g phrases takes about ipq*lo"^ minutes•

Syntactic timing is exceedingly

irregular with the old syntactic programs, and is effectively so slow that
nothing useful can be accomplished in a reasonable amount of time.

It ishoped that seme syntactic analysis runs can be performed with a new revised
analyzer to be distributed shortly*
Correlations: To correlate £ requests against q documents and thensort the correlations, and evaluate, on the order of ipq^lO*"3 minutes areused up.

Present experimental data exist only for the range of £ between 10

and 50, and g between 50 and ^00; these estimates should not be trusted far
outside this range.
Concept-concept correlations:lif £ concepts are involved (i.e. p » CONMAX 1 2_lj.

CONMIN) the first correlation takes about £p *10" minutes. Further iterations
should be fast, assinning reasonable cutoffs.
Hierarchy: . Most of the time is spent in tape shuffling, requiring
about five minutes for collections of about 50,000 words.
9«

Acknowledgments
The programs described here were written by Mark Cane, Tom Evslin,

Guy Hochgesang, Alan Lemmon, Michael Razar, George Shapiro, and the author.

n-6i
Other members of the SMART project have been Margaret Engel, Claudine Harris,
Richard LeSchack, Edward Nelson, Joseph Rocchio and Edward Sussenguth.
This project was supported by the National Science Foundation under grants
GN-82. GN-36O, and GN-^95-

The assistance of Prof. Susumu Kunofs researchgroup, Dr. Owen Gingerich of the Smithsonian Astrophysical Observatory,
and the operations staff of the Harvard Computing Center is gratefully
acknowledged*

II-62

References[l]

Information Storsige and Retrieval, Report No. ISR-9 to the National
Science Foundation, Harvard Computation Laboratory, August 1965.

[2]

G. Salton and M. E. Lesk, The SMART Automatic Document Retrieval
System, Communications of the ACM, Vol. 8, No. 6, June 1965*[3]

G. Salton, ^ogress in Automatic Information Retrieval, IEEE Spectrum,Vol. 2, No. 8, August 1965.

[^3

G. Salton, The Evaluation of Automatic Retrieval Procedures — Selected
Test Results Using the SMART System, American Documentation, Vol. 16,
No. 3, July 1965*[5]

M. E. Lesk, The SMART System -- Typical Processing Sequences, Report
No. ISR-8 to the National Science Foundation, Section I, Harvard
Computation Laboratory, December 196^.

[6]

M. E. Lesk, The SMART Automatic Text Processing and Document Retrieval
System, Report No. ISR-8 to the National Science Foundation, Section
II, Harvard Computation Laboratory, December I96U.

[73

Mathematical Linguistics and Automatic Translation, Report No. NSF-8
to the National Science Foundation, Harvard Computation Laboratory,
1962.

[8]

Mathematical Linguistics and Automatic Translation, Report No. NSF-9
to the National Science Foundation, Harvard Computation Laboratory,
May 1963.

[9]Mathematical Linguistics and Automatic Translation, Report No. NSF-13
to the National Science Foundation, Harvard Computation Laboratory,
196U.

[103

S. Kuno, Current Research in Computational and Mathematical Linguistics
at the Computation Laboratory of Harvard University, Report No. NSF-15
to the National Science Foundation, Section I, Harvard Computation
Laboratory, August 1965*[U]

M. Cane, Dictionary Lookup and Updating Procedures, Report No. ISR-7*
Section IV, and Report No. ISR-9, Section V to the National Science
Foundation, Harvard Computation Laboratory, June 196^ and August
1965.

[123

M. Cane, op. cit.

11-63

References (continued)[13] C Harris, Dictionary and Hierarchy Construction, Report No. ISR-7,
to the National Science Foundation, Section III, Harvard Computation
Laboratory, June 196^.
[ik]

J. Prowse, Syntactic Analysis of Incomplete Sentences in the SMART
System, Report No. ISR-9 to the National Science Foundation,
Section XI, Harvard Computation Laboratory, August 1965*[153 M. E. Lesk, Evaluation of Retrieval Results in the Extended SMART
System, Report No. ISR-9* to the National Science Foundation,
Section XVII, Harvard Computation Laboratory, August 1965*[l6] G. Hochgesang, SOCCER - A Concordance Program, Report No. ISR-11
to the National Science Foundation, Se.ction H I , Cornell University,
June 1966.