IXIX

C O N C L U S I O N° *Overview

The main body of the review discussed experimental or operational
approaches to the different activities involved in automatic indexing,
namely document analysis, document description, searching and index
language generation0 It is evident that the approaches to both syntax
and semantics are, linguistically speaking, not very sophisticated,particularly in operational systems. The methods being tried inquestion answering research are more ambitious, but their application
to large scale document retrieval systems is still some way off.
The most striking feature of research in automatic indexing is
how little there is of it0 It is difficult to see it having any impact
on the massive operational systems and services which are becoming
establishedu
It is true that there are not many basic ideas whichhave not been tested in some form or other, but the experiments performedin the last decade are simply an arbitrary choice from the very muchlarger set which could, and presumably should, be carried out. It istherefore particularly depressing that the experiments which have been
done should be, in the main, so unrelated to one another, and that theyshould typically involve such small test collections and/or such
defective evaluation0
A further difficulty is presented by the fact that so few directcomparisons between manual and automatic indexing have been made.
Admittedly those that have agree in showing that automatic indexing is
not markedly worse than manual, as appears in Section VIII. But itwould be dangerous to build too much on such slight foundations.
Very broadly, findings for automatic indexing seem to parallel thosefor manual experiments: for example on the value of syntax. But automatic
indexing in some cases offers opportunities which are not always readily
available in manual systemss for instance in connection with weighting.

Specific conclusions are as follows:loSyntaxIn general the syntactic techniques used either to identify
important content bearing words or to provide relational information
have not been particularly advanced ones. There is no evidence thatthey are particularly effective for either purpose.

9,1

2,

Semantics

The statistical approaches used for extraction do not seem very
useful; and those tested for classification generation are not obviously
advantageous. But statistically based weighting does seem to benefit
performance*
Some approaches to the treatment of other system components chiefly
developed in automatic indexing research appear to be advantageous: forexample, normalising matching coefficients which can be expected togenerate a quite discriminating ranking.
When the results of all the experiments are taken together with
operational practice, the specification for an automated system, simpler
than some current retrieval systems, but possibly as effective as they
are, or at least not conspicuously inferior, emerges. Some parts areprovided manually for reasons of convenience rather than principle. As
retrieval systems nowadays have names, this one may be called MODESTA.
1*

input

: abstracts (+titles).

2.

analysis: selection of all words not stigmatised by a manualstop list (this may include some 'fluff1 words aswell as syntactic operators).

3.

description: attachment of within-document frequency weights,

4.

index language : vocabulary control through application of suffix
stripping.

50

searching: exploitation of further vocabulary control throughuse of collection frequency based weights.

The system would usually operate with a normalising matching coefficient.
Its performance might be improved by relevance feedback, and its operation
made more convenient by document clustering. It would not be very
expensive.

IX o 2

Recommendations

The specification of MODESTA just outlined is in some ways rather
dank: she does not attract. But the all-singing, all-dancing systems
so enthusiastically predicted in the early days of mechanised information
retrieval are not readily achieved. It is more useful to think of genuine
research problems than pie in the sky. In particular, it is evident that
we are very far from understanding why many results are obtained. For
example, why do statistical association techniques perform so variably?
In my view, the real research need is for experiments which will provide
us with explanations for performance results, and not merely results.

9,2

A sensible medium range programme would therefore be to carry out
rather bigger,, and better designed, experiments over a proper rangeof collections and test variables. Hopefully, these should give us
some information about such factors as exhaustivity, for example, or
term cooccurrence patterns, which would help us to understand not
merely why automatic techniques behave in observed ways, but (ultimately)
retrieval systems in general. A specific point is that vocabularycontrol in MODESTA is rather rudimentary, and it is doubtful whether
these simple methods would be adequate for really large collections,
A very worthwhile line would therefore be to investigate simple automatic control techniques for large vocabularies, given that more
recognition of verbal relations would seem to be required.

9.3

B I B L I O G R A P H Y

The references are in alphabetical order of sole or lead author(see definition in the Preface), and in date order for each author.
Abbreviations s
I, So & R,=

Information Storage Retrieval

Jo Doc•Journal of Documentation

J. ASIS=

Journal of the ASISo

1972

Aitchison, JG and Gilchrist, A« Thesaurus Construction; A
Practical Manual, London: Aslib, 1972
1970
Aitchison, T0M0, Hall, A.M., Lavelle, K.H, and Tracy, JoM.
Comparative Evaluation of Indexing Languages, Part II:
Results, Project Inspec, Institute of Electrical Engineers,
London, 1970
1971
Andrews, K. Dissertation for the Diploma in Computer Science,
Computer Laboratory, University of Cambridge, 1971
Armitage
1967
Armitage, J.E0 and Lynch, MaF. "Articulation in the Generationof Subject Indexes by Computer", Journal of Chemical
Documentation, 7, 1967, 170-178
1968
Armitage, J c E e and Lynch, M„F0 "Some Structural Characteristics
of Articulated Subject Indexes", I. S, & Rc, 4, 1968, 101-111
1970
Armitage, J C E 0 , Lynch, M.F., Petrie, J.H. and Belton, M„"Experimental Use of a Program for Computer-Aided SubjectIndex Production", Io S0 & R., 6, 1970, 79-87
Artandi1969aArtandi, Sc "Computer Indexing of Medical Articles", J, D o c , 25,
1969, 214-223
1969b
Artandi, S* and Wolf, E„H. "The Effectiveness of AutomaticallyGenerated Weights and Links in Mechanical Indexing", American
Documentation, 20, 1969, 198-202
1968
Austin, C.J. MEDLARS 1963-1967, U,S.Department of Health Educationand Welfare, Public Health Service, Washington DCC», 1968
1968
Avram, H,D. The MARC Pilot Project, Final Report, Information
Systems Office, Library of Congress, 1968
1973
Barber, A,S 9 , Barraclough, E*Da and Gray, W.A« "On-line
Information Retrieval as a Scientists Tool", I» S. & R., 9,
1973, 429-440
Barker1972a
Barker, FoH,, Veal, D.C. and Wyatt, B„K„ "Comparative Efficiency
of Searching Titles, Abstracts and Index Terms in a Free-Text
Data Base", Jo D o c , 28, 1972, 22-36
1972b
Barker, F«H., Veal, D.C. and Wyatt, B.K. "Towards Automatic Profile
Construction", JP D o c , 28, 1972, 44-55

B.l

1972
1973

1972

1967
1970

1973

1972
1973
1969

1970
1973
1969

1970

1973
1965
1965

1966

1970

1971

Barlow, D.He "Information Retrieval", The Computer Bulletin,
16, 1972, 250-256
Bath University Library, Design of Information Systems in theSocial Sciences Project, "Clustering of Journal Titles
According to Citation Data", Working Paper No, 11,September 1973
Bennett, J,L. "The User Interface in Interactive Systems" inAnnual Review of Information Science and Technology (Ed*
Cuadra), Vol. 7, Washington DaC\s ASIS, 1972
Borko, H. (Edo) Automated Language Processing, New York: John
Wiley, 1967
Borkowski
Borkowski, C, et al0 "Structure and Effectiveness of *The
Citation Identifier7: An Operational Computer Program for
Automatic Identification of Case Citations in Legal Literature",
J, ASIS, 21f 1970, 8-15
Borkowski, C. and Martin, J0S. "Structure, Costs and Benefitsof LEXtractor, An Operational Computer Program for Automatic
Extraction of Case Summaries and Dispositions from Court
Decisions", Proceedings of the ASIS, Vol. 10, 1973, 21-22
Borman, L. and Mittman, Bo "Interactive Bibliographic Data Basesin an Academic Environment", Jq ASIS, 23, 1972, 164-171
Bousselety p 0 "Le Systeme Titus", Nachrichten„ Nouvclles, Notizie,
49, 1973, 157-163
Brauen, T.L* "Document Vector Modification in On-line Information
Retrieval Systems", Information and Storage Retrieval, Report No•
ISR-17, Department of Computer Science, Cornell University, 1969,
Cagan, C. "A Highly Associative Document Retrieval System", J. ASIS,
21, 1970, 330-337
Campey, L. "A Survey of Index Generation Programs", I, S, & R*, 9,
1973, 441-448
Carroll
Carroll, J.M. and Roelloffs, R„ "Computer Selection of Keywords
Using Word-Frequency Analysis", American Documentation, 20, 1969,
227-233
Carroll, J.M„ and DeBruyn, J0G, "On the Importance of Root-Stem
Truncation in Word-Frequency Analysis", JQ ASIS, 21, 1970,
368-369
Carroll, JcM. and Tague, J.M. "Use of an Automatic Analyser in
Preparation of SDI Profiles", J. ASIS, 24, 1973, 277-281
Chomsky, No Aspects of the Theory of Syntax, Cambridge, Mass.:
MIT Press, 1965
Clarke, DoC* and Wall, R 9 E 0 "An Economical Program for Limited
Parsing of English", AFIPS Conference Proceedings, Vol.27, Pt.l,
1965, 307-316
Cleverdon
Cleverdon, C<>W., Mills, J. and Keen, Me Factors Determining the
Performance of Indexing Systems, 2 vols. College of Aeronautics,
Cranfield, 1966
Cleverdon, C.W, "An Investigation into a Suitable Mechanised
Information Retrieval System at the Defence Operational Analysis
Establishment", Cranfield Institute of Technology, 1970
Clough, C.R« and Brarawell, K*M0 "A Single Computer-Based System for
Both Current Awareness and Retrospective Search: Operating
Experience with Assassin", J«, Doc., 27, 1971, 243-253

B.2

1973

Coates, E0JQ "Some Properties of Relationships in the Structureof Indexing Languages8', Jc Doc^ 29, 1973, 390-404
1970
Cooper, W 0 S, "The Potential Usefulness of Catalog Access Points
other than Author, Title and Subject", JQ ASIS, 21, 1970,
112-127
Coyaud1966
Coyaud, M0 Introduction h 1'Etude des Langages Documentaires,
Paris: Klincksieck, 1966
1972
Coyaud, MQ Linguistique et Documentation, Paris: Larousse, 1972
Cuadra1966Cuadra, C0A0 (Ed0) Annual Review of Information Science andTechnology, Vols 1-, 19661971
Cuadra, C0A0 "On-line Systems: Promise and Pitfalls", J« ASIS,
22, 1971, 107-114
1965
Damerau, F0J, "An Experiment in Automatic Indexing", American
Documentation, 16, 1965, 283-289
1969
Dattola, R 0 T 0 "Experiments with a Fast Algorithm for AutomaticClassification", in Information Storage and Retrieval,
Report No0 ISR-16, Department of Computer Science, Cornell
University, 1969; reprinted in Salton 1971aDennis1965
Dennis, SQF0 "The Construction of a Thesaurus Automatically froma Sample of Text" in Statistical Association Methods for
Mechanised Documentation (Ed« Stevens et alo), National
Bureau of Standards, Washington D o C , 1965
1967
Dennis, S0Fo "The Design and Testing of a Fully Automated IndexingSearching System for Documents Consisting of Expository Text"
in Information Retrieval - A Critical View (Ed, Schechter),
Washington D 0 C 0 : Thompson, 1967
1966
Doyle, LoB0 and Blankenship, D0A, "Technical Advances in AutomaticClassification", Proceedings of the API, Volo3, 1966, 63-71
Earl
1970
Earl, LoL*, "Experiments in Automatic Indexing and Extracting",
Ic S0 & Ro, 6, 1970, 313-334
1972
Earl, L0L0 "The Resolution of Syntactic Ambiguity in Automatic
Language Processing", IQ So & RQ, 8, 1972, 277-308
1973
Earl, L 0 L 0 "Use of Word Government in Resolving Syntactic and
Semantic Ambiguities", I0 Sa & R„, 9, 1973, 639-664
Edmundson
1961
Edmundson, HoPa and Wyllys, R«,E. "Automatic Abstracting andIndexing - Survey and Recommendations", Communications of
the ACM, 4, 1961, 226-234
1969
Edmundson, H0Po "New Methods in Automatic Extracting", Journalof the ACM, 16, 1969, 264-285
Fangmeyer
1969
Fangmeyer, Ho and Lustig, G0 "The Euratom Indexing Project" inInformation Processing 68 (Ed, Morrell), Amsterdam: NorthHolland, 1969
1970
Fangmeyer, He and Lustig, Go "Experiments with the CETIS
Automatic Indexing System" in Handling of Nuclear Information,
International Atomic Energy Agency, Vienna, 1970

B«3

Farradane
Farradane, J, "Concept Organisation for Information Retrieval",
L So & R„, 3, 1967, 297-311
1973
Farradane, J„, Russell, J.Ma and Yates-Mercer, P.A. "Problems
in Information Retrieval: Logical Jumps in the Expressionof Information", I0 S„ & Ro, 9, 1973, 65-77
1973
Gardin, J0-C0 "Document Analysis and Linguistic Theory", J. Doc«,29, 1973, 137-168
1972
Gerson, G.M„ "A Computer-Aided Classification and Retrieval System
for United States Patents", I« S« & Rc, 8, 1972, 137-150
1968
Gotlieb, CCo and Kumar, So "Semantic Clustering of Index Terms",
Journal of the ACM, 15, 1968, 493-513
1973
Hansen, I«B,3 "CA Condensates as a Retrospective Search Tool.
A Commentary", I. S, & R„, 9, 1973, 201-205
Hillman1968
Hillman, D.Ja "Negotiation of Inquiries in an On-line Retrieval
System", I0 SQ & R., 4, 1968, 219-238
1969
Hillman, D.Je and Kasarda, A 0 J 0 "The LEADER Retrieval System",
AFIPS Conference Proceedings, VolQ34, 1969, 447-455
1973
Hillman, D0J0 "Customerised User Services via Interactions withLEADERMART", I« So & R,, 9, 1973, 587-596
1971
Hines, T4CC and Harris, J0L* "Columbia University School of LibraryService System for Thesaurus Development and Maintenance",
L So & Ro, 7, 1971, 39-50
1971
Hisinger, B« "A Multidisciplinary and Computerised SDI Service forIndustry and Research - Practical Experience and Costs",
Io So & R., 7, 1971, 175-187
1973
Hoyle, W.Co "Automatic Indexing and Generation of Classification
Systems by Algorithm", Ic S, & R«, 9, 1973, 233-242
1969
Ide, Eo "Relevance Feedback in an Automatic Document Retrieval
System", Information Storage and Retrieval, Report No„ISR-15,
Department of Computer Science, Cornell University, 1969
IJCAI = Proceedings of the International Joint Conference on Artificial
Intelligence
1969
First, The Mitre Corporation, Bedford, Mass«, 1969
1971
Second, The British Computer Society, London, 1971
1973
Third, Stanford University, Stanford, Calif0, 1973
1966
Ivie, E,L„ Search Procedures Based on Measures of Relatedness Between
Documents, Thesis, Massachusetts Institute of Technology, 1966
1973
Jacquesson, AQ and Schieber, W 0 D 0 "Term Association Analysis on a
Large File of Bibliographic Data, Using a Highly-Controlled
Indexing Vocabulary", I« So & Ro, 9, 1973, 85-94
1971
Jardine, N, and Sibson, R. Mathematical Taxonomy, New York: John
Wiley, 1971
Jones
1967a
Curtice, R0M0 and Jones, P,E, "Distributional Constraints andthe Automatic Selection of an Indexing Vocabulary", Proceedingsof the ADIo, Vola4, 1967, 152-156
1967b
Jones, P 0 E 0 , and Curtice, RaM0 "A Framework for Comparing Term
Association Measures", American Documentation, 18, 1967,
153-161
1967

B«4

1968

1969
1963
1967

1972

1973
1971

1965

1973
1973a
1973b
1972

1965
1966

1968a
1968b
1969
1972a
1972b

Jones, P,Eop Curtice, RcM,, Giuliano, V.E. and Sherry,, MflEa"Application of Statistical Association Techniques to theNASA Document Collection", A,D.Little Inc0, Cambridge,
Mass,, 1968
Curtice, R0M0 and Jones, P.E* "An Operational Interactive
Retrieval System", A,D0Little Inc., Cambridge, Mass., 1969
Katz, J,JC and Fodor, J0Ao "The Structure of a Semantic Theory",
Language, 39, 1963, 170-210
Keen
Keen, E0M. Several chapters in Information and Storage Retrieval,
Report NOo ISR-13, Department of Computer Science, Cornell
University, 1967
Keen, EoM. and Digger, J,A* Report of an Information ScienceIndex Languages Test, 2 vols* College of Librarianship Wales,
Aberystwyth, 1972
Keen, E*Ma "The Aberystwyth Index Languages Test", Jo Doc0, 29,
1973, 1-35
Kerchner, M,D9 "Dynamic Document Processing in Clustered Collections",
Information Storage and Retrieval, Report No. ISR-19, Departmentof Computer Science, Cornell University, 1971
Kessler, MJ!0 "Comparison of the Results of Bibliographic Coupling
and Analytic Subject Indexing", American Documentation, 16,
1965, 223-233
Kimball, J e P 0 The Formal Theory of Grammar, Engleirood Cliffs, N.J.:
Prentice-Hall, 1973
Klingbiel
Klingbiel, P0H„ "Machine-Aided Indexing of Technical Literature",
L S, & R0, 9, 1973, 79-84
Klingbiel, P0H* "A Technique for Machine-Aided Indexing", I, So & Ro,
9, 1973, 477-494
Krevett, B« and Griffith, B8C. "A Comparison of Several Zipf-Type
Distributions in their Goodness of Fit to Language Data,
J. ASISo 23f 1972, 220-221
Kuno
Kuno, S. "The Predictive Analyser and a Path Elimination Technique",,
Communications of the ACM, 8, 1965, 453-462
Kuno, S. "Automatic Syntactic Analysis" in Seminar on Computational
Linguistics (Ed. Pratt et al0) U.S0Department of Health Educationand Welfare, Washington D„C0, 1966
Lancaster
Lancaster, F0W. Evaluation of the MEDLARS Demand Search Service,
National Library of Medicine, Bethesda, Md., 1968
Lancaster, F0W0 Information Retrieval Systems: Characteristics,
Testing and Evaluation, New York: John Wiley, 1968
Lancasterp F.WQ "MEDLARS: A Report on the Evaluation of its
Operating Efficiency", American Documentation, 20, 1969, 119-142
Lancaster, FwWtt Vocabulary Control for Information Retrieval,
Washington DoC0: Information Resources Press, 1972
Lancaster, F.W., Rapport, R.La and Kiffin Penry, J. "Evaluating
the Effectiveness of an On-line, Natural Language Retrieval
System", I0 S, & RQ, 8, 1972, 223-245

B.5

1972

1968
1969
1969

1968
1971
1968
1967

1970

1971a
1971b

1970a

1970b

1972

1973
1968
1972
1972
1972

1973

1971

Lefever, M„, Freedman, B, and Schultz, L, "Managing an
Uncontrolled Vocabulary Ex Post Facto", Je ASIS, 23,1972, 339-342
Lesk
Lesk, M«Eo "Performance of Automatic Information Systems",
L S 0 & R., 4, 1968, 201-218
Lesk, M 0 E 0 "Word-Word Associations in Document Retrieval Systems",
American Documentation, 20, 1969, 27-38
Litofsky, B, Utility of Automatic Classification Systems for
Information Storage and Retrieval, Thesis, University ofPennsylvania, 1969
LovinsLovins, J 0 B o "Development of a Stemming Algorithm", MechanicalTranslation, 11, 1968, 22-31
Lovins, J 0 B o "Error Evaluation for Stemming Algorithms as
Clustering Algorithms", J, ASIS, 22, 1971, 28-40
Lyons, J„ An Introduction to Theoretical Linguistics, Cambridge:
Cambridge University Press, 1968
Mathews, W 0 D. "The TIP Retrieval System at MIT" in Information
Retrieval - A Critical View (Ed« Schechter), Washington, D.C.:
Thompson, 1967
Matthews, F 0 W 0 "Weighted Term Search and Strategy for Searching aLarge File for Document Retrieval by Computer", Proceedings of
the ASIS, Vol 0 7, 1970, 315-317
MillerMiller, W 0 L o "The Efficiency of MEDLARS Titles for Retrieval",
Jo ASIS, 22, 1971, 318-321
Miller, W,L C "A Probabilistic Search Strategy for MEDLARS", J, Doc,,27, 1971, 254-266
MinkerAugustson, J,Go and Minker, Jo "An Analysis of Some Graph
Theoretical Cluster Techniques", Journal of the ACM, 17, 1970,
571-588
Augustson, J e G f and Minker, J0 "Deriving Term Relations for a
Corpus by Graph Theoretical Techniques", J, ASIS, 21, 1970,
101-111
Minker, J,, Wilson, G*Ae and Zimmerman, B.Ho "An Evaluation ofQuery Expansion by the Addition of Clustered Terms for a
Document Retrieval System", I, S. & R 0 , 8, 1972, 329-348
Minker, J., Peltola, E* and Wilson, G.A* "Document Retrieval
Experiments Using Cluster Analysis", J0 ASIS, 24, 1973, 246-257
Minsky, Mo Semantic Information Processing, Cambridge, Mass.:
MIT Press, 1968
Montgomery, C G A. "Linguistics and Information Science", J. ASIS, 23,1972, 195-219
Moody, D«W6 and Kays, 0 o "Development of the US Geological Survey
Bibliographic System Using GIPSY", J0 ASIS, 23, 1972, 39-47
Murray, D„M0 "Document Retrieval Based on Clustered Files", Information
Storage and ^^rieval, Report No. ISR-20, Department of Computer
Science, Cornell University, 1973
Myers, J g H 0 "Computers and the Searching of Law Texts in England andNorth America: A Review of the State of the Art", Jo D o c , 29,
1973, 212-228
Negus, A*E. and Hall, J 0 L 0 "Towards an Effective On-line Reference
Retrieval System", I. S 0 & R», 7, 1971, 249-270

Bo6

1970
1973
1968

1973
1972

1969
1972

1971
1972
1973

1966
1970
1970

1971

1973

1972
1973a

1973b

Niblett, G.B.Fo and Price, NoH. "Mechanised Searching of Actsof Parliament", I, SP & R., 6, 1970, 289-297
Olive, G., Terry, J0E. and Datta, So "Studies to Compare Retrieval
Using Titles with that Using Index Terms", J0 Doc., 29, 1973,
169-191
Parker, EflBfl "SPIRES (Stanford Public Information Retrieval System)",
1968 Annual Report, Institute for Communication Science,
Stanford University, 1968
Petrick, S„RQ "Transformational Analysis" in Natural Language
Processing (Ed* Rustin), New York: Algorithmics Press, 1973
Prywes, N.S* and Smith, D.P. "Organisation of Information" in
Annual Review of Information Science and Technology (Ed«Cuadra),
Vol,7, Washington D,C: ASIS, 1972
Reintjes, JCF. "System Characteristics of Intrex", AFIPS Conference
Proceedings, Vol*34, 1969, 457-460; see also adjoining papers*
Rettenmayer, JrJW. "File Ordering and Retrieval Cost", I. So & R., 8,
1972, 79-93
van Rijsbergen
Jardine, No and van Rijsbergen, C.J, "The Use of Hierarchic
Clustering in Information Retrieval", I, S, & R«, 7, 1971, 217-240
van Rijsbergen, C J , Automatic Information Structuring andRetrieval, Thesis, University of Cambridge, 1972
van Rijsbergen, CaJ, and Sparck Jones, K0 "A Test for the
Separation of Relevant and Non-relevant Documents in Experimental
Retrieval Collections", J. D o c , 29, 1973, 251-257
Rocchio, J,J. "Document Retrieval Systems - Optimisation and Evaluation",
Information Storage and Retrieval, Report No. ISR-10, The
Computation Laboratory, Harvard University, 1966
Rolling, L.N. "Compilation of Thesauri for Use in Computer Systems",
Io S. & R., 6, 1970, 341-350
Rowlands, D„G. "The Unilever Research SDI System", I. S. & R., 6, 1970,
53-71
Rush
Rush, J.E., Salvador, R. and Zamora, Ac "Automatic Abstracting andIndexing. II. Production of Indicative Abstracts by Applicationof Contextual Inference and Syntactic Coherence Criteria",
J, ASIS, 22, 1971, 260-274
Mathisp B.A., Rush, J,EC and Young, C.Eo "Improvement of Automatic
Abstracts by the Use of Structural Analysis", J, ASIS, 24, 1973,
101-109
Sager
Sager, Nc "Syntactic Formatting of Science Information", AFIPS
Conference Proceedings, Vol.41, 1972, 791-800
Sager, N. "The String Parser for Scientific Literature", inNatural Language Processing (Ed. Rustin), New York-. Algorithmics
Press, 1973
Grishman, R., Sager, N., Raze, Co and Bookchin, Bft "The Linguistic
String Parser", AFIPS Conference Proceedings, Vol. 42, 1973,
427-434

B.7

1966-

1968a
1968b

1968c
1968d
1969a
1969b

1969c

1969d
1970a
1970b
1970c
1971a
1971b
1972a

1972b
1972c
1972d
1973a
1973b
1973c

SaltonSalton, G. (Project Director), Information Storage andRetrieval, Reports Nos* ISR-11, 12, 13, 14, 16, 18, 21,
Department of Computer Science, Cornell University,
1966-1972
Salton, Gt Automatic Information Organisation and Retrieval,
New York: McGraw-Hill, 1968
Salton, Gc "Search Strategy and the Optimisation of Retrieval
Effectiveness" in Mechanised Information Storage, Retrieval
and Dissemination (Ed„ Samuelson), Amsterdam: North-Holland,
1968
Salton, G„ and Lesk, MeE« "Computer Evaluation of Indexing and
Text Processing", Journal of the ACM, 15, 1968, 8-36
Lesk, M0E« and Salton, Go "Relevance Assessments and Retrieval
System Evaluation", I, So & R,, 4, 1968, 343-359
Salton, G. "A Comparison between Manual and Automatic Indexing
Methods", American Documentation, 20, 1969, 61-71
Salton, G. "Search and Retrieval Experiments in Real-Time
Information Retrieval", Information Processing 68 (Ed. Morrell),
Amsterdam: North-Holland, 1969
Lesk, M,E„ and Salton, G* "Interactive Search and Retrieval Methods
Using Automatic Information Displays", AFIPS Conference
Proceedings, Vol,34, 1969, 435-446
Ide, E. and Salton, G„ "User Controlled File Organisation and
Search Strategies", Proceedings of the ASIS, 6, 1969, 183-191
Salton, G 0 "Automatic Text Analysis", Science, 168, 1970, 335-343
Salton, G. "Automatic Processing of Foreign Language Documents",
J, ASIS, 21, 1970, 187-194
Salton, G. "Evaluation Problems in Interactive Information Retrieval",
L S, & R., 6, 1970, 29-44
Salton, G* (Ed.) The SMART Retrieval System: Experiments in AutomaticDocument Processing, Englewood Cliffs, N»J 0 : Prentice-Hall, 1971
Salton, G. "Automatic Indexing Using Bibliographic Citations",
J. D o c , 27, 1971, 98-110
Salton, G. "Experiments in Automatic Thesaurus Construction for
Information Retrieval", in Information Processing 71 (Ed.
Freiman), Amsterdam: North-Holland, 1972
Salton, G. "The 'Generality* Effect and the Retrieval Evaluationof Large Collections", J„ ASIS, 23, 1972, 11-22
Salton, G. "A New Comparison between Conventional Indexing (MEDLARS)
and Automatic Text Processing (SMART)", Jc ASIS, 23, 1972, 75-84
Salton, G. "Dynamic Document Processing", Communications of the ACM,
15, 1972, 658-668
Salton, Go "Automatic Processing of Current Affairs Queries",
I. So & R., 9, 1973, 165-180
Salton, G« "Recent Studies in Automatic Text Analysis and Document
Retrieval", Journal of the ACM, 20, 1973, 258-278
Salton, G. and Yang, C.S* "On the Specification of Term Values inAutomatic Indexing", Jo D o c , 29, 1973, 351-372

B«8

1968

1971
1972

1971

1968
1965
1970
1963
1969

1964
1967

1970a
1970b
1971a
1971b
1971c

1972
1973a
1973b
1973c

Saracevic
Comparative Systems Laboratory, An Inquiry into Testing ofInformation Retrieval Systems, 3 vols. Western Reserve
University, 1968
Saracevic, T* "Selected Results from an Inquiry into Testingof Information Retrieval Systems", Jc ASIS, 22, 1971, 126-139
Scheffler, Fc, March, Ja and Bernados, JQ "An Experiment to Study
the Use of Boolean NOT Logic to Improve the Precision of
Selective Dissemination of Information, J. ASIS, 23, 1972,
58-65
Schiminovich, S« "Automatic Classification and Retrieval of Documents
by Means of a Bibliographic Pattern Discovery Algorithm",
Ic So & Ro, 6, 1971, 417-435
Schultz, CoKo (Edo)<, HQP,Luhn: Pioneer of Information Science Selected Works, London: Macraillan, 1968
Sharp, J0R0 Some Fundamentals of Information Retrieval, Londons
Andre Deutsch, 1965
Simmons, R,F. "Natural Language Question-Answering Systems",
Communications of the ACM, 13, 1970, 15-30
Sokal, RGRo and Sneath, P.H.A, Principles of Numerical Taxonomy,
London: Freeman, 1963
Sommar, HoG0 and Dennis, DeE. "A New Method of Weighted Term Searchingwith a Highly Structured Thesaurus", Proceedings of the ASIS,
Vol06, 1969, 193-198
Sparck Jones
Sparck Jones, Kc Synonymy and Semantic Classification, Thesis,
University of Cambridge, 1964
Sparck Jones, Kc and Jackson, D.M, "Current Approaches toClassification and Clump-Finding at the Cambridge Language
Research Unit, The Computer Journal, 10, 1967, 29-37
Sparck Jones, Kc "Some Thoughts on Classification for Retrieval",
J0 D o c , 26, 1970, 89-101
Sparck Jones, K« and Jackson, D„Me "The Use of AutomaticallyObtained Keyword Classifications for Information Retrieval",
Io So & Ro, 5, 1970, 175-201
Sparck Jones, Kc Automatic Keyword Classification for Information
Retrieval, London: Butterworths, 1971
Sparck Jones, K« and Barber, E«0, "What Makes an Automatic Keyword
Classification Effective?", J, ASIS, 22, 1971, 166-175
Sparck Jones, Kc "The Theory of Clumps" in Encyclopedia of Library
and Information Science (Ed„ Kent and Lancour), Vol*5, New York:
Marcel Dekker, 1971
Sparck Jones, K„ "A Statistical Interpretation of Term Specifity
and its Application in Retrieval", Jc Doc*, 28, 1972, 11-21
Sparck Jones, KP and Kay, Mo Linguistics and Information Science,
New York: Academic Press, 1973
Sparck Jones, KQ "Index Term Frequency and Quartile Deletion"(Letter), J, ASIS, 24, 1973, 166-167
Sparck Jones, K0 "Collection Properties Influencing Automatic
Term Classification Performance", I, S. & R«„ 9, 1973,
499-513Bo9

1973dSparck Jones, K. "Does Indexing Exhaustivity Matter?", J, ASIS,
24, 1973, 313-316
1973e
Sparck Jones, K. "Index Term Weighting", I, Sc & R», 9, 1973,
619-633
Stevens
1965a
Stevens, M.EC Automatic Indexing; A State of the Art Report,
Monograph 91, National Bureau of Standards, Washington D,C,
1965
1965b
Stevens, M a E 0 , Heilprin, L. and Giuliano, V,E. (Eds,) Statistical
Association Methods for Mechanised Documentation, National
Bureau of Standards, Washington, D # C G , 1965
Stone
1967a
Stone, D.Co Word Statistics in the Generation of Semantic Tools
for Information Retrieval Systems, Thesis, University ofPennsylvania, 1967
1967b
Rubinoff, MG and Stone, D0C. "Semantic Tools in Information
Retrieval", Proceedings of the API, Vol,4, 1967, 169-174
1968
Stone, DoCo and Rubinoff, M0 "Statistical Generation of a TechnicalVocabulary", American Documentation, 19, 1968, 411-412
1972
Svenonius, E, "An Experiment in Index Term Frequency", J. ASIS, 23,1972, 109-121
Syntol1964
Cros, RQC„, Gardin, J.Ce and Levy, F0 L6Automatisatj.on desRecherches Documentaires. Un Mod&le Generals Kfe SYNTOL, Paris:
Gauthier Villars, 1964? 2nd Edition, 1968
1965
Gardin, J,C. Syntol, Graduate School of Library Service, Rutgers,
The State University, New Brunswick, N«J<., 1965
1967
Coyaud, M. and Siot-Decauville, N. L*Analyse Automatigue desDocuments, The Hague: Mouton, 1967
1970
Bely, N., Borillo, Ac, Virbel, J« and Siot-Decauville, N.
Procedures d*Analyse Semantique Appliquees a la Documentation
Scientifigue, Paris: Gauthier Villars, 1970
Tell
1970
Tell, B.V«, Larsson, R« and Lindh, R« "Information Retrieval with
the Abacus Program" in Handling of Nuclear Information,
International Atomic Energy Agency, Vienna, 1970
1971
Tell, B0V. "Retrieval Efficiency from Titles and the Cost ofIndexing", I, S» & R,, 7, 1971, 241-243
1972
Thiel, L.HU and Heaps, H.S. "Program Design for Retrospective Searches
on Large Data Bases", IQ So & R,, 8, 1972, 1-20
1968
Thome, J,P., Bratley, P. and Dewar, Hc "The Syntactic Analysis ofEnglish by Machine" in Machine Intelligence 3 (Ed. Michie),
New York: American Elsevier, 1968
1970
Vaswani, P.K.T. and Cameron, JeB. The National Physical Laboratory
Experiments in Statistical Word Associations and their Use in
Retrieval, National Physical Laboratory, Teddington, 1970
1974(?)Vernimb, C O . "Abstracts on Microfiche for On-line Retrieval", New
Developments in Storage, Retrieval and Dissemination of Aerospace Information, AGARD, Neuilly sur Seine,, 1974 (?)
1971
Vickery, B„C. "Structure and Function in Retrieval Lanuages", J, Doc,27, 1971, 69-82

B.10

1973

1969

1972

1971
1973

1968a

1968b

1971
1972

1972
1971

1970
1972

1973

Walker, D.E. "Automated Language Processing" in Annual Review
of Information Science and Technology (Ed, Cuadra), Vol.8,
Washington D.C.: ASIS, 1973
Weiss, S a F 8 "Template Analysis and its Application to Natural
Language Processing" in Information Storage and Retrieval,
Report No, ISR-16, Department of Computer Science, Cornell
University, 1969
Wellisch, H. "A Flow Chart for Indexing with a Thesaurus", JP ASIS,
23, 1972, 185-194
Wilks
Wilks, Y,A0 Grammar, Meaning, and the Machine Analysis of Language,
London: Routledge, 1971
Wilks, Y.A. "The Stanford Machine Translation Project" in Natural
Language Processing (Ed. Rustin), New York: Algorithmics Press,
1973
J.Williams
Williams, J„H. "Computer Classification of Documents" in
Mechanised Information Storage, Retrieval and Dissemination (Ed*
Samuelson), Amsterdam: North-Holland, 1968
Williams, J.H. and Perriens, M.P. "Automatic Full Text Indexing
and Searching Systems", IBM Federal Systems Center, Gaithersburg,Md., 1968
Williams, JoH, "Functions of a Man-Machine Interactive Information
Retrieval System", J. ASIS, 22, 1971, 311-317
Williams, M.E. "Experiences of IIT Research Institute in Operatinga Computerised Retrieval System for Searching a variety ofData Bases", I. S. & R., 8, 1972, 57-78
Winograd, T. Understanding Natural Language, Edinburgh: Edinburgh
University Press, 1972
Wolff-Terroine, M. and Rimbert, D. "Computer-Aided Automatic Generationof a Structured Documentary Language: Preliminary Study". J. D o c ,27, 1971, 111-124
Woods
Woods, W.A. "Transition Network Grammars for Natural Language
Analysis", Communications of the ACM, 13, 1970, 591-606
Woods, W.Ao, Kaplan, R. and Nash-Webber, B. The Lunar Sciences
Natural Language Information System, Final Report, Bolt, Beranek
and Newman, Inc., Cambridge, Mass., 1972
Woods, W.A. "Progress in Natural Language Understanding - An
Application to Lunar Geology", AFIPS Conference Proceedings,
Vol.42, 1973, 441-450

B.ll