ABSTRACT. This paper describes new methods of automatically extracting documents for screening purposes, Le. the computer selection of sentences having the greatest potential for conveying to the reader the substance of the document. While previous work has focused on one component of sentence significance, namely, the presence of high-frequency content words (key words), the methods described here also treat three additional components: pragmatic words (cue words); title and heading words; and structural indicators (sentence location). The research has resulted in an operating system and a research methodology. The extracting system is parameterized to control and vary the influence of the above four components. The research methodology includes procedures for the compilation of the required dictionaries, the setting of the control parameters, and the comparative evaluation of the automatic extracts with manually produced extracts. The results indicate that the three newly proposed components dominate the frequency component in the production of better extracts. KEY WORDS AND PHRASES: automatic extracting, automatic abstracting, sentence selection, document screening, sentence significance, relevance, content words, key words, pragmatic words, cue words, title words, sentence location, research methodology; parameterization, comparative evaluation CR CATEGORIES: 3.10, 3.69, 3.71, 3.72, 3.74 Introduction HISTORICAL BACKGROUND. Shortly after research in automatic translation began, interest was aroused in the possibility of automatically abstracting documents. Emphasis was placed on the production of indicative abstracts (i.e. abstracts that allow a searcher to screen a body of literature to decide which documents deserve more detailed attention) rather than on the production of informative abstracts (Le. abstracts that can serve as substitutes for the document). Also, it was hypothesized that an extract of a document (Le. a selection of "significant" sentences of a document) can serve as an abstract. This hypothesis concerning the substitutability of extracts for abstracts has been discussed in [1, 2, and 4]. All automatic abstracting methods to date have only extracted documents, i.e. the computer has selected sentences of the document and listed them in text order. Although it has been recognized for some time that this kind of automatic abstract might better be called an "automatic extract," "abstract" is often used as the generic term. The original method of Luhn assigned weights to each sentence of the documen t according to statistical criteria, in fact, a simple function of the number of high frequency words occurring in the sentence. Common words such as "the," "and," "at," and "are" were disregarded because of their very high frequency alone, and the remaining words were assigned frequency weights that provided sentence weights for measuring sentence significance. The extracts produced by this strictly statistical method were of sufficient quality to encourage further research [3, 6]. However, a purely statistical method of producing extracts was suspected of being inadequate, and hence other methods were sought [7]. SUMMARY OF PRESENT STUDY. The objective of this study was twofold: first, an .extracting system to produce indicative extracts, and second, a research methodology to handle new text and new extracting criteria efficiently. The purpose of the present paper is to make the results of this study more readily available. The research methodology comprised a study of the abstracting behavior of humans, a general formulation of the abstracting problem and its relation to the problem of evaluation, a mathematical and logical study of the problem of assigning numerical weights to sentences, and a set of extracting experiments employing cyclic improvement. The extracting system. developed uses four basic methods to produce automatic extracts of technical documents approximately 4000 words in length with the IBM 7090-7094 (and has been reprogrammed for the UNIVAC 1103A and CDC 1604). The automatic extracts are believed to be of sufficiently high quality to be used in the screening of large collections of documents. The system has been parameterized to permit modification of both prestored and generated word lists, to readjust weights assigned to text characteristics, to permit selection from 15 combinations of the four basic methods of extracting, and to alter the length of an automatic extract. Three different evaluation schemes were applied to the resulting automatic extracts. Comparison of the automatic extracts and corresponding "target" extracts of 40 documents, which had not been used in the developmental research, showed that a mean of 44 percent of the sentences were coselected. Also, the mean similarity rating, in terms of a subjective evaluation of content by information type, was 66 percent. These are to be compared with a mean of 25 percent coselected sentences and a mean of 34 percent similarity rating between target extracts and random extracts, respectively. Statistical comparison of the automatic and the corresponding target extracts for the documents used in the developmental phase showed a mean of 57 percent coselected sentences with a standard deviation of 15 percent. A sentence-by-sentence analysis of the corresponding automatic and target extracts of 20 of these documents resulted in a judgment that 84 percent of the computer-selected sentences could be classified as extract-worthy; i.e. they would be worthy of selection in an extract of unrestricted length. The following sections of this paper treat the research methodology used in the study, the extracting system, and the outlook for further research. Research Methodology RESEARCH STEPS. The work reported here relies on the detection and' use of sentence "significance." Within sentences selected by humans in forming an extract,
H.P Edmundson REF in 1969 included two more features title word and cue words for determining the sentence weight.
New Methods in Automatic Extracting
JACM
Computer Science