Some Problems and Methods of Text Condensation

ing If we look at the guides written for abstractors it becomes soon apparent that abstracting is far more than just ‘summarising’. In addition to the distinction between ‘informative’ and ‘indicative’ abstracting mentioned earlier we find a number of specific recommendations and injunctions; to retain the balance and emphases of the original, to pass no comments (either favourable or critical), to state clearly the purpose of the work described, the methods used and the conclusions reached, and to produce a self-contained coherent text (ideally within a single paragraph) which might stand as a substitute for the original for some purposes. The need to produce coherent texts has meant that research on automatic abstracting has been more ambitious from the linguistic point of view than automatic indexing. Nevertheless in practice the primary approach has also been statistical. Initially, attempts were made to produce ‘abstracts’ by the extraction of sentences en bloc from texts on the basis of high frequency words (excluding function words and items of common vocabulary), e.g. Luhn’s (1958) pioneering work. The results were neither particularly good condensations nor very coherent texts. Later systems have combined more sophisticated statistical methods (similar to those in automatic indexing) with the use of textual ‘cues’ to identify important (topical) passages. Edmundson (1969) and Rush et al. (1971) have used three types of ‘cues’: (i) the recurrence of words in titles, subtitles and section headings, or the occurrence of words synonymous with them, (ii) the presence of such words as significant, impossible, hardly, which indicate authors’ views of the importance of the information presented, and (iii) the location of sentences within paragraphs and sections. The first type is obviously based on the observation that titles, subtitles and section headings tend to express the author’s notion of what the topic (‘theme’) is in the following text. The second type is clearly related to the kinds of discourse signals investigated by Hoey (1979). The third type formalises observations on the occurrence of ‘topic sentences’ in paragraphs, which are familiar from writings on rhetoric and composition (cf. Christensen 1967) and which Danes (1974) formalised in terms of theme-rheme articulations at the supra-sentential level of text structure. Whether based on vocabulary frequencies or textual cues it is clear that these procedures are all instances of Van Dijk’s macro-rule of selection. Indeed researchers recognise the limitations of their systems and concede that what they are doing should be more correctly called ‘automatic extracting’. However, some procedures have been developed to produce coherent sequences of sentences from those extracted, and these include a certain measure of generalisation. For example, among various refinements by Mathis et al. (1973) of the automatic abstracting system of Rush et al. (1971) we find the alteration of specific references (e.g. Table 2, figure 3, the second mechanism) to general references (a table, a figure, a mechanism) and the conflation of sentences by coordination and subordination. For example, the two extracted sentences: The system exceeded the capacity of its present auxiliary equipment. The system was modified for further testing. could be combined as: The system exceeded the capacity of its present auxiliary equipment and was modified for further testing. This operation involved limited parsing to identify noun, verb, and preposition phrases, to locate antecedents of pronouns, and to recognise parallel structures. Such simple syntactic manipulations (though not necessarily simple computationally) scarcely touch the real problem of generalisation in summarisation, namely to account for our ability to select a general term or expression to cover the content of a number of more specific expressions. The necessary lexical organisation may well be reflected or modelled in the thesauri (of scientific, medical, technical, etc, terms) which indexers use to guide them in the selection of