Linguistic Compression in Single-Sentence Human-Written Summaries (2023)

Abstract: Summarizing texts involves significant cognitive efforts to compress information. While advances in automatic summarization systems have drawn attention from the NLP and linguistics communities to this topic, there is a lack of computational studies of linguistic patterns in human-written summaries. This work presents a large-scale corpus study of human-written single-sentence summaries. We analyzed the linguistic compression patterns from source documents to summaries at different granularities, and we found that summaries are generally written with morphological expansion, increased lexical diversity, and similar positional arrangements of specific words compared to the source across different genres. We also studied how linguistic compressions of different factors affect reader judgments of quality through a human study, with the results showing that the use of morphological and syntactic changes by summary writers matches reader preferences while lexical diversity and word specificity preferences are not aligned between summary writers and readers.

## introduction summarization requires condensing important information from a longer text into a brief form @xcite . it involves complex cognitive capabilities related to logical thinking and language proficiency and has been commonly used in classroom settings to teach and assess these capabilities of students. extensive education and psychology research on the development of students has shown that summary writing is a rich resource to study the human cognitive process of comprehension, retention, and organization @xcite . recently, human summary writing has drawn increased attention from the nlp and psycholinguistics communities for the purposes of computational modeling of language production. given advances in transformer-based neural models, automatic text summarization systems have achieved a breakthrough in performance. but abstractive summaries generated by neural summarizers still suffer from hallucinated generation @xcite and verbose copying from the source text @xcite . studies have shown that mimicking human behaviors and preferences is a promising approach toward more naturalistic generations for machinelearning tasks @xcite and consistent patterns of human summary writing can provide helpful references to this end. previous literature has investigated cognitive operations behind human summary writing, especially the operation of compression. compressing text into a concise form requires the deletion of irrelevant information, generalization of concepts, and construction of fluent expressions @xcite . this entails complicated operations on the language features of the produced texts at the levels of syntax, semantics, and discourse. the strategies of compression during summarization can be studied using corpus linguistics @xcite . however, there is a lack of fine-grained analysis of the compression strategies present in summarization, and a dearth of large-scale quantitative studies that examine the general patterns of linguistic compression strategies in human summary writing. to provide more in-depth insight into human behaviors during summary writing, this study presents a large-scale computational analysis of linguistic compression patterns in human-written summaries of expository texts. to amplify any consistent linguistic compression patterns, we focus on extreme summaries @xcite , i.e. single-sentence summaries that are highly compressed and abstractive without verbatim copying from the source text. the patterns are extracted from large summarization corpora of different subgenres at multiple granularities to obtain a more fine-grained picture of human summary writing. we then conduct a human study to evaluate which linguistic compressions are most preferred by readers. ## related work the cognitive operations of compression during summary writing were formulated as "macrorules" by @xcite . brown and day proposed four common operations in summary writing as macrorules, which experienced high school and college students self-reported to intentionally and frequently use when writing summaries: deletion, selection, substitution, invention, and combination. empirical research has shown that linguistic features can reflect the usage of some of the macrorules @xcite . @xcite proposed a computational model using sentence representations and latent semantic analysis to model the use of macrorules by participants in an experimental study. recently, studies in journalism and education have investigated linguistic features of summaries for specific domains by viewing news headlines @xcite , research article abstracts @xcite , and paper reviews (leijen and leontjeva, 2012) as summaries. these works have identified patterns for each domain and shown that the patterns can help writers improve the quality and effectiveness of their summary writing. for example, @xcite examined the correlations between the linguistic features of peer reviews of academic papers and the acceptability of the revision suggestions provided in the reviews using a random forest model, and found that the paper authors are more likely to accept and implement the reviewer comments if the comments have more directives. @xcite studied news headlines as a form of extreme summaries and correlated the usage of sentiment, named entities, phrases, syntactic structure, and punctuation in news headlines with the popularity of the associated articles. the only cross-domain study of the linguistic influences on the general task of summary writing is @xcite , which compared the differences in the use of several language features, including part-of-speech tags, sentence sentiment, named entity tags, and relation tags from rhetoric structure theory, between humanwritten and machine-generated summaries. arroyo-fernández et al. found that machine-generated summaries show significantly different behaviors in the usage of named entities from humans. however, these existing works did not discuss the finegrained distributional differences between source document and summary pairs or evaluate how the identified linguistic patterns are related to the perceived summary quality by summary readers. in contrast, we propose a more fine-grained analysis framework of the linguistic patterns present in summarization, analyzing the pairwise relations between source documents and the associated summaries at different granularities in addition to linguistic features of the summaries in isolation. ## reader preference evaluation common summarization strategies among writers do not necessarily entail quality: good strategies of summary writing should help readers better understand the gist of the source, and readers' perceptions of summary texts might be different from the writers' intentions. therefore, we evaluated the extent to which readers preferred each of the compression patterns described in the previous section. ## discussion this work provides a large-scale computational corpus study towards understanding how humans generally write single-sentence summaries. we performed an analysis of changes in morphological, syntactic, and semantic features and used dynamic time warping with barycenter averaging to analyze the fine-grained positional patterns of linguistic features. our analyses found that syntactic and morphological changes are helpful when distinguishing whether a sentence is a summary or from a source document. single-sentence summaries are generally written with morphological expansion, increased specificity of words, and increased lexical diversity compared to the source sentences. at the word level, summaries are written to replicate the positional arrangements of their source sentences while the temporal arrangements of long dependencies in summary sentences are similar to those of random sentences. we also evaluated the impact of linguistic feature compression on human readers. using representational similarity analysis over correlations between linguistic features and quality ratings, we showed that morphological features and lexical diversity (type-token ratio) are correlated well with quality ratings. readers' perceptions of consistency, factual coverage, novelty, and abstraction level of summaries are correlated with changes in syntactic features such as the average dependency length. the results suggest that summary writers and readers both emphasize certain syntactic features, but type-token ratio and word specificity exhibit a mismatch between writer usage and reader preference. this implies that semantic compression and expansion need to be carefully applied by summary writers to effectively express the main information to readers. it should be noted that we only indirectly tested the influence of linguistic feature compression on readers through our human study since we constrained our survey to summaries obtained from corpora. in the future, we plan to incorporate direct testing by asking readers to rate summary stimuli that are specifically manipulated with particular compression patterns. future works might also include investigations of the incremental processing of text production under experimental setups with the help of techniques such as eye tracking and cloze tasks. as motivated by the emerging focus of text summarization in the nlp community, our work can serve as a psycholinguistically-informed resource for improving automatic summarization systems (e.g., explicitly imitating human behaviors during summary writing improves coherence and organization of text summarization; @xcite and probing model understanding of text generation.