Finding Pragmatic Differences Between Disciplines (2021)

Abstract: Scholarly documents have a great degree of variation, both in terms of content (semantics) and structure (pragmatics). Prior work in scholarly document understanding emphasizes semantics through document summarization and corpus topic modeling but tends to omit pragmatics such as document organization and flow. Using a corpus of scholarly documents across 19 disciplines and state-of-the-art language modeling techniques, we learn a fixed set of domain-agnostic descriptors for document sections and “retrofit” the corpus to these descriptors (also referred to as “normalization”). Then, we analyze the position and ordering of these descriptors across documents to understand the relationship between discipline and structure. We report within-discipline structural archetypes, variability, and between-discipline comparisons, supporting the hypothesis that scholarly communities, despite their size, diversity, and breadth, share similar avenues for expressing their work. Our findings lay the foundation for future work in assessing research quality, domain style transfer, and further pragmatic analysis.

## introduction disciplines such as art, physics, and political science contain a wide array of ideas, from specific hypotheses to wide-reaching theories. in scholarly research, authors are faced with the challenge of clearly articulating a set of those ideas and relating them to each other, with the ultimate goal of expanding our collective knowledge. in order to understand this work, human readers situate meaning in context @xcite . similarly, methods for scholarly document processing (sdp) have semantic and pragmatic orientations. the semantic orientation seeks to understand and evaluate the ideas themselves through information extraction @xcite , summariza-tion @xcite , automatic factchecking @xcite , etc. the pragmatic orientation, on the other hand, seeks to understand the context around those ideas through rhetorical and style analysis @xcite , corpus topic modeling @xcite , quality prediction (maillette de @xcite , etc. although both orientations are essential for understanding, the pragmatics of disciplinary writing are very weakly understood. in this paper, we investigate the structures of disciplinary writing. we claim that a "structural archetype" (defined in section 3) can succinctly capture how a community of authors choose to organize their ideas for maximum comprehension and persuasion. analogous to how syntactic analysis deepens our understanding of a given sentence and document structure analysis deepens our understanding of a given document, structural archetypes, we argue, deepen our understanding of domains themselves. in order to perform this analysis, we classify sections according to their pragmatic intent. we contribute a data-driven method for deriving the types of pragmatic intent, called a "structural vocabulary", alongside a robust method for this classification. then, we apply these methods to 19k scholarly documents and analyze the resulting structures. ## related work we draw from two areas of related work in sdp: interdisciplinary analysis and rhetorical structure prediction. in interdisciplinary analysis, we are interested in comparing different disciplines, whether by topic modeling between select corpora/disciplines @xcite or by domain-agnostic language modeling @xcite . these comparisons are more than simply interesting; they allow for models that can adapt to different disciplines, helping the generalizability for downstream tasks like information extraction and summarization. in rhetorical structure prediction, we are interested in the process of implicature, whether by describing textual patterns in an unsupervised way (ó séaghdha and teufel, 2014) or by classifying text as having a particular strategy like "statistics" @xcite or "analogy" @xcite . these works descend from argumentative zoning @xcite and the closely related rhetorical structure theory @xcite , which argue that many rhetorical strategies can be described in terms of units and their relations. these works are motivated by downstream applications such as predicting the popularity of a topic @xcite and classifying the quality of a paper (maillette de buy @xcite . most similar to our work is @xcite . here, the authors provide a method of describing wikipedia articles as a series of section-like topics (e.g. disease.symptom) by clustering section headings into topics and then labeling words and sentences with these topics. we build on this work by using domain-agnostic descriptors instead of domain-specific ones and by comparing structures across disciplines. ## methods in this section, we define structural archetypes (3.1) and methods for classifying pragmatic intent through a structural vocabulary (3.2).