Words as Gatekeepers: Measuring Discipline-specific Terms and Meanings in Scholarly Publications (2023)

Abstract: Scholarly text is often laden with jargon, or specialized language that can facilitate efficient in-group communication within fields but hinder understanding for out-groups. In this work, we develop and validate an interpretable approach for measuring scholarly jargon from text. Expanding the scope of prior work which focuses on word types, we use word sense induction to also identify words that are widespread but overloaded with different meanings across fields. We then estimate the prevalence of these discipline-specific words and senses across hundreds of subfields, and show that word senses provide a complementary, yet unique view of jargon alongside word types. We demonstrate the utility of our metrics for science of science and computational sociolinguistics by highlighting two key social implications. First, though most fields reduce their use of jargon when writing for general-purpose venues, and some fields (e.g., biological sciences) do so less than others. Second, the direction of correlation between jargon and citation rates varies among fields, but jargon is nearly always negatively correlated with interdisciplinary impact. Broadly, our findings suggest that though multidisciplinary venues intend to cater to more general audiences, some fields’ writing norms may act as barriers rather than bridges, and thus impede the dispersion of scholarly ideas.

## introduction specialized terminology, or jargon, naturally evolves in communities as members communicate to convey meaning succinctly. it is especially prevalent in scholarly writing, where researchers use a rich repertoire of lexical choices. however, niche vocabularies can become a barrier between fields @xcite @xcite , and between scientists and the general public @xcite @xcite freel- citing across disciplines? ## data our work involves several datasets: scholarly abstracts, wikipedia, and wiktionary. we use abstracts to calculate the association of words with disciplines and wikipedia to supplement our calculation of background word probabilities. later, in §4, we introduce and describe how we use wiktionary to validate our approach. ## methods language differences among subsets of data can be measured by a variety of approaches, from geometric to information theoretic @xcite @xcite . we calculate the association of a word's type or sense to subfields using normalized pointwise mutual information (npmi). we choose npmi over similar metrics (e.g. tf-idf, divergence, z-score) because of the nature of language difference it emphasizes: higher npmi scores reflect language that is not only commonly used in a community, but also highly specific to it @xcite . npmi offers an interpretable metric of association, where a score of 1 indicates perfect association, 0 indicates independence, and -1 indicates no association. we follow @xcite 's framework of calculating npmi separately for word types and senses, which they originally used to identify communityspecific language on social media. we update their approach with a more recent word sense induction (wsi) method, and use a different interpretation of type and sense npmi scores. ## language norms across fields the linguistic insularity of science varies across fields. for example, vilhena et al. @xcite found that phrase-level jargon separates biological sciences more so than behavioral and social sciences. we perform a similar analysis with the novel addition of word senses. to summarize the distinctiveness of word types in a field, we calculate the mean type npmi score of unique words in a field. before taking the mean, however, we adjust scores by zeroing negative values, since we are more interested in words associated with a field rather than those that are not. this zeroing practice is typically used in studies where pmi measures word relatedness @xcite @xcite . like @xcite , we also find that the biological sciences have very distinctive word types (figure 4 ). however, there is a considerable amount of overlap in word type distinctiveness across fields. similar to how natural sciences name molecules and chemicals, the arts and humanities name canons of writers, philosophers, and artists. we also examine what fields gain the most in npmi scores when common words are broken into their senses. we recalculate subfields' average adjusted npmi, but use max(t f (t), s f (t)) instead of t f (t) for words that have induced senses. based on their relative increases in average adjusted npmi, subfields in math/technology, physics, and economics often use common words in specialized contexts ( ## social implications in this section, we examine two social implications of our metrics: audience design and scholarly success. we limit these experiments to articles in contemporary s2orc that are published among 11,047 venues in the top 95th percentile by abstract count (at least 800 each in s2orc), to ensure solid estimation of venue-level information, such as their disciplinary focus and average citations per article. ## related work computational sociolinguistics often focuses on social media @xcite , with less attention on situation-dependent language varieties, or registers, in scholarly communities @xcite . here, language differences can indicate different factions of authors and disciplinary approaches @xcite @xcite . in addition to our present work, a few studies have examined word meaning or use, such as semantic influence or novelty @xcite and semantic uncertainty @xcite . research on lexical ambiguity in science also appears in education, with an emphasis on how to improve the teaching of overloaded terminology @xcite . other nlp studies of science have predicted responses to articles @xcite , measured impact and innovation @xcite @xcite , and classified topics' rhetorical functions @xcite . ## conclusion we use data-driven, interpretable methods to identify jargon, defined as discipline-specific word types and senses, across science at scale. by identifying senses, we are able to recall more words labeled as associated with a field in wiktionary than with word types alone. we then map language norms across subfields, showing that fields with distinctive word types differ from those with distinctive word senses. finally, we analyze implications of jargon use for communication with out-groups. we find that supposedly general-purpose venues have varying expectations around abstracts' use of jargon depending on the field, and jargon is negatively related to interdisciplinary impact. this suggests a potential opportunity for the reconsideration of abstract writing norms, especially for venues that intend to bridge disciplines.