Personalized Jargon Identification for Enhanced Interdisciplinary Communication (2024)

Abstract: Scientific jargon can confuse researchers when they read materials from other domains. Identifying and translating jargon for individual researchers could speed up research, but current methods of jargon identification mainly use corpus-level familiarity indicators rather than modeling researcher-specific needs, which can vary greatly based on each researcher’s background. We collect a dataset of over 10K term familiarity annotations from 11 computer science researchers for terms drawn from 100 paper abstracts. Analysis of this data reveals that jargon familiarity and information needs vary widely across annotators, even within the same sub-domain (e.g., NLP). We investigate features representing domain, subdomain, and individual knowledge to predict individual jargon familiarity. We compare supervised and prompt-based approaches, finding that prompt-based methods using information about the individual researcher (e.g., personal publications, self-defined subfield of research) yield the highest accuracy, though the task remains difficult and supervised approaches have lower false positive rates. This research offers insights into features and methods for the novel task of integrating personal data into scientific jargon identification.

## introduction an important challenge to communicating knowledge across scientific domains is aligning on a shared vocabulary @xcite . each scientific domain has unique terminology that optimizes communication within the field but can pose a barrier to researchers in other domains @xcite . as science becomes more specialized, so too does its terminology @xcite , somewhat not at all example background defini:on ## task description we conduct an initial study with 10 computer science researchers to (i) validate our intuition that term familiarity is important for interdisciplinary reading, and (ii) identify what information needs researchers have for unfamiliar terms when reading across domains. we recruited participants from two subdomains of computer science: from natural language processing (nlp) and from human-computer interaction (hci). participants were asked to read two paper abstracts, one from a closer domain (linguistics or psychology) and one from a more distant domain (medicine). for each paper, we provided two abstract variants: the original author-written abstract and a generated abstract personalized to the participant's background. to personalize the abstracts, we prompted gpt-3 (text-davinci-003) to rewrite the provided abstract as an abstract personalized to an author. the model was given the author's paper abstracts and a sampled list of citation sentences from the author's papers (details can be found in app. ## dataset as no pre-existing datasets exist for personalized scientific jargon identification, we construct a new dataset of terms from abstracts with human annotations of familiarity and additional information needs. we direct our focus to abstracts that are outside the individual's domain, with cs researchers as the annotators. ## prediction given our dataset and annotator backgrounds, we investigate the effectiveness of a set of features and methods for predicting individual term familiarity and additional information needs. ## discussion this paper introduces the novel task of personalized scholarly jargon detection. we collect a dataset of over 10k term familiarity and information need annotations from 11 cs researchers. our dataset reveals significant variation in term familiarity based on annotators' background ( §4.4). when predicting familiarity, subdomain information and sampled paper abstracts written by the author can improve prediction. this is particularly evident in prompt-based methodologies, where the use of paper abstracts was more beneficial than term familiarity labels from that researcher. this might be because researcher abstracts/metadata provide more generalizable information about the annotator than individual-specific term familiarity labels. the high levels of variation, of both familiarity labels ( §4.4) and model performance ( §6) across annotators, indicate that certain models work much better for certain annotators than others. personalized scientific jargon identification is a difficult task, with no clear best modeling strategy, and performance is dependent on the individual researcher being modeled. this opens up new research questions and avenues for future work. for example, we see that collaborative oracles saw noticeable performance boosts, suggesting that taking ratings from similar researchers might improve over generalpurpose jargon identification methods. subdomain information was also helpful for prediction, suggesting that modeling familiarity at the subdomain level can bring benefits beyond current general jargon identification techniques. furthermore, for researchers with a small publication history (and therefore a small amount of unlabeled individuallevel data), subdomain data can provide valuable information for predicting jargon familiarity. our methods provide an exciting first step to support researchers in reading and communicating outside their domain @xcite . combined with text simplification techniques @xcite , models that can predict an individual researcher's familiarity could assist researchers by rewriting an abstract tuned to particular research audiences, a capability that our formative study findings highlighted as a need for researchers ( §3). ## conclusion this paper introduces the novel task of scientific jargon detection for the individual researcher. we collect and release a dataset of over 10k term familiarity annotations from computer science researchers and investigate supervised and promptbased methods to predict term familiarity. we find that leveraging a researcher's publication history, self-reported subdomain, and general domain information can improve term familiarity prediction. our results provide insight on integrating an individual's knowledge into scientific jargon detection.