Generating Scientific Definitions with Controllable Complexity (2022)

Abstract: Unfamiliar terminology and complex language can present barriers to understanding science. Natural language processing stands to help address these issues by automatically defining unfamiliar terms. We introduce a new task and dataset for defining scientific terms and controlling the complexity of generated definitions as a way of adapting to a specific reader’s background knowledge. We test four definition generation methods for this new task, finding that a sequence-to-sequence approach is most successful. We then explore the version of the task in which definitions are generated at a target complexity level. We introduce a novel reranking approach and find in human evaluations that it offers superior fluency while also controlling complexity, compared to several controllable generation baselines.

## introduction unfamiliar concepts and complex language can make understanding scientific information difficult for readers @xcite @xcite , especially because understanding such terms is highly dependent on their domain knowledge. given the wide variation in such knowledge, providing a onesize-fits-all definition may not be sufficiently understandable for all readers. we envision a software tool designed to aid readers with varying domain knowledge by automatically defining scientific terms. such a tool would afford readers control over generated definitions, including their complexity. this hypothetical system motivates research on automated generation of scientific definitions and generation-time control of definition complexity. prior work in generating definitions and personalizing generations to a reader falls short of these goals. most definition generation has focused on common, general-usage words in english @xcite a molecule that binds to a hydrophobic surface. a substance that dissolves water in a liquid or solute. ## dataset collection we collect a new dataset of definitions that are answers to the question "what is (are) x?" where x is a scientific term or concept (e.g., carbon nanotubes). these questions are drawn roughly equally from an existing qa dataset or templated from scientific glossaries. ## definition generation: basic models our first goal is to generate fluent definitions that include relevant and accurate information about the term being defined. because this is a new task and there are multiple reasonable approaches to generating fluent text @xcite , we experiment with four methods that have performed strongly in question answering and general-purpose definition generation and evaluate their effectiveness in this novel domain. for additional details on term dictionary definition dataset definition neuroblastoma a malignant tumor formed of embryonic ganglion cells neuroblastoma is a type of cancer that most often affects children. cirrhosis widespread disruption of normal liver structure by fibrosis and the formation of regenerative nodules that is caused by any of various chronic progressive conditions affecting the liver cirrhosis is scarring of the liver. antibiotics a substance able to inhibit or kill microorganisms; specifically : an antibacterial substance (such as penicillin, cephalosporin, and ciprofloxacin) that is used to treat or prevent infections by killing or inhibiting the growth of bacteria in or on the body summary: antibiotics are powerful medicines that fight bacterial infections. ## controlling definition complexity automatically generating definitions is an important first step in supporting readers who come are later evaluated on. hyperparameter tuning and finetuning were done on split subsets of the training data. controlling or guiding text generation is an active research area with important applications like toxicity control @xcite and language debiasing @xcite . for a review, see prabhumoye et al., 2020. to the best of our knowledge, ours is the first work to evaluate decoding-time controllable generation methods for text complexity. one task that has considered changing text complexity is text simplification. work on text simplification has mostly used a machine translation setup based on parallel corpora @xcite to translate complex sentences into simple ones. these parallel corpora are rare and often expensive to create @xcite . this setup also assumes an input text to be simplified @xcite , whereas our task expects that the text will be generated with varying complexity. ## evaluating complexity here we evaluate how well our baseline and novel generation control methods can vary the complexity of definitions. for each generation method, we generate and evaluate 10 definitions for each term. ## evaluating fluency, relevance, and factuality our results suggest that our reranking method is a simple intervention that can control complexity with similar performance as other state-of-the-art methods. however, definitions of scientific terms also must be fluent, relevant, and factual. factuality can be especially difficult to achieve in generations @xcite . in science communication such failures could spread misinformation with fluent but incorrect definitions @xcite . we do two additional human evaluations for fluency and relevance ( §7.1), and factuality ( §7.2). we used two trained annotators, one of them an author, to rate the same 300 definitions used in the complexity evaluation ( §6.2). neither annotator saw the model generations before evaluation or knew which method had generated each definition. ## conclusion we introduce a new task and dataset for generating definitions of scientific terms with controllable complexity as a way of adapting to different readers' scientific background. we evaluate conventional generation methods and introduce a lightweight approach of reranking candidate generations based on a discriminator to control complexity. we find that this reranking is effective at controlling text complexity while also maintaining fluency and factuality. we release our dataset and code to encourage more work on making scientific terms more accessible to readers of diverse background knowledge. 12