Graphine: A Dataset for Graph-aware Terminology Definition Generation (2021)

Abstract: Precisely defining the terminology is the first step in scientific communication. Developing neural text generation models for definition generation can circumvent the labor-intensity curation, further accelerating scientific discovery. Unfortunately, the lack of large-scale terminology definition dataset hinders the process toward definition generation. In this paper, we present a large-scale terminology definition dataset Graphine covering 2,010,648 terminology definition pairs, spanning 227 biomedical subdisciplines. Terminologies in each subdiscipline further form a directed acyclic graph, opening up new avenues for developing graph-aware text generation models. We then proposed a novel graph-aware definition generation model Graphex that integrates transformer with graph neural network. Our model outperforms existing text generation models by exploiting the graph structure of terminologies. We further demonstrated how Graphine can be used to evaluate pretrained language models, compare graph representation learning methods and predict sentence granularity. We envision Graphine to be a unique resource for definition generation and many other NLP tasks in biomedicine.

## introduction obtaining the definition is the first step toward understanding a new terminology. the lack of precise terminology definition poses great challenges in scientific communication and collaboration @xcite , which further hinders new discovery. this problem becomes even more severe in emerging research topics @xcite , such as covid-19, where curated definitions could be imprecise and do not scale to rapidly proposed terminologies. neural text generation @xcite @xcite could be a plausible solution to this problem by generating definition text based on the terminology text. encouraging results by neural text generation have been observed on related tasks, such as paraphrase generation @xcite , description generation @xcite , synonym generation @xcite and data augmentation @xcite . however, it remains unclear how to generate definition, which comprises concise text in the input space (i.e., terminology) and longer text in the output space (i.e., definition). moreover, the absence of large-scale terminology definition datasets impedes the progress towards developing definition generation models. despite these challenges, scientific terminologies often form a directed acyclic graph (dag), which could be helpful in definition generation. each dag organizes related terminologies from general ones to specific ones with different granu-larity levels (figure 1 ). these dags have proved to be useful in assisting disease, cell type and function classification @xcite @xcite by exploiting the principle that nearby terms on the graph are semantically similar @xcite . likewise, terminologies that are closer on this dag should acquire similar definitions. moreover, placing a new terminology in an existing dag requires considerably less expert efforts than curating the definition, further motivating us to generate the definition using the dag. in this paper, we collectively advance definition generation in the biomedical domain through introducing a terminology definition dataset graphine and a novel graph-aware text generation model graphex. graphine encompasses 2,010,648 terminology definition pairs encapsulated in 227 dags. these dags are collected from three major biomedical ontology databases @xcite @xcite . all definitions are curated by domain experts. our graph-aware text generation model graphex utilizes the graph structure to assist definition generation based on the observation that nearby terminologies exhibit semantically similar definitions. our human and automatic evaluations demonstrate the substantial improvement of our method on definition generation in comparison to existing text generation methods that do not consider the graph structure. in addition to definition generation, we illustrate how graphine opens up new avenues for investigating other tasks, including domain-specific language model pretraining, graph representation learning and a novel task of sentence granularity prediction. finally, we present case studies of a failed generation by our method, pinpointing directions for future improvement. to the best of our knowledge, graphine and graphex build up the first large-scale benchmark for terminology definition generation, and can be broadly applied to a variety of tasks. ## future work motivated by an opposite generation despite the overall improved performance of graphex, we found that some definitions generated by graphex present an opposite meaning to the truth definition. we showed one of such example in figure 6 . although the definition generated by graphex for hyperlasia matches the true definition well, the generated definition has the opposite semantic meaning (e.g., reduction, reduced) to the true definition (e.g., increase, increased). notably, such failed generations cannot be captured by existing n-gram based metrics, leading to artificial improvement. after a closer examination, we found that this opposite generation is caused by using cell number defective decreased cell number increased cell number terminology: hyperplasia true definition: phenotype that is an increase in size of a tissue or organ due to increased numbers of cells, where the affected tissue or organ maintains its normal form. ## relate work existing works related to terminology definition mainly focus on definition extraction @xcite @xcite and technology entity recognition @xcite . definitions are extracted from different sources, such as wikipedia @xcite and scholarly articles @xcite . in contrast to previous work, we study the novel problem of terminology definition generation. notably, the proposed dataset graphine can also be used as a new benchmark to evaluate existing approaches on extracting definitions from the free text. many scientific literature datasets have been curated for a variety of tasks, such as hypothesis generation @xcite , scientific claim verification @xcite , paraphrase identification @xcite @xcite and citation recommendation @xcite . paraphrase identification datasets, such as mscoco, quora, msr, parasci, are most related to our work @xcite @xcite . distinct from these datasets, we focused on a different task (i.e., definition generation) and a different domains (i.e., biomedical domain). graph2text and data2text, which aim at generating text from structured data, have attracted increasing attention @xcite @xcite @xcite . among them, amr-to-text generation and knowledge graph to text generation also consider graph structures. the abstract meaning representation (amr) represents the semantic information of each sentence using a rooted directed graph, where each edge is a semantic relations and each node is a concept @xcite @xcite . knowledge graph to text generation has advanced tasks such as entity description generation and medical image report by generating text from a subgraph in the knowledge graph @xcite . despite all considering graph structures, our method generates one sentence for each node on a large directed acyclic graph, whereas amr-to-text and knowledge graph to text generate sentences for a subgraph or the entire graph. ## conclusion we have introduced a novel dataset graphine for studying definition generation. graphine includes 2,010,648 terminology definition pairs from three major biomedical databases. terminologies in graphine form 227 directed acyclic graphs, which make graphine a unique resource for exploring graph-aware text generation. we have proposed a graph-aware definition generation method graphex, which takes the graph structure into consideration. graphex has obtained substantial improvement against methods that do not consider graph structures. moreover, we have illustrated how graphine can be used to evaluate other tasks, including comparing pretrained language models, comparing graph representation learning methods and predicting sentence granularity. finally, we have analyzed the definition generated by our method and proposed future directions to improve. collectively, we envision our dataset to be a unique resource for definition generation and could be broadly utilized by other natural language processing applications.