R e S el: N-ary Relation Extraction from Scientific Text and Tables by Learning to Retrieve and Select (2022)

Abstract: We study the problem of extracting N-ary relation tuples from scientific articles. This task is challenging because the target knowledge tuples can reside in multiple parts and modalities of the document. Our proposed method ReSel decomposes this task into a two-stage procedure that first retrieves the most relevant paragraph/table and then selects the target entity from the retrieved component. For the high-level retrieval stage, ReSel designs a simple and effective feature set, which captures multi-level lexical and semantic similarities between the query and components. For the low-level selection stage, ReSel designs a cross-modal entity correlation graph along with a multi-view architecture, which models both semantic and document-structural relations between entities. Our experiments on three scientific information extraction datasets show that ReSel outperforms state-of-the-art baselines significantly.

## introduction scientific information extraction (sciie) @xcite @xcite , the task of extracting scientific concepts along with their relations from scientific literature corpora, is important for researchers to keep abreast of latest scientific advances. a key subtask of sciie is the n -ary relation extraction problem @xcite , which aims to extract the relations of different entities as n -ary knowledge tuples. this problem is challenging because the entities of the knowledge tuples often reside in multiple sections (e.g., abstracts, experiments) and modalities (e.g., paragraphs, tables, figures) of the document. effective scientific n -ary relation extraction requires not only understanding the semantics of different modalities, but also performing document-level inference based on interleaving sig1 our code is available on https://github.com/ night-chen/resel . nals such as co-occurrences, co-references, and structural relations, as shown in figure 1 . document-level n -ary relation extraction has been studied in literature @xcite @xcite . some works @xcite use graph-based approaches to model long-distance relations in the document with the focus on text only. however, for scientific articles, an equally if not more important data structure is the table, as scientific results are often reported in tables and then referred to and discussed in text. there are also works that pre-train large-scale transformer models on massive table and text pairs @xcite . these methods are designed for question answering, which are strong at retrieving answers that semantically match the query but fall short in inferring fine-grained entitylevel n -ary relations. besides, to perform well on sciie, they usually require large task-specific data to fine-tune the pre-trained model, especially for long documents that contain many candidates. but in practice, such large-scale annotation data can be expensive and labor-intensive to curate. therefore, extracting n -ary relations jointly from scientific text and tables still remains an important but challenging problem. we propose resel, a hierarchical retrieve-and-selection model for multi-modal and documentlevel sciie. in resel, we pose the n -ary relation extraction problem as a question answering task over text and tables (figure 1 ). resel then decomposes the challenging task into two simpler subtasks: (1) high-level component retrieval, which aims to locate the target paragraph/table where the final target entity resides, and (2) low-level entity extraction, which aims to select the target entity from the chosen component. for high-level component (i.e., paragraph or table) retrieval, we design a feature set that combines the strengths of two classes of retrieval methods: (1) sparse retrieval @xcite ) that represents the query-candidate pairs as high-dimensional sparse vectors to encode lexical features; (2) dense retrieval @xcite ) that leverages latent semantic embeddings to represent query and candidates. we design sparse and dense retrieval features for query-component pairs by augmenting bert @xcite -based semantic similarities with entity-level semantic and lexical similarities, allowing for training an accurate high-level retriever using only a small amount of labeled data. the low-level entity extraction stage aims to infer n -ary entity relations from complex and noisy signals across paragraphs and tables. in this stage, we first build a cross-modal entitycorrelation graph, which encodes different entityentity relations such as co-occurrence, co-reference, and table structural relations. while most of the existing methods @xcite use bert embeddings as node representations, we find bert embeddings limited in distinguishing adjacent table cells or similar entities. this issue is even more severe when the bert embeddings are propagated on the graph. to address this, we design a new bag-of-neighbors (bon) representation. it computes the lexical and semantic similarities between each candidate entity and its 1hop neighbors. we then feed the bon features into a graph attention network (gat) to capture both neighboring semantics and structural correlations. such gat-learned features and bert-based embeddings are treated as two complementary views, which are co-trained with a consistency loss. we summarize our key contributions as follows: n -ary relation extraction many existing methods @xcite @xcite treat n -ary relation extraction as a binary classification problem and predict whether the composition of n entities in the document are valid or not. however, the candidate space grows exponentially with n , and the performance of the binary classifiers can be largely influenced by the number and quality of negative tuples. some other methods @xcite formulate the problem as role-filler entity extraction and propose bert-based generative models to extract the correct entities for each element of the n -ary relation. none of these methods consider n -ary relation across modalities. @xcite leverages the layout information for extracting relations from web pages. however, the layout information in science articles are less prominent and harder to be utilized. stage i: high-level component retriever stage ii: low-level entity extractor @xmath0, english, arc-hybrid, @xmath1: what is the score corresponding @xmath2, english, arc-hybrid, @xmath3 ## problem formulation in sciie, we aim to extract information from a corpus of m scientific articles. each article, denoted as d, is a sequence of |d| components, where each component c @xmath4, @xmath5,|d|) can be either a paragraph or a table. a table is flattened and concatenated with its caption as a sequence of words. given a document d, we have a set of queries q with number |q|. each query contains n -1 elements q @xmath6,1 , • • • , e j,n -1 ], @xmath7,|q|) , and the task is to extract the correct n -th element from document d to form a valid n -ary relation. we assume a dataset {x k , y k } m @xmath8, each sample includes a document and a set of queries x @xmath9, q k ), and each ground-truth label y k indicates the target entity in the document d k . ## conclusion we proposed resel, a two-stage method for nary relation extraction jointly from scientific text and tables. resel consists of two key components: a high-level component retriever and a lowlevel entity extractor. the multiple features defined in the high-level retriever enables our model to leverage semantic and lexical information from both paragraphs/tables and entities. for low-level entity extractor, the multi-view aggregation effectively encodes both the topology information from the graph and the semantic information from pretrained bert embeddings. extensive experiments on three datasets show that resel consistently outperforms all baseline models significantly.