Exploiting Unary Relations with Stacked Learning for Relation Extraction (2022)

Abstract: Relation extraction models typically cast the problem of determining whether there is a relation between a pair of entities as a single decision. However, these models can struggle with long or complex language constructions in which two entities are not directly linked, as is often the case in scientific publications. We propose a novel approach that decomposes a binary relation into two unary relations that capture each argument’s role in the relation separately. We create a stacked learning model that incorporates information from unary and binary relation extractors to determine whether a relation holds between two entities. We present experimental results showing that this approach outperforms several competitive relation extractors on a new corpus of planetary science publications as well as a benchmark dataset in the biology domain.

## introduction for many scientific domains, information extraction (ie) systems can play a valuable role in harvesting information from the scientific literature to automatically populate knowledge bases. our work is motivated by the goal of populating a mars knowledge base by extracting information from the planetary science literature about observations made by the rovers on mars. specifically, we seek to extract information about the composition and properties of named "targets" (rocks, soils, dunes, etc.) on the surface of mars. figure 1 shows an example sentence from this domain. several hypotheses could explain the abundance of potassium feldspar observed by chemin x-ray diffraction of the windjana drill sample. our ie task requires identification of three types of entities (components, properties, and targets) and two types of relations @xcite . the component entities can be minerals or elements. the sentence in figure 1 mentions one target (windjana) and one component (potassium feldspar), which participate in a contains relation. intuitively, this relation means that potassium feldspar was detected at the windjana site on mars foot_0 . typically, relation extraction (re) systems determine whether a pair of entities participate in a relation. in many scientific domains, relation extraction can be challenging because of complex language constructions that do not directly link two relevant entities, even when they occur in the same sentence. for example, the relation in figure 1 derives from the following complex path: potassium feldspar was observed by x-ray diffraction ... diffraction of a drill sample ... a drill sample taken at the windjana site. this type of sentence structure is challenging for nlp systems to recognize, both lexically and syntactically. however, even in long or complex sentences, our intuition is that local context is often sufficient to recognize one argument of a relation, even when recognizing both arguments simultaneously is difficult. to explore this hypothesis, our research decomposes two-argument (binary) relations into a pair of one-argument unary relations and trains separate unary relation extractors for each argument. for example, let us revisit figure 1 and the contains relation. we can decompose the binary relation contains(x,y) into two unary relations: container(x) and containee(y). in figure 1 , the phrase potassium feldspar observed strongly suggests the unary relation containee(potassium feldspar) (i.e., potassium feldspar is part of the composition of something). the phrase windjana drill sample suggests the unary relation con-tainer(windjana) (i.e., windjana was studied (drilled) for its composition). when the local context around one argument is compelling, the unary relation extractor can provide a strong signal that the full binary relation may also exists. but challenges remain with unary relation extractors alone: (1) only one argument may be recognized, and (2) it can be challenging to pair up the individual unary relations correctly. consequently, we expect that unary relations can be most useful when considered alongside other features to accurately extract binary relations. in this paper, we present a stacked learning architecture for relation extraction that uses a traditional binary relation extraction model alongside new information from unary relation extractors and features about the entity pair (section 4). in our stacked learning framework, a meta-classifier makes a decision about a pair of entities based on two perspectives of the sentence context: the broad perspective of the binary relation extraction model and the local perspectives of the corresponding unary relation extractors. as a result, the metaclassifier can be more robust then either approach on its own. we evaluate this stacked learning model on relation extraction tasks for two scientific domains: the mars mission planetary science domain and a biology domain (chemical-protein interactions) (section 5). we find that our stacked learning model consistently outperforms traditional binary relation extraction models in both domains. ## related work many relation extraction models have used featurebased or kernel-based approaches, such as @xcite @xcite . recent relation extraction models often use deep learning methods to learn representations of entities and their contexts and avoid the need for manual feature engineering (e.g., @xcite @xcite @xcite . many of these methods also fine-tune pretrained language models to better capture contextual information. such models include bert @xcite , scibert @xcite , which is pretrained over scientific publications, and linkbert @xcite , which is pretrained to also capture dependencies between documents. pipeline architectures have been widely used for relation extraction, which perform entity recognition as the first stage and then extract relations among the detected entities (e.g., @xcite @xcite ). another approach is to perform entity recognition and relation extraction jointly, which aims to eliminate the problem of error propagation that can occur with pipelines (e.g., @xcite @xcite @xcite . nearly all previous systems make decisions about a relation based on all arguments at the same time. one exception that bears some similarity to our work is @xcite , which trains a classifier to recognize the first argument ("subject") of a relation before trying to detect its second argument ("object"). however, their classifier uses information about the subject to identify the object, and then uses both arguments to make its final decision. in contrast, our unary models completely decouple the tasks of recognizing the first and second arguments of a binary relation. there has been growing interest in information extraction from scientific publications across a variety of domains (e.g., @xcite @xcite @xcite @xcite . however, relatively little work has been done for planetary science. the geodeepdive project extracts information about rock formations and stratigraphy on earth from geology publications @xcite . the mars target encyclopedia project @xcite extracts named entities (targets, minerals, and elements) and compositional relations from planetary science publications. their relation extraction component used jsre @xcite , an svm classifier based on shallow parsing features. we included their data (covering one mars mission and one relation, contains) in our experiments, and we augmented it with three more missions, hundreds of additional documents, and a new relation, hasproperty (see section 5.1). we compare the performance of jsre models with our relation extraction models in section 5.4. ## mars target relations our study focuses on relation extraction tasks in the planetary science domain. rovers and landers have been exploring the surface of mars for decades, and the science teams directing their activities have identified and named thousands of individual observation targets (rocks, soils, dunes, etc.). these targets are mentioned in subsequent scientific publications in conference and journal venues. our goal is to construct a relation extraction system that can successfully identify statements about the composition and properties of mars targets. we assume that entities of type target, component (element or mineral), and property (e.g., "layers", "dusty", "pits") have already been identified within the text, and the relation extraction system must determine which pairs of entities exhibit a given relation. we study two relations of interest: contains(target, component) and hasproperty(target, property). an example of the contains relation was shown in figure 1 . the sentence below includes three instances of the hasproperty relation. the dark rocks such as barnacle bill are more silica-rich, while the bright rocks such as yogi and wedge are more sulfur-rich and probably more weathered. the complete set of relations includes: hasproperty(barnacle bill, dark), hasproperty(yogi, weathered), hasproperty(wedge, weathered), contains(barnacle bill, silica), contains(yogi, sulfur), and contains(wedge, sulfur). it is common in this domain for multiple targets to share a relation with the same property (or the same component in the contains relation). conversely, it is also common for multiple properties or components to share a relation with a single target. relation extraction for this domain can be quite complex even when focusing on a single sentence. additional challenges arise from the use of abbreviations for mineral names (e.g., fe for iron), locally defined shorthand such as bb for barnacle bill, complex grammar with multiple clauses per sentence, and "hedging language" @xcite that captures the uncertainty about properties of targets on another planet. examples of hedging occur as the words "likely" and "possibly" in this sentence: the big sky tailings were spectrally flat (similar to telegraph peak) likely from the presence of magnetite, and include a weak @xmath0, possibly from minor hematite. this complex sentence entails two relations: contains(big sky, magnetite) and contains(big sky, hematite). ## experiments we conducted experiments to evaluate the stacked relation extraction approach on the planetary science document collection as well as a benchmark data set to compare directly with recent prior work. we release the dataset and codes at https://github.com/yyzhuang1991/ stackedlearningwithunarymodels . ## analysis we performed additional manual analyses to better understand the behavior of our stacked models. we performed ablation experiments to assess the contributions of different components of the stacked ensemble by separately removing the unary relation features (unary), binary relation features (binary), and entity pair features (ep) from the best stacked model svm linkbert . next, we examined whether the stacked model extracts relations from sentences with more entities better. figure 3 shows a graph that plots the f1 scores of svm linkbert and linkbert against the number of entity pairs in a sentence for the contains relation. the stacked model performs comparably to the binary relation extractor over sentences with fewer entity pairs, but it consistently outperforms the binary relation extractor over sentences with more than 10 pairs of entities. we hypothesize that it is important to filter out entities that do not participate in any unary relation when there are many entities in a sentence. by recognizing unary relations, the stacked model is able to handle the complexity of a large number of entity pairs. finally, ## conclusions and future work the goal of this work is to perform an automated analysis of scientific publications that enables the construction of domain-specific knowledge bases. we focused on the planetary science discipline, which to date has not received much attention from automated information extraction work. the complex grammar often employed in scientific publications can pose problems for state-of-the-art relation extraction systems. we proposed the use of unary relation extractors to enable specialization for each argument of a relation, within a stacked learning framework. our approach performed well both in this domain and the chemprot benchmark (biology) data set. in future work, we plan to expand the scope of this approach to include relations that cross sentences, which is a major challenge for current relation extraction systems and for which local unary relation modeling is especially well suited.