Hierarchical Relation-Guided Type-Sentence Alignment for Long-Tail Relation Extraction with Distant Supervision (2022)

Abstract: Distant supervision uses triple facts in knowledge graphs to label a corpus for relation extraction, leading to wrong labeling and long-tail problems. Some works use the hierarchy of relations for knowledge transfer to long-tail relations. However, a coarse-grained relation often implies only an attribute (e.g., domain or topic) of the distant fact, making it hard to discriminate relations based solely on sentence semantics. One solution is resorting to entity types, but open questions remain about how to fully leverage the information of entity types and how to align multi-granular entity types with sentences. In this work, we propose a novel model to enrich distantly-supervised sentences with entity types. It consists of (1) a pairwise type-enriched sentence encoding module injecting both context-free and -related backgrounds to alleviate sentence-level wrong labeling, and (2) a hierarchical type-sentence alignment module enriching a sentence with the triple fact’s basic attributes to support long-tail relations. Our model achieves new state-of-the-art results in overall and long-tail performance on benchmarks.

## introduction human-curated knowledge graphs (kgs), play a critical role in many downstream tasks but suffer from the incompleteness @xcite . as a remedy, relation extraction is to distinguish the relation between two entities according to their semantics in text, but a major obstacle is a lack of sufficient labeled corpus. fortunately, distant supervision can be used to annotate a raw text corpus via kgs for relation extraction, a.k.a. distantly supervised relation extraction @xcite . this is based on a strong assumption that a sentence containing two entities will express the semantics of their relation in a kg @xcite . the assumption cannot always hold, leading to the wrong labeling problem. for example, both this is the tale of the depression-era boxer james_j._braddock, played by russell crowe, who was described by the new_york_city, police. subject entity types object entity types people.person film.actor people.measured_person people.deceased_person blank location.location location.statistical_region /people/deceased_person /people location.dated_location location.citytown blank /people/deceased_person/place_of_death a 49-year-old man arrested in belfast this week was charged with murder in the killing of robert_mccartney, a 33-year-old, catholic, who was... subject entity types object entity types people.person government.politician org.org_founder blank location.location location.statistical_region /people/deceased_person /people location.dated_location location.citytown blank /people/deceased_person/place_of_death figure 1: two sentences with the same long-tail relation. for each sentence, multi-granular relations from top to bottom are pointed by its best pairwise types, which indicates not all pairwise types provide the same contribution. blue is subject entity, and red is object entity. the 1st sentence relies on the direct pairwise types due to its relation-irrelevant semantics while the 2nd sentence integrates its relation-relevant semantics and pairwise types to enhance its representation. to mitigate the long-tail problem, some works @xcite @xcite resort to the hierarchy of relations for knowledge transfer from data-rich relations to the long-tail ones since the relations have coarsegrained overlap. they focus on interactive operations between hierarchical relations and intra-bag sentences, including relation-to-sentence attention @xcite as a hierarchical extension of selective attention, and sentence-to-relation attention @xcite enriching sentences with multigranular relations. as such, they achieve knowledge transfer by learning to distinguish coarsegrained relations for sentences with sufficient data, which provides a latent constraint for the long-tail relations. however, a coarse-grained relation usually denotes the only basic attribute of the distant oracle triple fact in kg, so a sentence scarcely contains its semantics and we can only imply the relation via background information. again, truelabeled "jobs founded apple", does not explicitly contain any semantics of its coarse-grained relation "/business/company", but we can directly reason it from the predicate founded and type of apple. thus, it is a challenge for a hierarchical dsre model to correctly imply coarse-grained relations based solely on sentences, not to mention the existence of the wrong labeling problem. a direct yet promising way to overcome this challenge is to incorporate extra information for entities @xcite @xcite . one popular source is the entity types, i.e., an entity's "isa" attributes in kg, which characterizes the entity from multiple perspectives @xcite . as figure 1 shows, although the 1st sentence's semantics is irrelevant to relation, the pairwise types people.deceased_person and location.location directly align with the fine grained relation. however, existing works @xcite ignore this potential of explicit structured types information. in this work, we aim to improve dsre by exploiting structured information in the entity types from both pairwise and hierarchical perspectives to alleviate the wrong labeling and the long-tail problems respectively. to this end, we first propose a context-free type-enriched embedding module to generate word embeddings with pairwise types associated with the entity pair in a bag. as shown in figure 1 , even without the corresponding semantic support, pairwise types can provide direct attributes of entities to align with the relation. besides, we develop a context-related type-sentence alignment module to generate robust sentence representation with pairwise types. since entities have specific characteristic in certain semantics, we leverage semantics to select proper pairwise types and then enrich sentence representation, as the 2nd sentence in figure 1 shows. such an alignment is enhanced by a guidance from the relation to auto-seek for associations between pairwise types and sentences. at the meantime, hierarchical information has been proven crucial in knowledge transfer for longtail relations @xcite @xcite ). thereby, we naturally extend the base alignment module into a hierarchy by proposing a hierarchical type-sentence alignment module. an intuitive example in figure 1 shows that different grained relations are pointed by various granular pairwise types. this indicates that these pairwise types contain hierarchical semantics, which makes it feasible to extend base alignment into hierarchy. thus, the strong association between pairwise types and coarse-grained relations can improve knowledge transfer for long-tail relations. we conduct extensive experiments on two popular benchmarks, nyt-520k and nyt-570k, showing that our model achieves new state-of-the-art overall and long-tail performance. further analyses reveal insights into our model. ## related work wrong labeling problem. many works @xcite @xcite propose various extensions of vanilla selective attention @xcite . @xcite combine intra-/inter-bag level selective attention for dsre. for one-sentence bags, @xcite design the entity-aware embedding in a context-free manner with a gate mechanism. long-tail relations. knowledge transfer via hierarchical relations is effective. @xcite design relation-to-sentence attention in hierarchies, and @xcite modify it to sentence-torelation attention. many works @xcite @xcite resort to extra knowledge, i.e., entity description and entity types. entity description @xcite ) mainly stems from the wikipedia page, which contains factual statements of the relation with other entities. such oracle knowledge can boost dsre performance but is impractical. ## conclusion in this work, we propose a new model, hiram, training on a single titan xp, except for roberta w/ rtx 6000, to alleviate wrong labeling and longtail problems in dsre. for the wrong labeling problem, we propose a context-free type-enriched word embedding to enrich each word with prior knowledge and a context-related type-sentence alignment module to complement sentences with semantics-fitted pairwise types. for the long-tail problem, we extend the base alignment into the hierarchy to utilize the multi-granular entity types. the experiments with extensive analyses show the superiority of hiram.