Set Generation Networks for End-to-End Knowledge Base Population (2021)

Abstract: The task of knowledge base population (KBP) aims to discover facts about entities from texts and expand a knowledge base with these facts. Previous studies shape end-to-end KBP as a machine translation task, which is required to convert unordered fact into a sequence according to a pre-specified order. However, the facts stated in a sentence are unordered in essence. In this paper, we formulate end-to-end KBP as a direct set generation problem, avoiding considering the order of multiple facts. To solve the set generation problem, we propose networks featured by transformers with non-autoregressive parallel decoding. Unlike previous approaches that use an autoregressive decoder to generate facts one by one, the proposed networks can directly output the final set of facts in one shot. Furthermore, to train the networks, we also design a set-based loss that forces unique predictions via bipartite matching. Compared with cross-entropy loss that highly penalizes small shifts in fact order, the proposed bipartite matching loss is invariant to any permutation of predictions. Benefiting from getting rid of the burden of predicting the order of multiple facts, our proposed networks achieve state-of-the-art (SoTA) performance on two benchmark datasets.

## introduction nowadays, knowledge bases (kbs) are valuable resources, which can provide back-end support for various knowledge-centric services of real-world applications, such as question answering systems @xcite , dialogue systems @xcite and recommendation systems @xcite . however, high-quality kbs still rely almost exclusively on human-curated structured or semistructured data @xcite . such a reliance on human curation is a major barrier to creating always-up-to-date kbs. to overcome this barrier, knowledge base population (kbp) is proposed @xcite , which is a task of automatically discovering facts about entities from texts and expanding the incomplete kb with these facts. as shown in conventionally, kbp is solved by several individual components in a pipeline manner @xcite @xcite @xcite , typically including (1) entity discovery or named entity recognition @xcite , (2) entity linking @xcite and (3) relation extraction @xcite . entity discovery seeks to locate and classify named entities mentioned in text into predefined categories (e.g., people, organizations and locations). entity linking is a task to disambiguate these recognized entity mentions by linking them to a reference kb. relation extraction aims to predict semantic relations between pairs of entities. though widely used in practice, this pipeline architecture is inherently prone to error propagation between its components @xcite . to alleviate error propagation, some end-to-end kbp methods are proposed, such as @xcite . these methods are all based on the sequence-to-sequence (seq2seq) framework @xcite . under this framework, end-to-end kbp is treated as a translation of a sentence into a sequence of fact elements (entity or predicate). considering the running example in ## methodology the goal of end-to-end kbp is to identify all possible facts @xmath3, r 1 , t 1 > , ..., < h n , r n , t @xmath4. to solve this task, we propose endto-end set generation networks, which are shown in figure 1 . four key components of the proposed neural networks will be elaborated in the following section. concretely, we first present the joint learning of word and entity embeddings in section 2.1, which are the basis of the proposed networks. next, we introduce the sentence encoder in section 2.2, which can represent each token in a given sentence based on its bidirectional context. then, we illustrate the set generator in section 2.3, which is based on a non-autoregressive decoder to generate a set of facts in a single pass. finally, we describe a set-based loss in section 2.4, called bipartite matching loss, which forces unique matching between predicted and ground truth facts. transformer encoder ... ... fact queries input sentence sentence encoder non-autoregressive decoder output layer predictions ground-@xmath5, p39, @xmath6, p39, @xmath7: the main architecture of set generation networks. the set generation networks predict the final set of facts in parallel by combining a transformer-based encoder with a non-autoregressive decoder. in the training phrase, bipartite matching uniquely assigns predictions with ground truths to provide accurate training signals. ## experiments in this section, we carry out an extensive set of experiments with the aim of answering the following research questions (rqs): in the remainder of this section, we describe the datasets, experimental settings (in the appendix), and all baselines. ## conclusion in this paper, we introduce set generation networks for end-to-end kbp. compared with previous seq2seq models, we formulate the kbp task as a set generation problem. in such a way, the model will be relieved of predicting the order between multiple facts. to solve the set generation problem, we combine non-autoregressive parallel decoding with the bipartite matching loss function. to validate the effectiveness of the proposed networks, we conduct extensive experiments. experimental results show our proposed networks outperform current sota baselines over different scenarios.