Injecting Entity Types into Entity-Guided Text Generation (2021)

Abstract: Recent successes in deep generative modeling have led to significant advances in natural language generation (NLG). Incorporating entities into neural generation models has demonstrated great improvements by assisting to infer the summary topic and to generate coherent content. To enhance the role of entity in NLG, in this paper, we aim to model the entity type in the decoding phase to generate contextual words accurately. We develop a novel NLG model to produce a target sequence based on a given list of entities. Our model has a multi-step decoder that injects the entity types into the process of entity mention generation. Experiments on two public news datasets demonstrate type injection performs better than existing type embedding concatenation baselines.

## introduction entity, as an important element of natural language, plays the key role of making the text coherent @xcite . recently, modeling entities into nlg methods has demonstrated great improvements by assisting to infer the summary topic @xcite or to generate coherent content @xcite . to enhance the representation of entity, entity type is often used in existing work -represented as a separate embedding and concatenated with the embedding of entity mention (i.e., surface name) in the encoding/decoding phase @xcite @xcite . although the concatenation performed better than using the entity mention embedding only, the relationship between entity mention and entity type was not reflected, making the signal from entity type undermined in the nlg. to address the above issue, our idea is to model the entity type carefully in the decoding phase to in this paper, we propose a novel method of utilizing the type information in nlg, called inj-type. it keeps the same encoder as seq2seq models @xcite . during decoding, it first predicts the probability that each token is a contextual word in the vocabulary or an entity from a given list. if the token is an entity, the model will directly inject the embedding of the entity type into the process of generating the entity mention by using a mention predictor to predict the entity mention based on the type embedding and current ent ent ent ent … us dick_cheney afghanistan country person country + + + > … us dick_cheney afghanistan monday vice president made a visit to on weekday country person country type injection mention predictor encoder decoder ## proposed method: injtype in this section, we first give the task definition, then introduce our proposed type injection method. we note that injtyp users the same encoder as in seq2seq models @xcite , i.e., a bidirectional gru. so, in figure 1 and the following sections, we only describe the decoding process. task definition given a list of entities @xmath0, . . . , x n ), where x @xmath1, x t @xmath2, where m is the set of entity mentions and t is the set of entity type. the expected output sequence is @xmath3, . . . , y m ) containing all the entity mentions. we denote the vocabulary of contextual words by v. so y @xmath4, @xmath5, . . . , m}. the task is to learn a predictive function f : @xmath6, mapping a list of entities to a target sequence. ## conclusions entity plays the key role of making the text coherent in news generation. in order to enhance the role input entities: china (country, c), russia (c), wto (organization, o), chinese (c), wen_jiabao (person, p), friday (weekday, w), moscow (l), interfax_news_agency (o) ground truth: china has agreed to back russia's entry into the wto, chinese prime minister wen jiabao said friday after talks in moscow, the interfax news agency reported . seq2attn: china has lodged a veto global wto plan to wto the global trade body at its senior chinese counterpart wen jiabao on friday, interfax news agency reported, as saying from moscow, interfax news agency said, quoted by the interfax news agency. [the second "wto" should be "russia". "interfax" appeared multiple times.] copynet: china and russia want to sell wto trade sanctions against the world_trade_organisation , warning the proposed sanctions against the wto agreement are at the wto agreement we are concerned , the report said after meeting with his meeting counterpart wen_jiabao on friday in moscow , the interfax_news_agency reported . injtype (our method): china has agreed russia's bid to join the wto opening, chinese foreign minister wen jiabao said friday moscow, report quoted the state official from interfax news agency.