Diversifying Neural Text Generation with Part-of-Speech Guided Softmax and Sampling (2022)

Abstract: Neural text generation models are likely to suffer from the low-diversity problem. Various decoding strategies and training-based methods have been proposed to promote diversity only by exploiting contextual features, but rarely do they consider incorporating syntactic structure clues. In this work, we propose using linguistic annotation, i.e., part-of-speech (POS), to guide the text generation. In detail, we introduce POS Guided Softmax to explicitly model two posterior probabilities: (i) next-POS, and (ii) next-token from the vocabulary of the target POS. A POS Guided Sampling strategy is further proposed to address the low-diversity problem by enriching the diversity of POS. Extensive experiments and human evaluations show that, compared with existing state-of-the-art methods, our POS Guided Softmax and Sampling (POSG) can generate more diverse text while maintaining comparable quality.

## introduction maximum likelihood estimation (mle) is a standard approach to training a neural text generation model, e.g. transformer @xcite , to generate human-like text. however, existing generation systems often suffer from the low-diversity problem @xcite , which leads to dull and repetitive generations. this problem unavoidably affects the overall generation quality. we conclude that the low-diversity problem is mainly manifested in two aspects: form and content @xcite @xcite . as shown though those encouraging progress has been made, we argue that current training-based methods only take plain contextual features to promote diversity, rarely considering incorporating syntactic structure clues. for example, when humans are writing articles, it is natural to predetermine the part-of-speech (pos) before giving the next token. existing studies have verified that incorporating pos can improve the translation quality in neural machine translation (nmt) @xcite . intuitively, since the vocabularies of different pos vary a lot, the diversity of pos will certainly lead to the diversity of text. unfortunately, we observe that existing methods with no consideration of the inner pos structure fail to learn the diversity of pos in human language (shown in all these factors motivate us to address the lowdiversity problem with the guidance of pos. thus, in this work, we first present the pos guided soft-context: the nk 2nd division , concentrated in the sinban-ni area west of the river , had , in effect , attacked straight east across the river and was trying to seize the two avenues of advance into changnyong. text 1: they were joined by the 27th battalion, us 24th infantry regiment, the 27th regiment, and the 27th regiment. the us 24th infantry division, under the command of major general john r. dempsey, was ordered to charge the us 24th infantry division, and capture it from the west. text 2: the nk 2nd division, which had been involved in the assault, was forced to withdraw from the area north of the river. the nk 3rd division, which had been fighting in the area since the beginning of the battle, was moved to the south. the nk 2nd division, which had been fighting in the area since the start of the battle, had been pushed back to the south. text 3: the 2nd division had been moving north from alcester's position on the road, but were not expecting another attack. in the immediate aftermath of the attack, to keep the 2nd division in reserve, which had been preparing for an attack on hill 131. along with the 3rd battalion of the us 2nd infantry regiment, attacked hill 129 at pakchon on the way to beaulieu. to sum up, the contributions of our work are three-fold. recently, stochastic decoding methods have been widely used, and some studies propose to sample from a truncated and renormalized softmax distribution. top-k sampling @xcite only samples from the top-k most probable tokens. nucleus sampling @xcite only samples from the smallest set whose cumulative probability is at least α. however, those decoding-based methods are lack of controllability. combined with above methods, our proposed method can further promote diversity using pos as a more controllable clue. training-based methods. as a standard approach to training a neural text generation model, mle has been proved to be defective. @xcite have shown that mle may mislead the model because of the imbalanced token distribution. thus, they design a greedy approach mefmax and factorize softmax to ensure a balanced training according to the word frequency. face @xcite utilizes the target word frequency to modify the cross-entropy loss with a frequency-based weight factor. @xcite introduce an unlikelihood loss to implicitly reduce the frequent tokens and potential repeats. other approaches, such as negative training @xcite , reinforcement learning @xcite , and imitation learning (zhou and lampouras, 2020), have recently been applied to promote the diversity during the training phase. all above training-based methods only learn from plain contextual features, while ignoring other linguistic features. our focus is on leveraging pos features to guide both phases of training and decoding. ## language modeling the goal of language models is to assign a probability to text (i.e. word sequence) @xmath0, . . . , x t ], where each x t in the sequence is a token from a vocabulary v, i.e., x @xmath1, and @xmath2. we train the language models to learn a distribution p θ (x) with the goal to fit the ground-truth distribution p ⋆ (x) for all x. specifically, when the language model is a neural network, θ is regarded as the model parameters of the neural network, and we can factorize p θ (x) as p θ (x) = π t @xmath3. the conven-tional approach for learning the language model parameters θ is to maximize the log-likelihood by minimizing: ## methodology in this section, we describe an overview of our proposed method, pos guided softmax and sampling (posg). posg is designed to exploit syntactic structure, i.e., pos tags for text generation in both the training and decoding phases. specifically, giving text sequence @xmath4, . . . , x t ], we first use off-the-shelf pos tagger @xcite to annotate corresponding pos sequence ρ = [ρ 1 , . . . , ρ t ], where each ρ t is a pos tag from the pos vocabulary p, i.e., ρ @xmath5, and @xmath6. we define all the tokens whose pos is ρ as a vocabulary v ρ , where v ρ ⊂ v. ## experiments we systematically evaluate our proposed methods on language modeling task (section 5.2) and paraphrase generation task (section 5.3). ## analysis 6.1 interpretability compared with one-stage sampling such as top-k sampling, posg will lead to the entropy increasing of a language model's distribution, and thus lead to more diverse outputs (see appendix b for the proof, appendix c.1 for experimental results). ## conclusion in this paper, we have introduced pos guided softmax and sampling, simple but effective methods to address the low-diversity problem in text generation. posg guides models to capture contextual and syntactical information by leveraging pos as an observed and controllable clue in both the training and decoding phases. experimental results and human evaluation on language modeling and paraphrase generation have demonstrated the effectiveness of our methods. international conference on learning representations, iclr 2018, vancouver, bc, canada, april 30 -may 3, 2018, conference track proceedings. open-review.net. tianyi zhang, varsha kishore, felix wu, kilian q. weinberger, and yoav artzi. 2020. bertscore: evaluating text generation with bert. in in proceedings of the 8th international conference on learning representations. giulio zhou and gerasimos lampouras. 2020. generating safe diversity in nlg via imitation learning. arxiv preprint arxiv:2004.14364. yaoming zhu, sidi lu, lei zheng, jiaxian guo, weinan zhang, jun wang, and yong yu. 2018. texygen: a benchmarking platform for text generation models. in proceedings of the 41st international acm sigir conference on research & development in information retrieval, pages 1097-1100.