Gradient-guided Unsupervised Lexically Constrained Text Generation (2020)

Abstract: Lexically constrained generation requires the target sentence to satisfy some lexical constraints, such as containing some specific words or being the paraphrase to a given sentence, which is very important in many real-world natural language generation applications. Previous works usually apply beam-search-based methods or stochastic searching methods to lexically-constrained generation. However, when the search space is too large, beam-search-based methods always fail to find the constrained optimal solution. At the same time, stochastic search methods always cost too many steps to find the correct optimization direction. In this paper, we propose a novel method G2LC to solve the lexically-constrained generation as an unsupervised gradient-guided optimization problem. We propose a differentiable objective function and use the gradient to help determine which position in the sequence should be changed (deleted or inserted/replaced by another word). The word updating process of the inserted/replaced word also benefits from the guidance of gradient. Besides, our method is free of parallel data training, which is flexible to be used in the inference stage of any pre-trained generation model. We apply G2LC to two generation tasks: keyword-to-sentence generation and unsupervised paraphrase generation. The experiment results show that our method achieves state-of-the-art compared to previous lexically-constrained methods.

## introduction in many natural language generation applications, there are usually some constraints required to be satisfied by the generated sequences. the constraints can be classified into two types: previous works for lexically-constrained generation can be divided into two branches: enhanced beam search @xcite and stochastic search @xcite . among the various enhanced beam search methods, grid beam search @xcite is the most representative approach, which proposed to add candidate sequences that meet the lexical constraints to the beam in each step to constrain the search space. dynamic beam allocation methods @xcite are the extension of grid beam search, which groups the candidates that meet the same amount of constraints into banks to accelerate the inference process. however, the reason why beam search based methods work well with machine translation tasks is that the number of potential candidates in each step is relatively small. for general natural language generation tasks with a larger sentence space, beam search based methods will cost too much time to find a solution or even failed. stochastic search methods are very promising to solve the above problems. cgmh @xcite uses metropolis-hastings sampling to determine a series of editing actions to a given prototype sentence, including insertion, deletion, and replacement. upsa @xcite is a discrete optimization method, which focuses on generating paraphrased sentences in an unsupervised way. upsa creates a discrete objective function to evaluate semantic similarity and language fluency for a given candidate sentence, and use simulated annealing to search for the optimum solution with three editing actions (insertion, deletion, and replacement). however, in each of the stochastic search steps, many decisions are randomly chosen, such as which position is going to be edited, and which word is going to be replaced with. then, an accepted rate is applied to decide whether this action should be taken. this stochastic trial-and-error strategy will potentially lead to a waste of search steps. in this paper, we propose a novel method named g2lc for lexically-constrained generation. g2lc proposes a differentiable objective function that is able to evaluate the semantic similarity, the language fluency, and whether the constraints are satisfied. we use back-propagation to obtain the gradient on the representation of each word, then we propose to take the position with the largest gradient norm as the editing position for insertion, replacement, or deletion. when we choose to insert or replace, we first use a fuzzy word as the inserted or replacement word, which is then to be updated by an optimizer, such as adagrad @xcite . the word token will be replaced with a new word if the updated word representation is closed enough to the embedding of that new word. our g2lc can be applied to a large variety of tasks. in the experiment, we apply g2lc to two tasks: keyword-to-sentence generation and unsupervised paraphrase generation. the experiment results show that our method achieved state-of-theart performance compared to previous lexicallyconstrained generation methods foot_0 . our contributions can be summarized as follows: ## related works earlier works about lexically-constrained generation are mainly relying on auxiliary inputs to constrain the decoder outputs. for example, with the desired prefix, machine translation models are designed to search for the best suffix for the target sentence output @xcite @xcite @xcite . to make the keyword able to occur in the middle of the target sentence, @xcite proposed a backwardforward generation method which guarantees to contain only one keyword. for multiple keyword constraints, @xcite and @xcite proposed an interactive post-editing method, but these methods tend to bind the lexical constraints to the original model, which will lead to the retraining of the whole model if we would like to use existing generation model to conduct lexicallyconstrained generation. enhanced beam search is proposed as a plug-andplay method to lexically-constrained generation. grid beam search @xcite conduct beam search in two dimensions searching for the candidate sentences that satisfied the given lexical constraints. @xcite did a similar searching process using finite state automata. dynamic beam allocation methods @xcite @xcite are the extension of grid beam search, which groups the candidates that meet the same amount of constraints into banks to accelerate the inference process. although enhanced beam search works well in the tasks with limited search space such as machine translation, it will cost a lot of time on searching candidate sentences or even failed in general generation tasks when there is a much larger search space. stochastic search is also a plug-and-play method that can be applied to large search space. cgmh @xcite extends gibbs sampling with word insertion and deletion, then use metropolis-hastings sampling to choose new words for the editing position. for each searching step, an accept rate is calculated to decide whether this step should be conducted. the sampling process should be continued until the markov chain converges to a stationary distribution. upsa @xcite also uses a sampling method to decide editing actions and choose new words. instead of the markov chain monte carlo (mcmc) approach in @xcite , upsa models lexicallyconstrained generation as a discrete optimization problem and use simulated annealing to solve it. without the help of gradient, such approaches usually require more search steps to find the correct optimization paths. by contrast, our method is a gradient-guided method, which uses gradient's norm and direction to decide the editing position and the new word for replacement or insertion. nearly all generation model can be extended by plug-and-play method, including machine translation @xcite , table-to-text generation @xcite , dialogue response generation @xcite , and neural turing machine (ntm) driven methods @xcite . there exist two methods: (1) directly replace the original searching method. for example, enhanced beam search @xcite replaced the generation model's original beam search method. (2) modify the generated word token sequence. for example, stochastic search methods @xcite modify the sequence to maximize a designed score function. our proposed method can also be taken as a plug-in for existing generation methods, which belongs to the second plug-in method described above. in our method, the original generation model is used as a part of the score function. the largest difference between our method and previous works is that all of our score functions are differentiable, and the gradients can be used to indicate the edit positions and actions. ## approach although we also use the gradient for optimizing, our approach is inverse to the normal neural network training procedure. in our method, all model weights are separately trained and then fixed, instead, the words' representations are taken as the parameters to be trained. so, our objective must ensure to be differentiable w.r.t the words' representations (which is the input). ## conclusion & future works in this paper, we propose a gradient guided method to conduct unsupervised lexical constraint generation. the lexical constraints include hard constraints (keywords) and soft constraints (paraphrasing). we first defined a series of differentiable loss functions which represents the fluency of the generated sentence as well as whether the constraints are satisfied. then, we use the value of the gradient norm to decide which word in the sentence has the most urgent need to be edited, including being inserted in the front or the back, being deleted, and being changed. we applied our method in two tasks, keyword-to-sentence generation, and unsupervised paraphrasing. our method achieved state-of-the-art performance on both of these tasks. using post-editing methods for lexical constraint generation can be taken as an initial step of controlling the generation result. future research works can be conducted to make the generation process more robust and interpretable.