Improving Controllable Text Generation with Position-Aware Weighted Decoding (2022)

Abstract: Weighted decoding methods composed of the pretrained language model (LM) and the controller have achieved promising results for controllable text generation. However, these models often suffer from a control strength/fluency trade-off problem as higher control strength is more likely to generate incoherent and repetitive text. In this paper, we illustrate this trade-off is arisen by the controller imposing the target attribute on the LM at improper positions. And we propose a novel framework based on existing weighted decoding methods called CAT-PAW, which introduces a lightweight regulator to adjust bias signals from the controller at different decoding positions. Experiments on positive sentiment control, topic control, and language detoxification show the effectiveness of our CAT-PAW upon 4 SOTA models.

## introduction controllable text generation is a challenging task in natural language generation, which aims to generate diverse text related to specified attributes. dominating studies follow pplm @xcite and adopt a weighted decoding strategy @xcite @xcite . they usually employ an external controller with weight λ to bias the output distribution of a fixed pretrain lm. and the weight λ is positively correlated to control strength, thereby achieving strength-adjustable controllable text generation. however, those weighted decoding methods suffer from a trade-off problem between control strength and text fluency. as illustrated in figure 1 , when control strength increases, fluency of text generated by these sota models such as pplm @xcite , fudge @xcite , gedi @xcite , and dexperts 1 cat-paw stands for controllable text generation with position-aware weighted decoding. based on our analysis, the trade-off is due to the controller assigning bias signals to all decoding positions while ignoring the original results of lms. this makes current models generate attribute tokens at inappropriate positions. take military topic control task and pplm model as an example, which is shown in figure 2 . with prefix the potato and a relatively high weight λ = 0.09, pplm attempts to generate text highly relevant to military. when it comes to the decoding step at token first, candidate tokens of the lm are unrelated to the military topic, but the controller enforces a military bias, which causes pplm to generate the sentence pplm: the potato is the most popular vegetable in europe and is used in many european countries, including belgium, greece, italy... pplm: the potato plant has been the main target of a massive anti-pest attack by the government in china. the plant was the target of a massive attack from the army... λ = 0.09 λ = 0.06 λ = 0.03 pplm: the potato was a great food staple, and it was also one of the world's first war weapons. the potato was the first weapon to make war possible, and it was war war for war... cat-paw: navy / army / army / navy / military / empire / royal / east / royal / troops gpt-2: empire / army / isles / navy / navy / east / army / people / colonies / royal gpt-2: domest / vegetables / crops / fruits / foods / plants / edible / food / cultivated / to cat-paw: major / domest / crops / foods / vegetables / great / food / fruits / known / to λ = 0.09 gpt-2: domest / vegetables / crops / fruits / foods / plants / edible / food / cultivated / to pplm: war / mass / food / inventions / to / industrial / major / nuclear / weapons / foods ## method in this section, we first introduce current weighted decoding methods and analyze how they induce the trade-off. then we describe the general framework cat-paw composed of an lm, a controller, and our regulator module. last we illustrate two designs of our regulator. ## experiments in this section we first describe our evaluation metrics and baseline models. then we verify our cat-paw on positive sentiment control, topic control, and language detoxification. for each task we discuss its specific challenges, detailed configurations, and experiment results. ## related work controllable text generation @xcite is widely studied by previous work using custom neural networks @xcite @xcite and vae architectures @xcite . with the advancement of language modeling and pretraining @xcite @xcite @xcite attempt to modify or fine-tune a pretrained lm controlled by target attributes. as the size of lms expands exponentially @xcite , there emerge two main control methods with lm fixed. one is the prompt-tuning-based method @xcite , which attempts to guide the lm's generation behavior with prompts learned by fine-tuning @xcite or reinforcement learning @xcite . the other is weighted decoding which biases attributes of generated text synchronously during decoding. pplm @xcite ) biases lm's decoding with gradients from an attribute specified classifier. gedi @xcite applies bayes rule to decompose conditional generation probability into an lm and a generative classifier. fudge @xcite tries bayes rule similarly while training a classifier considering one future token ahead. dexperts @xcite ensembles probabilities from general lm and attribute-conditioned lms. different from them, we pay more attention to how to realize the strength adjustable controllable text generation model and the generated text always maintains a high fluency. ## conclusion in this work, we focus on weighted decoding based controllable text generation and devote to alleviating the control strength/fluency trade-off. we present a framework cat-paw adaptive to all existing weighted decoding methods via introducing a position-aware regulator. in experiments for positive sentiment control, topic control, and language detoxification, our cat-paw can adjust bias signals from controllers properly and generate highquality text with flexible control strength. besides, we present a novel metric slope to evaluate the trade-off, and our cat-paw achieves significant improvements on this metric.