Towards Attribute-Entangled Controllable Text Generation: A Pilot Study of Blessing Generation (2022)

Abstract: Controllable Text Generation (CTG) has obtained great success due to its fine-grained generation ability obtained by focusing on multiple attributes. However, most existing CTG researches overlook how to utilize the attribute entanglement to enhance the diversity of the controlled generated texts. Facing this dilemma, we focus on a novel CTG scenario, i.e., blessing generation which is challenging because high-quality blessing texts require CTG models to comprehensively consider the entanglement between multiple attributes (e.g., objects and occasions). To promote the research on blessing generation, we present EBleT, a large-scale Entangled Blessing Text dataset containing 293K English sentences annotated with multiple attributes. Furthermore, we propose novel evaluation metrics to measure the quality of the blessing texts generated by the baseline models we designed. Our study opens a new research direction for controllable text generation and enables the development of attribute-entangled CTG models.

## introduction controllable text generation (ctg) aims to automatically generate the text under the restrictions of given conditions @xcite @xcite . as the mainstream, controlling multiple attributes enriches the information contained by generation and matches the demand of application scenarios, such as generating chinese poetry @xcite , restaurant reviews @xcite , and product descriptions @xcite . take the chinese poetry generation task as an example, one beautiful poetry sentence should contain multiple attributes and reflect the entanglement (or mixture) of them through reasonable connection, e.g., in the sentence "胡马南来路已荒(the enemy's warhorses march to the south, through destroyed roads)", "胡马(enemy's warhorses)" is a representative element of the military career, "南 来(march to the south)" and "荒(destroyed)" represent the attribute of troubled times. this poetry sentence vividly depicts a picture of war in troubled times through the entanglement of attributes in just seven characters. @xcite also claim that considering the entanglement among attributes can effectively enhance the quality and diversity of generated poetry. therefore, we believe that better ctg models must focus on the effect of attribute entanglement, i.e., enhancing the reflection of multiple attributes through the use of various representative elements in the generated text. for chinese ctg, with poetry generation as a typical scenario, researchers have conducted indepth research on attribute entanglement, but in the english ctg field, the research on attribute entanglement has not been explored. therefore, to promote research on attribute-entangled ctg in the english community, in this paper we focus on blessing generation, a new ctg task that plays a key role in social scenarios. the automatically generated blessings will greatly promote interpersonal communication and enrich people's daily life. more crucially, the blessing generation task is challenging due to its high requirement for entanglement between attributes, such as objects and occasions. as shown in figure 1 , "santa's workshop" connects the occasion (christmas) and object (boss) into one phrase, making the blessing wonderful. a more vivid blessing embodies these two attributes in an intertwined manner, such as "keeps the office humming along like santa's workshop". facing the vacant of blessing generation, we construct eblet, a large-scale entangled blessing text dataset annotated with multiple attributes. particularly, the eblet is constructed with the following two features: (1) eblet contains 23 occasions and 34 objects annotated on 293,403 blessing texts from 12 blessing websites. (2) as 92% of the blessing texts are personalized for the corresponding attributes, eblet has at least 82% data containing the entanglement between attributes. additionally, the common generation evaluation metrics cannot reflect the characteristics of blessings clearly. to evaluate the generated blessings more comprehensively, we propose novel metrics to automatically calculate the degree of attribute entanglement and the quality of blessings. our experiments demonstrate that mainstream ctg methods struggle to contain the entanglement. moreover, existing methods can not balance the fluency, diversity, and entanglement between attributes. these results indicate that the blessing generation task we focus on is challenging and could serve as a useful benchmark for ctg research. ## task definition the blessing generation task aims to obtain a generation model g(x 1 , x 2 ; θ) parameterized by θ. given the input attributes containing an object x 1 ∈ x 1 and an occasion x 2 ∈ x 2 , the model g should output a blessing text y sent to x 1 for x 2 , where @xmath0, y 2 , ..., y n } is a sequence containing n words, and x i (@xmath1, 2) is a word or a phrase belonging to a collection of objects or occasions. the generated text y should reflect not only the language style of blessing, but also effective entanglement between both attributes. additionally, the evaluation metrics for the language style of blessing and entanglement are described in section 4. ## conclusion to explore the entanglement between attributes, we present eblet, a blessing dataset that presents a new controllable generation task. we propose novel metrics to automatically measure attribute entanglement and the quality of blessings. we also provide several baselines and conduct experiments for blessing generation. experimental results demonstrate that eblet could serve as a useful benchmark for attribute entanglement in ctg.