Learning Disentangled Meaning and Style Representations for Positive Text Reframing (2023)

Abstract: The positive text reframing (PTR) task which generates a text giving a positive perspective with preserving the sense of the input text, has attracted considerable attention as one of the NLP applications. Due to the significant representation capability of the pre-trained language model (PLM), a beneficial baseline can be easily obtained by just fine-tuning the PLM. However, how to interpret a diversity of contexts to give a positive perspective is still an open problem. Especially, it is more serious when the size of the training data is limited. In this paper, we present a PTR framework, that learns representations where the meaning and style of text are structurally disentangled. The method utilizes pseudo-positive reframing datasets which are generated with two augmentation strategies. A simple but effective multi-task learning-based model is learned to fuse the generation capabilities from these datasets. Experimental results on Positive Psychology Frames (PPF) dataset, show that our approach outperforms the baselines, BART by five and T5 by six evaluation metrics. Our source codes and data are available online.

## introduction text style transfer (tst) has been a long history from the early works, i.e., the eariler attempts are the frame language-based systems @xcite and schema-based natural language generation @xcite in the 1980s, and more recent attempts such as ctpm (contrastive transfer pattern mining) @xcite and tst bt (text style transfer back translation) @xcite . the goal is to change the text style, such as formality, and politeness with preserving the sense of the input text. with a recent surge of interest in deep learning (dl) techniques, positive text reframing (ptr) has been explored as one of the sub-fields in the tst study. likewise, humanannotated data such as positive psychology frames (ppf) has been created for this task @xcite . one major approach for the tst task is to apply supervised learning for parallel data. xu et al. @xcite and zhang et al. @xcite attempted multi-task learning for parallel data. to mitigate the small size of the parallel data, rao @xcite presented data augmentation strategies. another attempt is to utilize a non-parallel dataset and train a model in an unsupervised manner @xcite . @xcite a method that disentangles content-and style-related features and makes the decoder generate ideal output by using the disentangled features @xcite . lai et al. designed two types of rewards for target style and content based on reinforcement learning @xcite . many of these methods attained significant progress on the tst task while they still fail to handle the fine-grained transfer, i.e., disentangle style from content with preserving the meaning of the input that is required for the ptr task. the main challenge in the ptr task is how to control diversity and the extent of style transfer. the concept of our ptr can be illustrated in figure 1 . the straightforward fine-tuning of plm, proposed by @xcite , is shown in the path, p ath 1 . we regard this strategy as our baseline it's difficult to finish this work. this work is easy. ## conclusion in this paper, we proposed a method for ptr that learns disentangled meaning and style representations. the method utilizes pseudo-positive refram-sentence 1 why is it so hard to get one day of alone time, of peace and quiet??? ground truth i could really do with some peace and quiet for a day. st 2p g i hope i get one day of alone time, of peace and quiet . baseline it's hard to get one day of alone time, of peace and quiet , but it's worth it when you get it. sentence 2 its like friday righ now with the stock market like wtf is going on lmaoo. ground truth it may be a confusing time, but i know that i can remain calm in the storm.