On Text Style Transfer via Style-Aware Masked Language Models (2023)

Abstract: Text Style Transfer (TST) is performable through approaches such as latent space disentanglement, cycle-consistency losses, prototype editing etc. The prototype editing approach, which is known to be quite successful in TST, involves two key phases a) Masking of source style-associated tokens and b) Reconstruction of this source-style masked sentence conditioned with the target style. We follow a similar transduction method, in which we transpose the more difficult direct source to target TST task to a simpler Style-Masked Language Model (SMLM) Task, wherein, similar to BERT (CITATION), the goal of our model is now to reconstruct the source sentence from its style-masked version. We arrive at the SMLM mechanism naturally by formulating prototype editing/ transduction methods in a probabilistic framework, where TST resolves into estimating a hypothetical parallel dataset from a partially observed parallel dataset, wherein each domain is assumed to have a common latent style-masked prior. To generate this style-masked prior, we use “Explainable Attention” as our choice of attribution for a more precise style-masking step and also introduce a cost-effective and accurate “Attribution-Surplus” method of determining the position of masks from any arbitrary attribution model in O(1) time. We empirically show that this non-generational approach well suites the “content preserving” criteria for a task like TST, even for a complex style like Discourse Manipulation. Our model, the Style MLM, outperforms strong TST baselines and is on par with state-of-the-art TST models, which use complex architectures and orders of more parameters.

## introduction text style transfer can be thought of as a form of controllable language generation @xcite with tighter criteria. @xcite show that a classifier trained on the final embeddings of any arbitrary large language model to learn class labels of any dataset, can guide subsequent generations to resemble this dataset's style by backpropagating signals from the classifier to the llms activation layers. this approach, as well as other generational approaches, generally suffer from the lack of the model's ability to preserve content. this content preservation criteria, where only style influencing words/phrases must be affected, forms the critical challenge in the style transfer task. learning style transfer from a parallel dataset is easy, i.e. where the output sentence for every target style is known and the model learns a straightforward mapping of input-output sentence pairs in a supervised manner. like most work, we assume the more realistic case, where the dataset is non-parallel, and the task is unsupervised. the style transfer task is equivalent to the task of estimating the hypothetical parallel dataset from a non-parallel or "partiallyobserved" parallel dataset. we use this notion in later sections to elucidate the mechanisms which enable our approach. style is usually represented using a data-driven approach, i.e. the class labels present in an annotated dataset. one could argue that for a majority of style transfer applications, only a subset of words participate in giving the input sentence its corresponding style. as examples, we can think of this being true in the case of sentiment, toxicity, formality, politeness etc, where a few word edits can lead to a style change while the other attributes of the sentence are disaffected. as expected, this type of approach has been vastly used in the style transfer task with good success. this prototype editing approach consists a) ranking tokens according to how much they affect the underlying style b) determining which subset of tokens to mask to produce a style absent sentence, and c) transforming this style absent sentence into one that contains the target style. we show that a single self-attention encoder block trained to reconstruct style-masked sentences to the original versions using a non-parallel dataset also learns style transfer automatically. this method, resembling a masked language model, compares with state-of-the-art models albeit having a simpler training process, lesser parameters and without using pre-trained language models. we consider a novel discourse manipulation task and show that the sa-mlm outperforms another stateof-the-art model in this respect. ## learning style transfer from only reconstruction many similarities appear between the editing approach for style transfer and bert's masked language model objective. the only difference being that we reconstruct the original sentence from its style-masked version, rather than random or perplexity-based-masking sentences in typical mlms. we postpone discussing our approach for style masking to the next section. in this section, we explore the question "how does an mlm trained only on a reconstruction task automatically gain the ability to perform the unseen style-transfer task?". an ensemble approach. intuitively, we can reason that just training a model to reconstruct sentences from style-masked versions ("the food @xmath0. since a) this does not give context to the model about styles present and b) does not give us any knob to control the output style. a simple solution would be training n different models, each to reconstruct one particular style. style transfer can be performed by feeding a style-masked sentence to that corresponding target style model. this however a) is not computationally efficient or scale-able b) limits the learning of each model to only a fraction of the dataset, thereby over-fitting to that target domain. we perform the style transfer task using this ensemble model consisting of two generational encoder-decoder models, denoted as ensemble in tables 3 , 5 , 4 and 6. a single style-aware encoder. as an alternative to training n models, we can train a single model and contextualize style by concatenating @xmath1. training on the reconstruction task this way (e.g. "the food was good" from "the food @xmath2. <@xmath3. this allows us to perform style transfer by simply manipulating the target style token. the model estimates the unseen portion of the hypothetical parallel dataset. furthermore, we hypothesize that a word-by-word generational approach using an autoregressive decoder might degrade performance in content preservation criteria. we juxtapose the performances of these approaches, denoted as generational and encoder-only in tables 3 , 5 , 4 and 6. to summarize, a simple modification (in the form of label concatenation) to the traditional mlm task enables style transfer by training solely on a reconstruction/denoising task. this forms the basis of our approach. ## masking style with attention we explore the question, "what constitutes a good style-masking step?". intuitively, we can reason that our style-masking approach must a) produce accurate attribution scores for each token and b) use an appropriate masking policy that determines which tokens to mask using these attribution scores. the final style-masked sequences (input to the sa-mlm) must be a) completely devoid of style information and b) accurate, i.e. not done at the expense of content information. ## related work recent work focuses on various common paradigms such as disentanglement @xcite , cycle-consistency losses @xcite @xcite , induction @xcite . @xcite and @xcite provide surveys detailing the current state of style transfer and lay down useful taxonomies to structure the field. in this section, we only discuss contemporary work similar to ours (prototype editing approaches) assuming the same unsupervised setting. @xcite present the earliest known work using the prototype editing method, in which a "delete" operation is performed on tokens based on simple count-based methods, and the retrieval of the target word is done by considering tf-idf weighted word overlap. malmi (2020) first train mlms for the source and target domains and perform style transfer by first masking text spans where the models disagree (in terms of perplexity) the most, and use the target domain mlm to fill these spans. @xcite introduce the attribute-conditional mlm, which most closely aligns to the working of the sa-mlm, also uses an attention classifier for attribution scores, a count and frequency-based method to perform masking, and a pretrained bert model fine-tuned on the style transfer task. @xcite and @xcite also follow roughly the same pipeline but uses a generational transformer encoder-decoder approach and also fine-tunes using signals from a pretrained classifier. @xcite uses a hierarchical reinforced sequence operation method is used to iteratively revise the words of original sentences. @xcite uses n-gram tf-idf based methods to identify style tokens and modify them as either "add" or "replace" tag tokens, which are then substituted by the decoder to perform style transfer. similar to the sa-mlm, @xcite also uses attribution-based methods from a self-attention classifier. however, they use an lstm @xcite based approach, one to generate sentences from each domain. @xcite performs unsupervised synthesis to create a pseudo-parallel dataset and uses multi-span editing techniques to fill in the style using a fine-tuned pretrained language model. ## methodology in this section, we describe the working of the sa-mlm during training and inference using a formal probabilistic framework. ## analysis of style-masking approach in this section, we evaluate and justify our choice of style-masking architecture, i.e. "explainable attention" + "attention-surplus" masking policy. ## experiments we perform style transfer on all the datasets and analyse the results. henceforth, we refer to the encoder-only variant (described in section 5.3) as our "sa-mlm" flagship model while comparing with baselines in tables 3 , 5 , 4 and 6. the other two sa-mlm variants i.e. ensemble and generational described in section 2, serve as ablation studies to support our hypothesis that a single nongenerational self-attention encoder only approach is optimal for style transfer. ## conclusion we introduce the sa-mlm, a modification of the standard mlm, which we show is capable of performing tst by using a style-masked input and performing a simple same-style reconstruction task with a lightweight transformer encoder block. on fine-tuning the sa-mlm for the tst%, it is on par with state-of-the-art models with orders of more parameters and sophisticated architectures in the sentiment tst task. we show that complex styles such as flow of logic/ discourse can be manipulated even with using this simple style masking assumption. we empirically show that the sa-mlm performs well in this discourses manipulation task and outperforms another strong baseline in this task, also seen through human evaluations.