SlowGuess/NAACL_2021_OCR huggingface dataset:
To evaluate style variation across language models fine-tuned on different styles, we compare the generations of the fine-tuned models. For single-dimensional style evaluation, we generate sentences from models fine-tuned on negative corpus and positive corpus and compare the style accuracy of generated sentences. The style accuracy is evaluated by employing a FastText (Joulin et al., 2016) classifier trained on the corresponding style dimension. For instance, the classifier for evaluating sentiment accuracy is trained on sentiment corpus tagged with positive and negative class in IMDB and Yelp data. Table 1 shows the accuracy of sentences generated by a model fine-tuned on style  $s_i$  as belonging to the class  $s_i$ . For both sentiment and formality, the fine-tuned language models are able to generate text faithful to the target style dimension. Thus, we conclude that the language models trained on style  $s_i$  are able to capture the essence of the corresponding style reasonably well.
These accuracies are an indication of the style awareness in these fine-tuned LMs. We, therefore, employ the perplexities of these fine-tuned language models to gauge the style of the input text
to guide our style transfer model. As discussed in discriminative modeling (Section 3.2), the model fine-tuned with corpus from a certain style is expected to have high perplexity on sentence not from that style and low perplexity otherwise. To this end, we experiment with two models independently fine-tuned on positive and negative corpus. We calculate the perplexity of each of these models on the test corpus from the same style and from the opposite style. As seen in Table 2, the perplexity for each model is substantially lower on the same corpus as compared to that on the opposite corpus. This implies that a language model fine-tuned on positive corpus shows higher perplexity for negative sentences and lower for positive sentences and vice versa. This corroborates the effectiveness of these fine-tuned language models to serve as discriminators for training the style transfer module.
We measure the performance of our model and the baselines based on the style control, content preservation and fluency. To measure the accuracy of style transfer, we train two Fasttext $^2$  classifiers independently for sentiment and formality using the train corpus, as described in Section 4.1. These classifiers have accuracy of  $93.74\%$  and  $88.95\%$  respectively on test corpus of respective datasets. We note that formality as a style is more intricately designed, so we also check lexical scoring by Brooke et al. (2010) to evaluate formality, which uses a formality lexicon to assign formality score between  $-1$  (informal) and 1 (formal) to each word and averages it. We scale these scores between 0-100, where higher (100) lexical score signifies formal style and lower (0) score signifies informal style. For informal target style, we report lexical score as  $100 - n$ , so that a higher average lexical score signifies a better transfer for either polarity.
To measure content preservation on transfer, we calculate the BLEU score (Papineni et al., 2002) between the transferred sentence and the input sentence (self-BLEU). Besides this, we also calculate BLEU score between the transferred sentence generated by our model and the corresponding human reference transferred sentence, available for GYAFC and Yelp corpus (ref-BLEU). Since both these corpus account for transfer across only one style dimension each, the provided references are only partial indication of expected outcome. This
$^{2}$ https://github.com/facebookresearch/fastText
Model	Style Accuracy	Content Preservation	Fluency
Classifier ↑ Sentiment	Formality	Lexical Scoring ↑ Formality	BLEU ↑ -self	-ref	Perplexity ↓
Cascaded Style Transformer (Dai et al., 2019)	72.17	64.08	81.29	0.6066	0.3479	8.8657
Adapted Rewriting LM (Syed et al., 2020)	52.59	36.39	72.21	0.7917	0.4259	6.5963
Cascaded Discriminative LM	69.30	48.18	83.02	0.6634	0.3579	6.6846
Joint Discriminative LM	79.78	65.33	85.39	0.7710	0.4136	6.4574
Table 3: Quantitative Comparison of our proposed approach (Joint Discriminative LM) against Cascaded Style Transformer (Dai et al., 2019), Cascaded Discriminative LM method and multi-style transfer using Adapted Rewriting LM (Syed et al., 2020). The upward arrow signifies that higher is better and vice versa. Score of near 100 on formality lexical scoring imply the transferred text is close in formality to the target corpus.
is also apparent from low ref-BLEU scores for our model as well as baselines. Since, the results are presented on aggregated dataset from both these style dimensions, this evaluation is still able to provide reasonable indication of content preservation.
To measure the fluency of the text, we calculate perplexity assigned to the generated text sequence by a language model trained on the train corpus, as is standard in style transfer literature (Dai et al., 2019; Subramanian et al., 2018). The perplexity is the measure of log likelihood of the generated sentence on the language model. A lower perplexity is indicative of a more fluent sentence. We use a generative transformer-based language model trained on the dataset combined from two styles.
Dai et al. (2019) use transformer-based model (Style Transformer) for single-dimensional style transfer. We train two independent Style Transformer models for sentiment and formality transfer and then perform transfer one after another to compare results with our model. We term this as Cascaded Style Transformer setup. The Style Transformer model is shown to have state-of-the-art performance in single-dimensional style transfer; thus it provides an estimate of the performance of sequential single style transfer. We also experiment with Adapted Rewriting LM (Syed et al., 2020) as another baseline. Their work on style rewriting to match author-specific style does not require explicit annotations for the various aspects that constitutes an author's style, but is based on the assumption that the training corpus reflects the target style. In this context, we train their framework on the mixture of data from the respective target styles and report the performance. These are the closest baselines to our proposed approach, since other works dealing with multi-style transfer assume presence
of jointly annotated dataset, which is a stronger assumption that we aim to relax. In addition to our proposed model with multiple style transfer, we also train our encoder-decoder architecture with single discriminative LM for one style at a time and perform two stage transfer, similar to one with Cascaded Style Transformer (Dai et al., 2019) setup.
The results in Table 3 show that our model achieves better style control than the Cascaded Style Transformer (Dai et al., 2019) as well as the joint transfer using Syed et al. (2020) for both sentiment and formality. As seen in Table 3, cascaded style transfer models perform poorly on content preservation. This is because transferring style one after other leads to huge loss in content, thus both the two-stage models score lower on content preservation metrics, both w.r.t. the input text and the reference transferred text. This demonstrates the advantage of using single model to control for multiple styles. The effect can also be observed in Table 4 which demonstrates qualitative results for Cascaded Style Transformer model and our model. We can see in many cases content loses the underlying meaning of source sentence during the two-stage transfer, whereas our model is able to retain original meaning of the sentence well, corroborating the findings of automatic evaluation. Among the cascaded models, the Discriminative LM scores marginally better on content preservation than the Style Transformer model. We attribute this to initialization with the same pre-trained LM resulting in shared content space in the underlying single style transfer models. However, due to independent training of the two single style transfer models, they are not able to model interplay between these styles and hence perform worse on style control than our proposed model trained jointly on multiple styles.
Our model also scores better on fluency, as seen in Table 3. This is also apparent from the exam
Target style	Source sentence	Transferred Sentence
Style Transformer	Our model (multi-style)
Positive+Formal	That's not funny. I don't think she'll like it.	So funny movie. I really like it.	That was very funny. I am sure she will appreciate it.
Give your brother some money and tell him to take a hike.	Just give your brother some time and it will be good again.	Give your brother some money and request him to leave.
Negative+Formal	An intelligent, rewarding film that I look forward to watching again.	ludricious, shallow film that look forward to watching again.	An unintelligent, poor film that I would not look forward to watching again.
super friendly staff, quick service and amazing and simple food was done right!	says wait staff, quick not amazing before overcooked food done were okay.	dirty staff and slow service and simple food was not done right.
Positive+Informal	You need to separate the bad thing and move on.	need to the great thing and move on.	You need to enjoy the good stuff and move on.
The evening started out slow.	The evening spent in professional show.	The evening began amazing.
Negative+Informal	Great food recommendations steak and tuna were both great.	terrible food 9am steak and were both terrible.	Disappointing food recommendations steak and tuna were horrible.
That person in hilarious.	You person in worse!	That guy in so boring.
Table 4: Qualitative results for transfer to different target style combination across different models. (Different colors highlight the transferred segments corresponding to underlined input sentence; Text in bold highlights adherence to target formality in text generated by our model.)
Model	Style Accuracy	Content Preservation	Fluency	Transfer Quality
Sentiment	Formality
Cascaded Style Transformer (Dai et al., 2019)	3.5909	2.7424	3.2803	2.7424	2.9318
Joint Discriminative LM (Our Model)	3.8561	3.0379	4.1061	4.1894	4.1091
Table 5: Results for Human Evaluation across different metrics. Each value represents the average of rating between 1 (Very bad) and 5 (Very good).
plies in Table 4, where sentences generated by Cascaded Style Transformer are much less coherent. Qualitative experiments also highlight the ability of our model to incorporate intricacies of formality stylistic dimension (shown in bold) better than the Cascaded Style Transformer model. Among single step transfer models (Syed et al. (2020) and our proposed approach), we note that content preservation is marginally better for Syed et al. (2020)'s model, however, our model is able to yield much better style transfer owing to feedback on style control by multiple discriminators.
To augment automatic evaluation results, we conduct a human study to evaluate the model outputs across various dimensions such as content preservation, style control, fluency, and overall trans
fer quality. Based on comparable style control in Cascaded Style Transformer and our proposed approach on automatic metrics, we compare the transfer quality across these two models by a small-scale human study. We select 40 sentences, with 10 examples from each combinations of sentiment and formality as target style, and collect annotations from 4-5 participants for each example. Out of resulting annotations, more than  $85\%$  annotations favoured our results over baseline. The average participant rating across different dimensions is shown in Table 5. We test the statistical significance of these results using z-test statistic. With  $\alpha = 0.05$ , the preferences indicated in human study are significant across all metrics. These results are in line with our automatic evaluations and add confidence to the efficacy of our proposed approach in achieving style transfer across multiple dimensions.
We propose an approach to extend currently existing style transfer work to multiple style setting without imposing any extra constraints on availability of dataset. Our method makes use of disjoint corpus from separate styles to enable one step transfer across multiple target styles. We exploit multiple discriminative language models with an encoder-decoder framework, all emerging from large transformer-based language models pretrained on Masked Language Modeling objective and fine-tuned separately for transfer and discriminative purposes. We show that unified single step transfer approach is able to achieve better transfer while offering much better content preservation which is paramount to any style transfer task.
Further improvements are in scope for adding modularity to the proposed transfer module. In the current setup, each version of model is trained for a specific combination of target style(s). The utility of such a model increases manifold with added ease of transfer across multiple style combinations within a single model. This could be attempted by employing a controlled language model as a unified discriminator for multiple styles, which would be the subject of further research.
Ethics Statement. We recognise the ethical implication of employing large language models trained on data infused with unchecked biases. As with any generative task, style transfer too suffers from the potential misuse for fact distortion, plagiarism and more. The paper aims at establishing academic utility of proposed framework. To meet ethical standards, this solution has to coupled with strict misrepresentation, offensiveness and bias checks.
Julian Brooke, Tong Wang, and Graeme Hirst. 2010. Automatic acquisition of lexical formality. In *Coling 2010: Posters*, pages 90–98, Beijing, China. Coling 2010 Organizing Committee.
Stephane Clinchant, Kweon Woo Jung, and Vassilina Nikoulina. 2019. On the use of BERT for neural machine translation. In Proceedings of the 3rd Workshop on Neural Generation and Translation, pages 108-117, Hong Kong. Association for Computational Linguistics.
Alexis Conneau and Guillaume Lample. 2019. Crosslingual language model pretraining. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc,
E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems 32, pages 7059-7069. Curran Associates, Inc.
Ning Dai, Jianze Liang, Xipeng Qiu, and Xuanjing Huang. 2019. Style transformer: Unpaired text style transfer without disentangled latent representation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5997-6007, Florence, Italy. Association for Computational Linguistics.
Cristian Danescu-Niculescu-Mizil, Moritz Sudhof, Dan Jurafsky, Jure Leskovec, and Christopher Potts. 2013. A computational approach to politeness with application to social factors. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 250-259, Sofia, Bulgaria. Association for Computational Linguistics.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Zhenxin Fu, Xiaoye Tan, Nanyun Peng, Dongyan Zhao, and Rui Yan. 2017. Style Transfer in Text: Exploration and Evaluation. arXiv e-prints, page arXiv:1711.06861.
Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P. Xing. 2017. Toward controlled generation of text. In Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 1587-1596, International Convention Centre, Sydney, Australia. PMLR.
Armand Joulin, Edouard Grave, Piotr Bojanowski, Matthijs Douze, Herve Jégou, and Tomas Mikolov. 2016. Fasttext.zip: Compressing text classification models. arXiv preprint arXiv:1612.03651.
Guillaume Lample, Myle Ott, Alexis Conneau, Ludovic Denoyer, and Marc'Aurelio Ranzato. 2018. Phrase-based & neural unsupervised machine translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 5039-5049, Brussels, Belgium. Association for Computational Linguistics.
Juncen Li, Robin Jia, He He, and Percy Liang. 2018. Delete, retrieve, generate: a simple approach to sentiment and style transfer. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1865-1874, New Orleans, Louisiana. Association for Computational Linguistics.
Yang Liu and Mirella Lapata. 2019. Text summarization with pretrained encoders. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3730-3740, Hong Kong, China. Association for Computational Linguistics.
Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 142-150, Portland, Oregon, USA. Association for Computational Linguistics.
Aman Madaan, Amrith Setlur, Tanmay Parekh, Barnabas Poczos, Graham Neubig, Yiming Yang, Ruslan Salakhutdinov, Alan W Black, and Shrimai Prabhumoye. 2020. Politeness Transfer: A Tag and Generate Approach. arXiv e-prints, page arXiv:2004.14257.
François Mairesse and Marilyn A. Walker. 2011. Controlling user perceptions of linguistic style: Trainable generation of personality traits. Computational Linguistics, 37(3):455-488.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311-318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics.
Alec Radford. 2018. Improving language understanding by generative pre-training.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2018. Language models are unsupervised multitask learners.
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383-2392, Austin, Texas. Association for Computational Linguistics.
Sudha Rao and Joel Tetreault. 2018. Dear sir or madam, may I introduce the GYAFC dataset: Corpus, benchmarks and metrics for formality style transfer. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 129-140, New Orleans, Louisiana. Association for Computational Linguistics.
Abhilasha Sancheti, Kundan Krishna, Balaji Vasan Srinivasan, and Anandhavelu Natarajan. 2020. Reinforced rewards framework for text style transfer. In Advances in Information Retrieval, pages 545-560, Cham. Springer International Publishing.
Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715-1725, Berlin, Germany. Association for Computational Linguistics.
Tianxiao Shen, Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2017. Style transfer from non-parallel text by cross-alignment. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30, pages 6830-6841. Curran Associates, Inc.
Sandeep Subramanian, Guillaume Lample, Eric Michael Smith, Ludovic Denoyer, Marc'Aurelio Ranzato, and Y-Lan Boureau. 2018. Multiple-attribute text style transfer. CoRR, abs/1811.00552.
Yu Sun, Shuohuan Wang, Yukun Li, Shikun Feng, Hao Tian, Hua Wu, and Haifeng Wang. 2020. Ernie 2.0: A continual pre-training framework for language understanding. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05):8968-8975.
Richard S. Sutton, David McAllester, Satinder Singh, and Yishay Mansour. 1999. Policy gradient methods for reinforcement learning with function approximation. In Proceedings of the 12th International Conference on Neural Information Processing Systems, NIPS'99, page 1057-1063, Cambridge, MA, USA. MIT Press.
Bakhtiyar Syed, Gaurav Verma, Balaji Vasan Srinivasan, Anandhavelu Natarajan, and Vasudeva Varma. 2020. Adapting language models for non-parallel author-stylized rewriting. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05):9008-9015.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NIPS, pages 6000-6010.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 353-355, Brussels, Belgium. Association for Computational Linguistics.
Hu Xu, Bing Liu, Lei Shu, and Philip Yu. 2019. BERT post-training for review reading comprehension and aspect-based sentiment analysis. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2324-2335, Minneapolis, Minnesota. Association for Computational Linguistics.
Zichao Yang, Zhiting Hu, Chris Dyer, Eric P Xing, and Taylor Berg-Kirkpatrick. 2018. Unsupervised text style transfer using language models as discriminators. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems 31, pages 7287-7298. Curran Associates, Inc.