SlowGuess/ACL_2020_OCR huggingface dataset:
The training data (Fig. 4) for the add-tags is given by pairs where the input is  $\{\mathbf{x}_i^{(2)}\backslash a(\mathbf{x}_i^{(2)}):$ $\mathbf{x}_i^{(2)}\in \mathbf{X}_2\}$  and the output is  $\{z(\mathbf{x}_i):\mathbf{x}_i^{(2)}\in \mathbf{X}_2\}$ . Essentially, for the input we take samples  $\mathbf{x}_i^{(2)}$  in the target style  $S_{2}$  and explicitly remove style phrases  $a(\mathbf{x}_i^{(2)})$  from it. For the output we replace the same phrases  $a(\mathbf{x}_i^{(2)})$  with [TAG] tokens. As indicated in Fig. 4, we remove the style phrases "you would like to" and "please" and replace them with [TAG] in the output. Note that we only use samples from  $\mathbf{X}_2$  for training the add-tags; samples from the style neutral  $\mathbf{X}_1$  are not involved in the training process at all. For example, in the case of politeness transfer, we only use the sentences labeled as "polite" for training. In effect, by training in this fashion, the tagger learns to add [TAG] tokens at appropriate locations in a style neutral sentence. The loss objective  $(\mathcal{L}_a)$  given by Eq. 4 is crucial for tasks like politeness transfer where one of the styles is poorly defined.
$$
\mathcal {L} _ {a} (\theta_ {t}) = - \sum_ {i = 1} ^ {| \mathbf {X} _ {1} |} \log P _ {\theta_ {t}} (z (\mathbf {x} _ {i}) | \mathbf {x} _ {i} ^ {(2)} \backslash a (\mathbf {x} _ {i} ^ {(2)}); \theta_ {t}) (4)
$$
4.3 Style Targeted Generation
The training for the generator model is complementary to that of the tagger, in the sense that the generator takes as input the tagged output  $z(\mathbf{x}_i)$  inferred from the source style and modifies the [TAG] tokens to generate the desired sentence  $\hat{\mathbf{x}}_i^{(v)}$  in the target style  $S_v$ .
$$
\mathcal {L} \left(\theta_ {g}\right) = - \sum_ {i = 1} ^ {| \mathbf {X} _ {v} |} \log P _ {\theta_ {g}} \left(\mathbf {x} _ {i} ^ {(v)} \mid z \left(\mathbf {x} _ {i}\right); \theta_ {g}\right) \tag {5}
$$
The training data for transfer into style  $S_{v}$  comprises of pairs where the input is given by  $\{z(\mathbf{x}_i): x_i^{(v)} \in \mathbf{X}_v, v \in \{1,2\}\}$  and the output is  $\mathbf{X}_v$ , i.e. it is trained to transform a style agnostic representation into a style targeted sentence. Since the generator has no notion of the original style and it is only concerned with the style agnostic representation  $z(\mathbf{x}_i)$ , it is convenient to disentangle the training for tagger & generator.
Finally, we note that the location at which the tags are generated has a significant impact on the distribution over style attributes (in  $\Gamma_{2}$ ) that are used to fill the [TAG] token at a particular position. Hence, instead of using a single [TAG] token, we use a set of positional tokens  $[\mathrm{TAG}]_t$  where  $t\in \{0,1,\dots T\}$  for a sentence of length  $T$ . By training both tagger and generator with these positional  $[\mathrm{TAG}]_t$  tokens we enable them to easily realize different distributions of style attributes for different positions in a sentence. For example, in the case of politeness transfer, the tags added at the beginning ( $t = 0$ ) will almost always be used to generate a token like "Would it be possible ..." whereas for a higher  $t$ ,  $[\mathrm{TAG}]_t$  may be replaced with a token like "thanks" or "sorry."
Baselines We compare our systems against three previous methods. DRG (Li et al., 2018), Style Transfer Through Back-translation (BST) (Prabhumoye et al., 2018), and Style transfer from nonparallel text by cross alignment (Shen et al., 2017) (CAE). For DRG, we only compare against the best reported method, delete-retrieve-generate. For all the models, we follow the experimental setups described in their respective papers.
Implementation Details We use 4-layered transformers (Vaswani et al., 2017) to train both tagger and generator modules. Each transformer has 4 attention heads with a 512 dimensional embedding layer and hidden state size. Dropout (Srivastava et al., 2014) with p-value 0.3 is added for each layer in the transformer. For the politeness dataset the generator module is trained with data augmentation techniques like random word shuffle, word drops/replacements as proposed by (Im
Politeness	Gender	Political
Acc	BL-s	MET	ROU	Acc	BL-s	MET	ROU	ACC	BL-s	MET	ROU
CAE	99.62	6.94	10.73	25.71	65.21	9.25	14.72	42.42	77.71	3.17	7.79	27.17
BST	60.75	2.55	9.19	18.99	54.4	20.73	22.57	55.55	88.49	10.71	16.26	41.02
DRG	90.25	11.83	18.07	41.09	36.29	22.9	22.84	53.30	69.79	25.69	21.6	51.8
OURS	89.50	70.44	36.26	70.99	82.21	52.76	37.42	74.59	87.74	68.44	45.44	77.51
Table 1: Results on the Politeness, Gender and Political datasets.
et al., 2017). We empirically observed that these techniques provide an improvement in the fluency and diversity of the generations. Both modules were also trained with the BPE tokenization (Sennrich et al., 2015) using a vocabulary of size 16000 for all the datasets except for Captions, which was trained using 4000 BPE tokens. The value of the smoothing parameter  $\gamma$  in Eq. 2 is set to 0.75. For all datasets except Yelp we use phrases with  $p_1^2(w) \geq k = 0.9$  to construct  $\Gamma_2$ ,  $\Gamma_1$  ( $\S 4.1$ ). For Yelp  $k$  is set to 0.97. During inference we use beam search (beam size=5) to decode tagged sentences and targeted generations for tagger & generator respectively. For the tagger, we re-rank the final beam search outputs based on the number of [TAG] tokens in the output sequence (favoring more [TAG] tokens).
Automated Evaluation Following prior work (Li et al., 2018; Shen et al., 2017), we use automatic metrics for evaluation of the models along two major dimensions: (1) style transfer accuracy and (2) content preservation. To capture accuracy, we use a classifier trained on the nonparallel style corpora for the respective datasets (barring politeness). The architecture of the classifier is based on AWD-LSTM (Merit et al., 2017) and a softmax layer trained via cross-entropy loss. We use the implementation provided by fastai. For politeness, we use the classifier trained by (Niu and Bansal, 2018). The metric of transfer accuracy (Acc) is defined as the percentage of generated sentences classified to be in the target domain by the classifier. The standard metric for measuring content preservation is BLEU-self (BL-s) (Papineni et al., 2002) which is computed with respect to the original sentences. Additionally, we report the BLEU-reference (BL-r) scores using the human reference sentences on the Yelp, Amazon and Captions datasets (Li et al., 2018). We also report ROUGE (ROU) (Lin, 2004) and METEOR (MET) (Denkowski and Lavie,
2011) scores. In particular, METEOR also uses synonyms and stemmed forms of the words in candidate and reference sentences, and thus may be better at quantifying semantic similarities.
Table 1 shows that our model achieves significantly higher scores on BLEU, ROUGE and METEOR as compared to the baselines DRG, CAE and BST on the Politeness, Gender and Political datasets. The BLEU score on the Politeness task is greater by 58.61 points with respect to DRG. In general, CAE and BST achieve high classifier accuracies but they fail to retain the original content. The classifier accuracy on the generations of our model are comparable (within  $1\%$  ) with that of DRG for the Politeness dataset.
In Table 2, we compare our model against CAE and DRG on the Yelp, Amazon, and Captions datasets. For each of the datasets our test set comprises 500 samples (with human references) curated by Li et al. (2018). We observe an increase in the BLEU-reference scores by 5.25, 4.95 and 3.64 on the Yelp, Amazon, and Captions test sets respectively. Additionally, we improve the transfer accuracy for Amazon by  $14.2\%$  while achieving accuracies similar to DRG on Yelp and Captions. As noted by Li et al. (2018), one of the unique aspects of the Amazon dataset is the absence of similar content in both the sentiment polarities. Hence, the performance of their model is worse in this case. Since we don't make any such assumptions, we perform significantly better on this dataset.
While popular, the metrics of transfer accuracy and BLEU have significant shortcomings making them susceptible to simple adversaries. BLEU relies heavily on n-gram overlap and classifiers can be fooled by certain polarizing keywords. We test this hypothesis on the sentiment transfer task by a Naive Baseline. This baseline adds "but overall it sucked" at the end of the sentence to transfer it to negative sentiment. Similarly, it appends "but overall it was perfect" for transfer into a positive sentiment. This baseline achieves an average accuracy score of  $91.3\%$  and a BLEU score of 61.44 on the Yelp
<sup>5</sup>https://docs.fast.ai/
This is trained on the dataset given by (Danescuulescu-Mizil et al., 2013).
Yelp	Amazon	Captions
Acc	BL-s	BL-r	MET	ROU	Acc	BL-s	BL-r	MET	ROU	Acc	BL-s	BL-r	MET	ROU
CAE	72.1	19.95	7.75	21.70	55.9	78	2.64	1.68	9.52	29.16	89.66	2.09	1.57	9.61	30.02
DRG	88.8	36.69	14.51	32.09	61.06	52.2	57.07	29.85	50.16	79.31	95.65	31.79	11.78	32.45	64.32
OURS	86.6	47.14	19.76	36.26	70.99	66.4	68.74	34.80	45.3	83.45	93.17	51.01	15.63	43.67	79.51
Table 2: Results on the Yelp, Amazon and Captions datasets.
Con	Att	Gra
DRG	Ours	DRG	Ours	DRG	Ours
Politeness	2.9	3.6	3.2	3.6	2.0	3.7
Gender	3.0	3.5	-	-	2.2	2.5
Political	2.9	3.2	-	-	2.5	2.7
Yelp	3.0	3.7	3	3.9	2.7	3.3
Table 3: Human evaluation on Politeness, Gender, Political and Yelp datasets.
dataset. Despite high evaluation scores, it does not reflect a high rate of success on the task. In summary, evaluation via automatic metrics might not truly correlate with task success.
Changing Content Words Given that our model is explicitly trained to generate new content only in place of the TAG token, it is expected that a well-trained system will retain most of the non-tagged (content) words. Clearly, replacing content words is not desired since it may drastically change the meaning. In order to quantify this, we calculate the fraction of non-tagged words being changed across the datasets. We found that the non-tagged words were changed for only  $6.9\%$  of the sentences. In some of these cases, we noticed that changing non-tagged words helped in producing outputs that were more natural and fluent.
Human Evaluation Following Li et al. (2018), we select 10 unbiased human judges to rate the output of our model and DRG on three aspects: (1) content preservation (Con) (2) grammaticality of the generated content (Gra) (3) target attribute match of the generations (Att). For each of these metrics, the reviewers give a score between 1-5 to each of the outputs, where 1 reflects a poor performance on the task and 5 means a perfect output. Since the judgement of signals that indicate gender and political inclination are prone to personal biases, we don't annotate these tasks for target attribute match metric. Instead we rely on the classifier scores for the transfer. We've used the same instructions from Li et al. (2018) for our human study. Overall, we evaluate both systems on a total of 200 samples for Politeness and 100 samples each for Yelp, Gender and Political.
Table 3 shows the results of human evaluations.
We observe a significant improvement in content preservation scores across various datasets (specifically in Politeness domain) highlighting the ability of our model to retain content better than DRG. Alongside, we also observe consistent improvements of our model on target attribute matching and grammatical correctness.
Qualitative Analysis We compare the results of our model with the DRG model qualitatively as shown in Table 4. Our analysis is based on the linguistic strategies for politeness as described in (Danescu-Niculescu-Mizil et al., 2013). The first sentence presents a simple example of the counterfactual modal strategy inducing "Could you please" to make the sentence polite. The second sentence highlights another subtle concept of politeness of 1st Person Plural where adding "we" helps being indirect and creates the sense that the burden of the request is shared between speaker and addressee. The third sentence highlights the ability of the model to add Apologizing words like "Sorry" which helps in deflecting the social threat of the request by attuning to the imposition. According to the Please Start strategy, it is more direct and insincere to start a sentence with "Please". The fourth sentence projects the case where our model uses "thanks" at the end to express gratitude and in turn, makes the sentence more polite. Our model follows the strategies prescribed in (Danescu-Niculescu-Mizil et al., 2013) while generating polite sentences.<sup>7</sup>
Ablations We provide a comparison of the two variants of the tagger, namely the replace-tags and add-tags on two datasets. We also train and compare them with a combined variant. We train these tagger variants on the Yelp and Captions datasets and present the results in Table 5. We observe that for Captions, where we transfer a factual (neutral) to romantic/humorous sentence, the add
We provide additional qualitative examples for other tasks in the supplementary material.
${}^{8}$  Training of combined variant is done by training the tagger model on the concatenation of training data for add-tagger and replace-tags.
Input	DRG Output	Our Model Output	Strategy
what happened to my personal station?	what happened to my mother to my co???	could you please let me know what happened to my personal station?	Counterfactual Modal
yes, go ahead and remove it.	yes, please go to the link below and delete it.	yes, we can go ahead and remove it.	1st Person Plu-ral
not yet-i'll try this wkend.	not yet to say-i think this will be a <unk> long.	sorry not yet-i'll try to make sure this wk	Apologizing
please check on metromedia energy,	thanks again on the energy in- dustry,	please check on metromedia en- ergy, thanks	Mitigating please start
tagger provides the best accuracy with a relatively negligible drop in BLEU scores. On the contrary, for Yelp, where both polarities are clearly defined, the replace-tags gives the best performance. Interestingly, the accuracy of the add-tags is  $\approx 50\%$  in the case of Yelp, since adding negative words to a positive sentence or vice-versa neutralizes the classifier scores. Thus, we can use the add-tags variant for transfer from a polarized class to a neutral class as well.
To check if the combined tagger is learning to perform the operation that is more suitable for a dataset, we calculate the fraction of times the combined tagger performs add/replace operations on the Yelp and Captions datasets. We find that for Yelp (a polar dataset) the combined tagger performs  $20\%$  more replace operations (as compared to add operations). In contrast, on the CAPTIONS dataset, it performs  $50\%$  more add operations. While the combined tagger learns to use the optimal tagging operation to some extent, a deeper understanding of this phenomenon is an interesting future topic for research. We conclude that the choice of the tagger variant is dependent on the characteristics of the underlying transfer task.
Table 4: Qualitative Examples comparing the outputs from DRG and Our model for the Politeness Transfer Task
Yelp	Captions
Acc	BL-r	Acc	BL-r
Add-Tagger	53.2	20.66	93.17	15.63
Replace-Tagger	86.6	19.76	84.5	15.04
Combined	72.5	22.46	82.17	18.51
Table 5: Comparison of different tagger variants for Yelp and Captions datasets
We introduce the task of politeness transfer for which we provide a dataset comprised of sentences curated from email exchanges present in the Enron corpus. We extend prior works (Li et al., 2018; Sudhakar et al., 2019) on attribute transfer by introducing a simple pipeline - tag & generate which is an interpretable two-staged approach for content preserving style transfer. We believe our approach is the first to be robust in cases when the source is style neutral, like the "non-polite" class in the case of politeness transfer. Automatic and human evaluation shows that our approach outperforms other state-of-the-art models on content preservation metrics while retaining (or in some cases improving) the transfer accuracies.
This material is based on research sponsored in part by the Air Force Research Laboratory under agreement number FA8750-19-2-0200. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the Air Force Research Laboratory or the U.S. Government. This work was also supported in part by ONR Grant N000141812861, NSF IIS1763562, and Apple. We would also like to acknowledge NVIDIA's GPU support. We would like to thank Antonis Anastasopoulos, Ritam Dutt, Sopan Khosla, and, Xinyi Wang for the helpful discussions.
Philip Bramsen, Martha Escobar-Molano, Ami Patel, and Rafael Alonso. 2011. Extracting social power relationships from natural language. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1, HLT '11, pages 773-782, Stroudsburg, PA, USA. Association for Computational Linguistics.
Penelope Brown, Stephen C Levinson, and Stephen C Levinson. 1987. Politeness: Some universals in language usage, volume 4. Cambridge university press.
Khyathi Chandu, Shrimai Prabhumoye, Ruslan Salakhutdinov, and Alan W Black. 2019. my way of telling a story: Persona based grounded story generation. In Proceedings of the Second Workshop on Storytelling, pages 11-21.
Liz Coppock. 2005. Politeness strategies in conversation closings. unpublished paper available online at http://www.stanford.edu/~coppockface.pdf(last accessed 23 December 2007).
Cristian Danescu-Niculescu-Mizil, Moritz Sudhof, Dan Jurafsky, Jure Leskovec, and Christopher Potts. 2013. A computational approach to politeness with application to social factors. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 250-259, Sofia, Bulgaria. Association for Computational Linguistics.
Michael Denkowski and Alon Lavie. 2011. Meteor 1.3: Automatic metric for reliable optimization and evaluation of machine translation systems. In Proceedings of the sixth workshop on statistical machine translation, pages 85-91. Association for Computational Linguistics.
Zhenxin Fu, Xiaoye Tan, Nanyun Peng, Dongyan Zhao, and Rui Yan. 2018. Style transfer in text: Exploration and evaluation. In Thirty-Second AAAI Conference on Artificial Intelligence.
Chuang Gan, Zhe Gan, Xiaodong He, Jianfeng Gao, and Li Deng. 2017. Stylenet: Generating attractive visual captions with styles. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3137-3146.
Ruining He and Julian McAuley. 2016. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In *proceedings of the 25th international conference on world wide web*, pages 507-517. International World Wide Web Conferences Steering Committee.
Matthew Honnibal and Ines Montani. 2017. spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing. To appear.
Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P. Xing. 2017. Toward controlled generation of text. In Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 1587-1596, International Convention Centre, Sydney, Australia. PMLR.
Daniel Im Jiwoong Im, Sungjin Ahn, Roland Memisevic, and Yoshua Bengio. 2017. Denoising criterion for variational auto-encoding framework. In Thirty-First AAAI Conference on Artificial Intelligence.
Harsh Jhamtani, Varun Gangal, Eduard Hovy, and Eric Nyberg. 2017. Shakespearizing modern language using copy-enriched sequence to sequence models. In Proceedings of the Workshop on Stylistic Variation, pages 10-19, Copenhagen, Denmark. Association for Computational Linguistics.
Vineet John, Lili Mou, Hareesh Bahuleyan, and Olga Vechtomova. 2019. Disentangled representation learning for non-parallel text style transfer. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 424-434, Florence, Italy. Association for Computational Linguistics.
Daniel Jurafsky, Elizabeth Shriberg, and Debra Bíasca. 1997. Switchboard SWBD-DAMSL shallow-discourse-function annotation coders manual, draft 13. Technical Report 97-02, University of Colorado, Boulder Institute of Cognitive Science, Boulder, CO.
Daniel Z Kádár and Sara Mills. 2011. Politeness in East Asia. Cambridge University Press.
Dongyeop Kang and Eduard Hovy. 2019. xslue: A benchmark and analysis platform for cross-style language understanding and evaluation.
Bryan Klimt and Yiming Yang. 2004. Introducing the enron corpus. In CEAS.
Juncen Li, Robin Jia, He He, and Percy Liang. 2018. Delete, retrieve, generate: a simple approach to sentiment and style transfer. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1865-1874, New Orleans, Louisiana. Association for Computational Linguistics.
Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74-81.
Andrew McCallum, Xuerui Wang, and Andrés Corrada-Emmanuel. 2007. Topic and role discovery in social networks with experiments on enron and academic email. J. Artif. Int. Res., 30(1):249-272.
Ardith J Meier. 1995. Defining politeness: Universality in appropriateness. Language Sciences, 17(4):345-356.