SlowGuess/NAACL_2022_OCR huggingface dataset:
Domain	Model	BLEURT	BLEU	ACC	HM	Model	BLEURT	BLEU	ACC	HM
E&M	(A) INFORMAL ↔ FORMAL	(B) INFORMAL → FORMAL
NMT-Combined	-0.100	0.501	0.797	0.615	GPT-CAT (train on E&M and F&R)	0.176	0.725	0.876	0.793
BART + SC & BLEU	0.044	0.577	0.859	0.690	Chawla's	0.260	0.762	0.910	0.829
GPT-2 + SC & BLEU	-0.007	0.542	0.923	0.683	BART large + SC & BLEU	0.274	0.765	0.929	0.839
OT4 (Ours)	0.102	0.602	0.949	0.736	OT4 (Ours)	0.322	0.812	0.951	0.876
(C) INFORMAL ↔ FORMAL & COMBINED DOMAINS	(D) BLEU EVALUATED AGAINST THE FIRST REFERENCE
Bi-directional FT	0.023	0.554	0.818	0.661	TS→CP	-	0.292	-	-
BART large + SC BLEU	0.078	0.596	0.905	0.719	BART + SC & BLEU	-	0.306	-	-
OT4 (Ours)	0.192	0.671	0.939	0.782	OT4 (Ours)	-	0.352	-	-
F&R	(A) INFORMAL ↔ FORMAL	(B) INFORMAL → FORMAL
NMT-Combined	-0.089	0.527	0.798	0.635	GPT-CAT (train on E&M and F&R)	-	0.769	-	-
BART + SC & BLEU	0.068	0.595	0.882	0.711	Chawla's	0.302	0.799	0.910	0.851
GPT-2 + SC & BLEU	0.038	0.572	0.915	0.704	BART large + SC & BLEU	0.324	0.793	0.920	0.852
OT4 (Ours)	0.112	0.618	0.942	0.746	OT4 (Ours)	0.401	0.825	0.961	0.887
(C) INFORMAL ↔ FORMAL & COMBINED DOMAINS	(D) 10% PARALLEL TRAINING DATA
Bi-directional FT	0.037	0.568	0.839	0.677	CPLS	-	0.379	-	-
BART large + SC & BLEU	0.100	0.611	0.900	0.728	BART + SC & BLEU	-	0.571	-	-
OT4 (Ours)	0.185	0.652	0.933	0.767	OT4 (Ours)	-	0.644	-	-
Table 3: Automatic evaluation result on GYAFC dataset. The performance of the baselines are taken from (Lai et al., 2021).
Model	S-ACC	ref-BLEU	self-BLEU	PPL	G-score	BERT-P	BERT-R	BERT-F1
Cross-Alignment	74.2	4.2	13.2	53.1	32.0	87.8	86.2	87.0
ControlledGen	83.7	16.1	50.5	146.3	65.0	90.6	89.0	89.8
Style Transformer	87.3	19.8	55.2	73.8	69.4	91.6	89.9	90.7
Deep Latent	85.2	15.1	40.7	36.7	58.9	89.8	88.6	89.2
RACoLN	91.3	20.0	59.4	60.1	73.6	91.8	90.3	91.0
OT4	93.4	26.7	71.2	42.1	81.4	97.7	96.7	97.2
Table 4: Automatic evaluation result on Yelp dataset. G-Score is the geometric mean of self-BLEU and S-ACC. The evaluation results of the baselines are taken from (Lee et al., 2021b)
S-ACC	self-BLEU	PPL	G-score
Cross-Alignment	63.9	1.1	29.9	8.4
ControlledGen	81.2	63.8	119.7	71.2
Style Transformer	74.0	70.4	71.2	72.2
Deep Latent	59.3	64.0	41.1	61.6
RACoLN	83.1	70.9	45.3	76.8
OT4	86.2	80.5	39.8	83.30
Table 5: Automatic evaluation result on IMDB dataset. Since human references are not available for IMDB dataset, self-BLEU and BERT scores are omitted. The evaluation results of the baselines are taken from (Lee et al., 2021b)
employs BLEU score to encourage content preservation. In contrast, we equip our GPT-2 model with OT-based reward that can incorporate both semantics and syntax of the sentences and achieve the best results; (2) Informal  $\rightarrow$  Formal: In this setting, only the conversion from informal to formal text is evaluated. compared to the previous setting, we see an improvement in the style conversion and content preservation for the equivalent models. It shows that this direction of conversion is relatively easier. However, the proposed OT4 model still significantly outperform the baselines in this setting too, indicating the importance of content preservation for this setting; (3) Informal  $\leftrightarrow$
Formal & Combined Domains: In this setting, the data from both domains are combined for training and the model is evaluated for conversion in both direction. The results show that in this setting, all baselines benefit from the extra training data from the other domain, however, the proposed OT4 enjoys the largest improvement. Our hypothesis for such improvement in OT4 is that the existence of the other domain data provides more syntactic structure to the model, therefore, compared to the baselines that miss this information, the proposed OT4 baseline can benefit from more training signals. (5) Evaluation with the first reference: To conduct a comprehensive comparison, we also compare our model with the baselines that only report the performance of the models evaluated on the first reference sentence. In this setting, we see that the proposed model achieves the highest BLEU score; and finally (6)  $10\%$  Parallel Data: To show the effectiveness of the proposed model in the case of low-resource setting, we compare the performance of the proposed model when only  $10\%$  of the training parallel data is employed. We see in this setting the improvement achieved by our proposed model is higher, especially in terms of BLEURT, reflecting its superiority to benefit more
Yelp	IMDB
Style	Content	Fluency	Style	Content	Fluency
Cross-Alignment	2.5	2.1	3.2	2.0	2.0	1.9
ControlledGen	3.1	3.5	3.4	3.0	3.4	3.2
Style Transformer	3.3	3.6	3.5	3.5	3.1	3.5
Deep Latent	3.5	2.9	4.1	2.6	3.1	3.1
RACoLN	3.6	3.7	3.8	3.3	3.5	3.7
OT4 (Ours)	4.1	4.6	4.2	3.9	4.6	4.0
Table 6: Human evaluation of the baseline outputs for Yelp and IMDB datasets. Numbers are the average score of the four annotators.
Model	Style	Content	Fluency
TS→CP	2.1	2.5	2.7
CPLS	3.2	2.6	3.0
NMT-Combined	3.0	2.6	2.2
Bi-directional FT	3.3	3.0	2.0
Chawla's	2.9	2.7	3.4
GPT-CAT	3.4	3.4	3.8
GPT-2 + SC & BLEU	3.1	3.4	3.1
BART + SC & BLEU	3.1	3.3	3.5
OT4 (Ours)	3.9	4.2	3.9
Table 7: Human evaluation of the baseline outputs for GYAFC dataset. Numbers are the average score of the four annotators.
efficiently from training signals.
Unsupervised: The results of the evaluation of the unsupervised model on the Yelp and IMDB datasets are presented in Table 4 and 5, respectively. Note that due to the lack of reference target sentences in the IMDB dataset, we omit self-BLEU and BERT scores in Table 5. There are several observations from these tables. First, the proposed OT4 model outperforms all baselines with respect to style conversion and content preservation. Specifically, for style conversion, our model improves the S-ACC on Yelp and IMDB by  $2.1\%$  and  $3.1\%$ , respectively. Compared the baselines, we attribute the style conversion improvement to the explicit rewards employed in our model to directly train the model for better style conversion. More importantly, since our model is equipped with OT to improve content preservation, we see a significant improvement for this metric. In particular, on the Yelp dataset, our model improves BERT-F1 score by  $6.2\%$  and self-BLEU by  $11.8\%$ . Considering the improvement on ref-BLEU on this dataset also indicates that while our model improves the content preservation, it is not repeating the input sentence. Finally, comparison of the fluency of the generated sentences shows that our model is competitive with baselines by achieving the second-lowest PPL on both datasets.
In order to shed more light on the contribution of the proposed OT-based content preservation reward, in this section we study the performance of alternative architecture designs: (1) No Semantics: Here, the cost function  $C(x,y)$  is replaced by the constant function  $C(x,y) = 1$ , hence removing all information regarding the semantics of the sentence; (2) No Syntax: In this baseline, the probability distribution  $p(x)$  and  $q(y)$  are represented by uniform distribution, thus removing all information about the syntactic structure; (3) NO OT: In this baseline, the content preservation reward is completely removed; (4) Reconstruct: Following prior work (Ma et al., 2019), instead of using OT-based reward, we add another auxiliary task in which the model is trained to reconstruct the POS tag of the input sentence. Note that, for this auxiliary task, the model is trained to re-convert the generated sentence  $D'$  back to  $D$ ; (5) Graph-based: Instead of directly encoding the interaction between the syntax and semantics via OT-based distance, in this baseline we encode the syntax and the semantics together using a Graph Convolution Network (GCN) (Kipf and Welling, 2017). Specifically, before generating the words of  $D'$ , the representations  $H$  obtained from the GPT-2 model are further abstracted using a two-layer GCN that takes the dependency tree of the input sentence as the input graph. We evaluate the models on the development set of E&M domain for formal and informal style transfer (i.e., both direction) $^3$ .
Table 9 shows the results. This table shows that removing both syntax and semantics scores from OT will hurt the performance. However, syntactic information has more importance as removing them results in more performance loss. Moreover, it is clear from this table that reconstructing syntactic features is not as effective as OT-based reward. This inferiority is better evident from the loss in BLEU and BLEURT scores. Our hypothesis for better performance of the OT-based reward is that OT can encode the interaction between the syntax and semantics while reconstruction makes these two tasks separate. Finally, this table shows that the graph-based model has poor performance compared to OT4. Our hypothesis for this observation is that while the GCN model can encode the syntactic structure of both sentences, it cannot encode the
Note that the same pattern is observed in other settings and domains too
ID	Sentence	Important Alignments (source → generated)
Source #1	Oh, I'm literally dismal in my writing skills but a hero in singing	(literally → very), (dismal → poor), (skills → abilities)
Generated #1	My writing abilities are very poor but my singing is excellent.	(hero → excellent), (reading → reading)
Source #2	though she is not his gf but he hangs out with her a lot!	(though → However), (gf → girlfriend, (hangs → spend), (hangs → time), (lot → much), (her → her)
Generated #2	However she is not his girlfriend, he spends too much time with her.
Table 8: Case Study for Informal to Formal conversion. The important alignments are the alignments with the highest probability predicted by solving the Optimal Transport problem for the given two sentences.
Model	BLEURT	BLEU	ACC	HM
No Semantics	0.079	0.545	0.921	0.684
No Syntax	0.064	0.521	0.912	0.663
No OT	0.050	0.509	0.889	0.647
Reconstruct	0.059	0.560	0.929	0.698
Graph-based	0.063	0.555	0.921	0.692
OT4 (Full)	0.114	0.621	0.940	0.747
alignment between the words of the input sentence and the generated sentence. Hence, hindering the content preservation computation.
To provide more insight into the performance of the proposed model, in this section we conduct a qualitative analysis. Specifically, we study how the OT-based model is able to find the perfect alignment between the words of the input sentence and the generated sentence. Note that in case of a successful style conversion, there should be a small Wasserstein distance between the two sentences, hence, the semantically related words will be aligned with each other. Table 8 shows two informal sentences along with their converted formal counterparts generated by OT4. To study the role of Optimal Transport, we report the alignments with the highest probability which are obtained by solving the OT problem for the given two sentences. This table shows that there is a high semantic similarity between the aligned words. More importantly, the aligned words have the same syntactic connections with the other words in their sentence. For instance, in the first example, the word "dismal" and its child in the dependency tree, i.e., "skills", are aligned with the word "poor" and its child in the dependency tree, i.e., "abilities". This example shows that the OT alignment considers both semantic and syntactic relations between aligned words. However, OT is not restricted to semantic or syntactic structures and it can relax the alignments whenever it is needed. For instance, in the second example, we observe that the word "time" and "hangs" are aligned with each other while serving different syn
Table 9: Ablation study on the development set of E&M domain for formal  $\leftrightarrow$  informal conversion
Original Input	It actually turned out to be pretty decent as far as B-list horror/suspense films go
RACoLN	It is a terrible movie for a category of this genre.
OT4 (Ours)	It seems to be an unsatisfactory movie for the genre of horror.
Table 10: Generated text by the proposed model and the prior SOTA model for an IMDB sample text.
tactic roles in the sentence. It shows that when the semantic relation is more important, OT can break the syntactic constraints to find a perfect alignment and the lowest Wasserstein distance. This example shows that the prior work for reconstructing the syntactic roles regardless of their semantic importance will be inferior to our proposed OT-based approach.
Finally, to qualitatively study the improvement obtained by the proposed model on the unsupervised setting, we present the generated text for a sample text from the IMDB dataset. Specifically, we compare our model output with the prior SOTA model, i.e., RACoLN. Table 10 shows the samples. It is clear from this table that the proposed model can retain more content from the input text. In particular, while RACoNL omits the genre of the movie, OT4 successfully keeps this information in the generated text. We hypothesis that the similar distance of the word "horror" to the opinion words in the input and the generated text, i.e., "decent" and "unsatisfactory", are helping to keep this information in OT4 output.
We propose a new model for encouraging content preservation in text style transfer. We demonstrate that both syntax and semantics of the input sentence and the generated sentence should be taken into account for content preservation. More importantly, we empirically show that the interaction between syntax and semantics of the input and target sentences is necessary for TST. We conducted extensive experiments on benchmark datasets in supervised and unsupervised settings, achieving state-of-the-art performance on multiple datasets.
Kunal Chawla and Diyi Yang. 2020. Semi-supervised formality style transfer using language model discriminator and mutual information maximization. In *Findings of the Association for Computational Linguistics: EMNLP* 2020, pages 2340-2354, Online. Association for Computational Linguistics.
Liqun Chen, Shuyang Dai, Chenyang Tao, Haichao Zhang, Zhe Gan, Dinghan Shen, Yizhe Zhang, Guoyin Wang, Ruiyi Zhang, and Lawrence Carin. 2018. Adversarial text generation via feature-mover's distance. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, pages 4671-4682.
Ning Dai, Jianze Liang, Xipeng Qiu, and Xuanjing Huang. 2019. Style transformer: Unpaired text style transfer without disentangled latent representation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5997-6007, Florence, Italy. Association for Computational Linguistics.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Junxian He, Xinyi Wang, Graham Neubig, and Taylor Berg-Kirkpatrick. 2020. A probabilistic formulation of unsupervised text style transfer. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net.
Kenneth Heafield. 2011. KenLM: Faster and smaller language model queries. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 187-197, Edinburgh, Scotland. Association for Computational Linguistics.
Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P. Xing. 2017. Toward controlled generation of text. In Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pages 1587-1596. PMLR.
Fei Huang,zikai Chen,Chen Henry Wu, Qihan Guo, Xiaoyan Zhu,and Minlie Huang.2021.NAST:A non-autoregressive generator with word alignment for unsupervised text style transfer. In Findings of the Association for Computational Linguistics: ACLIJCNLP 2021, pages 1577-1590, Online. Association for Computational Linguistics.
Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1746-1751, Doha, Qatar. Association for Computational Linguistics.
Thomas N. Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In 5th International Conference on Learning Representations, ICLR 2017, Toulouse, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net.
Nicholas I. Kolkin, Jason Salavon, and Gregory Shakhnarovich. 2019. Style transfer by relaxed optimal transport and self-similarity. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 10051-10060. Computer Vision Foundation / IEEE.
Kalpesh Krishna, John Wieting, and Mohit Iyyer. 2020. Reformulating unsupervised style transfer as paraphrase generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 737-762, Online. Association for Computational Linguistics.
Huiyuan Lai, Antonio Toral, and Malvina Nissim. 2021. Thank you BART! rewarding pre-trained models improves formality style transfer. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 484-494, Online. Association for Computational Linguistics.
Dongkyu Lee, Zhiliang Tian, Lanqing Xue, and Nevin L. Zhang. 2021a. Enhancing content preservation in text style transfer using reverse attention and conditional layer normalization. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 93-102, Online. Association for Computational Linguistics.
Dongkyu Lee, Zhiliang Tian, Lanqing Xue, and Nevin L. Zhang. 2021b. Enhancing content preservation in text style transfer using reverse attention and conditional layer normalization. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 93-102, Online. Association for Computational Linguistics.
Juncen Li, Robin Jia, He He, and Percy Liang. 2018. Delete, retrieve, generate: a simple approach to sentiment and style transfer. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1865-1874, New Orleans, Louisiana. Association for Computational Linguistics.
Yixin Liu, Graham Neubig, and John Wieting. 2021. On learning text style transfer with direct rewards. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4262-4273, Online. Association for Computational Linguistics.
Yun Ma, Yangbin Chen, Xudong Mao, and Qing Li. 2019. A syntax-aware approach for unsupervised text style transfer. In OpenReview.
Xing Niu, Sudha Rao, and Marine Carpuat. 2018. Multitask neural models for translating between styles within and across languages. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1008-1021, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
Gabriel Peyre and Marco Cuturi. 2019. Computational optimal transport: With applications to data science. In Foundations and Trends in Machine Learning.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
Sudha Rao and Joel Tetreault. 2018. Dear sir or madam, may I introduce the GYAFC dataset: Corpus, benchmarks and metrics for formality style transfer. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 129-140, New Orleans, Louisiana. Association for Computational Linguistics.
Eric Risser. 2020. Optimal textures: Fast and robust texture synthesis and style transfer through optimal transport. volume abs/2010.14702.
Abhilasha Sancheti, Kundan Krishna, Balaji Vasan Srinivasan, and Anandhavelu Natarajan. 2020. Reinforced rewards framework for text style transfer. In Advances in Information Retrieval, volume 12035, page 545. Nature Publishing Group.
Thibault Sellam, Dipanjan Das, and Ankur Parikh. 2020. BLEURT: Learning robust metrics for text generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7881-7892, Online. Association for Computational Linguistics.
Mingyue Shang, Piji Li, Zhenxin Fu, Lidong Bing, Dongyan Zhao, Shuming Shi, and Rui Yan. 2019. Semi-supervised text style transfer: Cross projection in latent space. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4937-4946, Hong Kong, China. Association for Computational Linguistics.
Tianxiao Shen, Tao Lei, Regina Barzilay, and Tommi S. Jaakkola. 2017. Style transfer from non-parallel text by cross-alignment. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 6830-6841.
Yunli Wang, Yu Wu, Lili Mou, Zhoujun Li, and Wenhan Chao. 2019. Harnessing pre-trained neural networks with rules for formality style transfer. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3573-3578, Hong Kong, China. Association for Computational Linguistics.
Ronald J. Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. In Kluwer Academic.
Jingjing Xu, Hao Zhou, Chun Gan, Zaixiang Zheng, and Lei Li. 2021. Vocabulary learning via optimal transport for neural machine translation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume I: Long Papers), pages 7361-7373, Online. Association for Computational Linguistics.
Wei Xu, Alan Ritter, Bill Dolan, Ralph Grishman, and Colin Cherry. 2012. Paraphrasing for style. In Proceedings of COLING 2012, pages 2899-2914, Mumbai, India. The COLING 2012 Organizing Committee.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Evaluating text generation with BERT. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net.