SlowGuess/ACL_2020_OCR huggingface dataset:
4. Fluency modeling loss  $L_{lm}$ : Finally, we follow Yang et al. (2018) to introduce a bidirectional GRU based language model, which is pre-trained on the training instances with target style to ensure that our model can generate fluent output sentences.
For the forward direction, we aim to reduce the distribution divergence between the prediction probability vector of our model and that of the forward language model by minimizing their cross-entropy as follows:
$$
\overrightarrow {L} _ {l m} \left(\theta_ {s 2 s}, \theta_ {\lambda}, \theta_ {\Delta h}\right) = \sum_ {j = 1} ^ {| Y |} \mathbf {P} \left(* | y _ {<   j}, X\right) ^ {\top} \log \left(\overrightarrow {\mathbf {P}} \left(* | y _ {<   j}\right)\right), \tag {15}
$$
where  $\mathbf{P}(*|y_{< j},X)$  and  $\overrightarrow{\mathbf{P}} (*|y_{< j})$  are the predicted probability distributions produced by our model and the forward language model, respectively. Note that at each timestep, we fed the continuous approximation of the output word, which is defined as the weighted sum of word embeddings with the current probability vector, into the language model. For the back direction, we directly reverse the output sentence and calculate the reverse language model loss  $\overleftarrow{L}_{lm}$  in a similar way. Finally, we directly define the total fluency modeling loss  $L_{lm}$  as the average of  $\overrightarrow{L}_{lm}$  and  $\overleftarrow{L}_{lm}$ .
YELP: This dataset is comprised of restaurant and business reviews and has been widely used in sentiment transfer. To evaluate our model, we adopt the human references released by Luo et al. (2019),
which has four references for each sentence in the test set. Following common practices (Shen et al., 2017; Li et al., 2018), we choose reviews over 3 stars as positive instances and those under 3 stars as negative instances. The splitting of train, development and test sets are in accordance with the setting in (Li et al., 2018). Moreover, we filter the sentences with more than 16 words, leaving roughly 448K, 64K and 1K sentences in the train and development and test sets, respectively.
GYAFC: The parallel data of GYAFC (Rao and Tetreault, 2018) consists of formal and informal texts while providing four human references for each test sentence. Particularly, we use this dataset in a non-alignment setting during training. There are roughly 102K, 5K and 1K sentences remaining in the train, development and test sets, respectively.
We compare our model with several competitive baselines: CrossAlign (Shen et al., 2017), DelRetri (Li et al., 2018), Unpaired (Xu et al., 2018), UnsuperMT (Zhang et al., 2018), DualRL (Luo et al., 2019), PoiGen (Wu et al., 2019). Among these models, DualRL and PoiGen exhibit the best performance.
As for the threshold  $\epsilon$  for filtering noise, we set it to 0.3 by observing model performances on development set. For the training of Stage 2, we empirically set the learning rate to  $1\times 10^{-5}$  and clip the gradients if their norms exceed  $10^{-2}$ . We draw  $\alpha$  from 0.5 to 1.5 with the step 0.1,  $\beta$  from 0.5 to 5 with the step 0.5 and  $\gamma$  from 0.1 to 2 with the step 0.1. Similarly, the overall performance of our model on the development set is employed to guide the hyperparameter search procedure. Finally, we choose  $\alpha = 1$ ,  $\beta = 2$  and  $\gamma = 0.5$ .
4.4 Automatic Evaluation
We evaluate the quality of output sentences in terms of transfer accuracy and content preservation. Following previous work (Luo et al., 2019), we use the pre-trained style classifier to calculate the transfer accuracy of output sentences. The classifier achieves  $97.8\%$  and  $88.3\%$  accuracy on the test sets of YELP and GYAFC, respectively. Moreover, we compute the BLEU scores of output sentences to measure content preservation. Finally, we report the geometric mean and harmonic mean of these
Model	YELP	GYAFC
Acc	BLEU	G2	H2	Acc	BLEU	G2	H2
CrossAlign (Shen et al., 2017)	75.3	17.9	36.7	28.9	70.5	3.6	15.9	6.8
DelRetri (Li et al., 2018)	89.0	31.1	52.6	46.1	55.2	21.1	34.2	30.6
Unpaired (Xu et al., 2018)	64.9	37.0	49.0	47.1	79.5	2.0	12.6	3.9
UnsuperMT (Zhang et al., 2018)	95.4	44.5	65.1	60.7	70.8	33.4	48.6	45.4
DualRL (Luo et al., 2019)	85.6	55.2	68.7	67.1	71.1	41.9	54.6	52.7
PoiGen (Wu et al., 2019)	91.5	59.0	73.5	71.8	46.2	45.8	46.0	46.0
Our Model	94.0	60.4	75.4	73.6	81.4	47.7	62.3	65.2
Table 1: Performance of different models in YELP and GYAFC. Acc measures the percentage of output sentences that match the target style. BLEU measures the content similarity between the output and the corresponding four human references.  $G2$  and  $H2$  denotes the geometric mean and harmonic mean of Acc and BLEU, respectively. Numbers in bold mean that the improvement to the best performing baseline is statistically significant (t-test with p-value  $< 0.05$ ).
Model	YELP	GYAFC
Acc	Con	Flu	Avg	Sty	Con	Flu	Avg
DelRetri (Li et al., 2018)	2.18	2.21	2.40	2.26	1.53	1.55	1.62	1.57
UnsuperMT (Zhang et al., 2018)	3.26	3.07	3.24	3.19	2.46	2.42	2.75	2.54
DualRL (Luo et al., 2019)	3.31	3.43	3.47	3.40	2.26	2.28	2.36	2.30
PoiGen (Wu et al., 2019)	3.42	3.51	3.54	3.49	1.39	1.52	1.43	1.45
Our Model	3.38	3.70	3.66	3.58	2.91	3.00	3.14	3.02
metrics, which quantify the overall performance of various models.
Experimental results in Table 1 show that our model achieves the best performance among all models.
We invite 3 annotators with linguistic background to evaluate the output sentences. The accuracy of style transfer (Acc), the preservation of original content (Con) and the fluency (Flu) are the three aspects of model performance we consider. Following the criteria introduced in (Zhang et al., 2018), the annotators are required to score each aspect of sentences from 1 to 5.
Table 2 shows the human evaluation results. Our model achieves the best performance on both datasets in terms of almost every aspects, except that the Acc score of our model is slightly lower than PoiGen on YELP. It may be due to the error of the pre-trained style classifier that the transfer accuracy of our model is higher than PoiGen on YELP. Note that the content preservation of our model is significantly higher than others, showing our word-level control actually preserves more
Table 2: Human evaluation results. We show human rating (1-5) for transfer accuracy (Acc), content preservation (Con), fluency (Flu). The average ratings (Avg) are also calculated as overall scores. Numbers in bold mean that the improvement to the best performing baseline is statistically significant (t-test with p-value  $< 0.05$ ).
Model	Acc	BLEU	G2	H2
Our Model	94.0	60.4	75.4	73.6
-NSC	88.3	55.7	70.1	68.3
NSC-λyj	98.1	10.1	31.5	18.3
-Lxλ	95.2	57.7	74.1	71.8
Lcp→L'cp	78.2	61.6	69.4	68.9
-Lyλ	94.3	59.4	74.8	72.8
-Llm	96.8	56.5	73.9	71.3
Finetuning-	91.5	55.8	71.4	68.9
Table 3: Ablation study results.
style-independent content of original sentences.
Compared with previous studies, the training of our model contains two stages, involving a neural style component and several novel loss terms, such as  $L_{x\lambda}$  (see Equation 3),  $L_{y\lambda}$ ,  $L_{cp}$  and  $L_{lm}$  (see Equation 11). To fully investigate their effects on our model, we conduct extensive ablation studies on YELP, which is larger than GYAFC. Specifically, we compare ours with the following variants:
- -NSC: A variant of our model, where the proposed neural style component is removed from the model. It should be noted that this variant is actually the basic model, which is only trained at Stage 1.
-  $NSC - \lambda_{yy}$ : It is a variant of our model, which
$^{\dagger}$ We use Fleiss kappa to quantify the agreement among them. The Fleiss kappa score is 0.76 for the Yelp dataset and 0.80 for the GYAFC dataset.
Figure 3: Sampled outputs of our model. Note that our model successfully transfers the style of these input sentences by not only replacing but also adding tokens. We also present the word-level style relevance of all sentences.
is equipped with a neural style component but without  $\lambda_{yj}$  (see Equation 8). Note that this variant does have the ability of fine-grained controlling the influence from the target style on the generation process.
-  $L_{x\lambda}$ : In this model, the loss term  $L_{x\lambda}$  is directly removed from Equation 3 at Stage 1.
-  $L_{cp} \to L_{cp}^{\prime}$ : A variant of our model, where its content preservation loss is modified as
$$
L _ {c p} ^ {\prime} \left(\theta_ {s 2 s}, \theta_ {\lambda}, \theta_ {\Delta h}\right) = \left(\sum_ {i} ^ {| X |} \mathbf {e} \left(x _ {i}\right) - \sum_ {j} ^ {| Y |} \mathbf {e} \left(y _ {j}\right)\right) ^ {2}. \tag {16}
$$
Compared  $L_{cp}^{\prime}$  with  $L_{cp}$  (see Equation 14), we find that all words are equally considered in  $L_{cp}^{\prime}$ . Thus, through this experiment, we can investigate the impact of differential word-level style relevance modeling on our model.  $-L_{y\lambda}$ : It is also a variant of our model where the weight  $\alpha$  of  $L_{y\lambda}$  is set as 0 (see Equation 11).
-  $L_{lm}$ : A variant of our model with the weight  $\gamma$  of  $L_{lm}$  as 0 (see Equation 11).
- Finetuning-: For this model, we fix all parameters involved in Stage 1 at Stage 2.
Table 3 lists the experimental results. We can observe that most variants are significantly inferior to our model in terms of BLEU scores. Particularly,
although the  $Acc$  of some variants increase, these models may overly change the original content to conduct transfer, still resulting in lower BLEU scores. These results demonstrate the effectiveness of our introduced neural style component, different loss terms and two stage training strategy. As an exception of above observations, when we replace  $L_{cp}$  with  $L_{cp'}$ , the BLEU score increases but the Acc drops significantly. This is due to the fact that  $L_{cp'}$  does not discriminate words of different style relevances and overly constrain the model to keep its original content.
4.7 Case Study
We conduct case study to understand the advantage of our model. Figure 3 displays several instances of input and output sentences with word-level style relevance. For example, according to the word-level style relevance from LRP, we can observe that the words of the first input YELP sentence, including "slow" and "lazy" are most related to the original style, while other words hardly contribute to the style of the whole sentence. Meanwhile, some words of the first output sentence, such as "really", "good" and "friendly" are predicted to be most relevant to the target style. Our model successfully exploits the predicted style relevance and change the word "good" and "friendly" while keeping other parts unchanged. In the second informal-to-formal transfer case, besides replacing "just" with "simply", our model also appends a ".." token with high predicted style relevance, which is a sign character
$^{\ddagger}$ Because our model would collapse if  $L_{cp}$  is completely removed from the objective function, we do not compare our model with its variant without  $L_{cp}$  loss.
izing formality.
From Figure 3, we can see that the relevance scores of words in GYAFC are less discriminative compared to those in YELP. Thus, we provide the corpus-level statistics by counting the frequencies of some typical words of GYAFC in Figure 3, showing that the predicted scores often indicate the distribution of words across different styles. "simply" appears 29 and 380 times in the informal and formal sets, respectively. The period token '.' is an indicative marker of text formality since lots of informal sentences end with no punctuation. Besides, "your", "about" and "this" are distributed uniformly across styles. There are 4,590 and 5,357 "your", 2,489 and 2,353 "about", 2,062 and 2,211 "this" appearing in the informal and formal sets, respectively.
These results are consistent with our intuition, verifying the correlation between the predicted style relevance of each word and its actual stylistic outcome.
This paper has proposed a novel attentional Seq2seq model equipped with a neural style component for unsupervised style transfer. Using the quantified style relevance from a pre-trained style classifier as supervision information, our model is first trained to reconstruct input sentences and repredict the word-level style relevance simultaneously. Then, equipped with the style component, our model can exploit the word-level predicted style relevance for better style transfer. Experiments on two benchmark datasets prove the superiority of our model over several competitive baselines.
In the future, we plan to adapt variational neural network to refine our style transfer model, which has shown effectiveness in other conditional text generation tasks, such as machine translation (Zhang et al., 2016; Su et al., 2018).
This work was supported by the Beijing Advanced Innovation Center for Language Resources (No. TYR17002), the National Key R&D Project of China (No. 2018AAA0101900), the National Natural Science Foundation of China (No. 61672440), and the Scientific Research Project of National Language Committee of China (No. YB135-49).
Sebastian Bach, Alexander Binder, Gregoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojciech Samek. 2015. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. *PloS one*.
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of ICLR 2015.
Ning Dai, Jianze Liang, Xipeng Qiu, and Xuanjing Huang. 2019. Style transformer: Unpaired text style transfer without disentangled latent representation. In Proceedings of ACL 2019.
Zhenxin Fu, Xiaoye Tan, Nanyun Peng, Dongyan Zhao, and Rui Yan. 2018. Style transfer in text: Exploration and evaluation. In Proceedings of AAAI 2018.
Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P. Xing. 2017. Toward controlled generation of text. In Proceedings of ICML 2017.
Parag Jain, Abhijit Mishra, Amar Prakash Azad, and Karthik Sankaranarayanan. 2019. Unsupervised controllable text formalization. In Proceedings of AAAI 2019.
Eric Jang, Shixiang Gu, and Ben Poole. 2017. Categorical reparameterization with gumbel-softmax. In Proceedings of ICLR 2017.
Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of EMNLP 2014.
Guillaume Lample, Sandeep Subramanian, Eric Michael Smith, Ludovic Denoyer, Marc'Aurelio Ranzato, and Y-Lan Boureau. 2019. Multiple-attribute text rewriting. In Proceedings of ICLR 2019.
Jiwei Li, Will Monroe, Tianlin Shi, Sébastien Jean, Alan Ritter, and Dan Jurafsky. 2017. Adversarial learning for neural dialogue generation. In Proceedings of EMNLP 2017.
Juncen Li, Robin Jia, He He, and Percy Liang. 2018. Delete, retrieve, generate: a simple approach to sentiment and style transfer. In Proceedings of NAACL 2018.
Dayiheng Liu, Jie Fu, Yidan Zhang, Chris Pal, and Jiancheng Lv. 2019. Revision in continuous space: Fine-grained control of text style transfer. CoRR, abs/1905.12304.
Lajanugen Logeswaran, Honglak Lee, and Samy Bengio. 2018. Content preserving text generation with attribute controls. In Proceedings of NIPS 2018.
7143
Fuli Luo, Peng Li, Jie Zhou, Pengcheng Yang, Baobao Chang, Zhifang Sui, and Xu Sun. 2019. A dual reinforcement learning framework for unsupervised text style transfer. In Proceedings of IJCAI 2019.
Shrimai Prabhumoye, Yulia Tsvetkov, Ruslan Salakhutdinov, and Alan W. Black. 2018. Style transfer through back-translation. In Proceedings of ACL 2018.
Sudha Rao and Joel R. Tetreault. 2018. Dear sir or madam, may I introduce the GYAFC dataset: Corpus, benchmarks and metrics for formality style transfer. In Proceedings of NAACL 2018.
Tianxiao Shen, Tao Lei, Regina Barzilay, and Tommi S. Jaakkola. 2017. Style transfer from non-parallel text by cross-alignment. In Proceedings of NIPS 2017.
Jinsong Su, Shan Wu, Deyi Xiong, Yaojie Lu, Xianpei Han, and Biao Zhang. 2018. Variational recurrent neural machine translation. In Proceedings of AAAI 2018.
Jinsong Su, Jiali Zeng, Jun Xie, Huating Wen, Yongjing Yin, and Yang Liu. 2019. Exploring discriminative word-level domain contexts for multi-domain neural machine translation. Transactions on Pattern Analysis and Machine Intelligence.
Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Proceedings of NIPS 2014.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of NIPS 2017.
Chen Wu, Xuancheng Ren, Fuli Luo, and Xu Sun. 2019. A hierarchical reinforced sequence operation method for unsupervised text style transfer. In Proceedings of ACL 2019.
Jingjing Xu, Xu Sun, Qi Zeng, Xiaodong Zhang, Xu-ancheng Ren, Houfeng Wang, and Wenjie Li. 2018. Unpaired sentiment-to-sentiment translation: A cycled reinforcement learning approach. In Proceedings of ACL 2018.
Zichao Yang, Zhiting Hu, Chris Dyer, Eric P. Xing, and Taylor Berg-Kirkpatrick. 2018. Unsupervised text style transfer using language models as discriminators. In Proceedings of NIPS 2018.
Jiali Zeng, Jinsong Su, Huating Wen, Yang Liu, Jun Xie, Yongjing Yin, and Jianqiang Zhao. 2018. Multi-domain neural machine translation with word-level domain context discrimination. In Proceedings of EMNLP 2018.
Biao Zhang, Deyi Xiong, Jinsong Su, Hong Duan, and Min Zhang. 2016. Variational neural machine translation. In Proceedings of EMNLP 2016.
Zhirui Zhang, Shuo Ren, Shujie Liu, Jianyong Wang, Peng Chen, Mu Li, Ming Zhou, and Enhong Chen. 2018. Style transfer as unsupervised machine translation. In Proceedings of AAAI 2019.
7144