Towards Formality-Aware Neural Machine Translation by Leveraging Context Information
Authors: Dohee Kim
Yujin Baek
Soyoung Yang
Jaegul Choo
Venue: EMNLP 2023 Findings (2023)
Keywords: Neural Machine Translation
Context-Aware Translation
Formality-Aware Translation
Formality Control

Abstract: Formality is one of the most important linguistic properties to determine the naturalness of translation. Although a target-side context contains formality-related tokens, the sparsity within the context makes it difficult for context-aware neural machine translation (NMT) models to properly discern them. In this paper, we introduce a novel training method to explicitly inform the NMT model by pinpointing key informative tokens using a formality classifier. Given a target context, the formality classifier guides the model to concentrate on the formality-related tokens within the context. Additionally, we modify the standard cross-entropy loss, especially toward the formality-related tokens obtained from the classifier. Experimental results show that our approaches not only improve overall translation quality but also reflect the appropriate formality from the target context.

Towards Formality-Aware Neural Machine Translation by Leveraging Context Information
Dohee Kim
Yujin Back Soyoung Yang
Jaegul Choo
KAIST AI
[doheel 121, yujinbaek, sy-yang, jchoo) ac.
Abstract
Source sentence
Si
Do you have plans this weekend? actually pretty busy this weekend.
Formality is one of the most important lin- guistic properties to determine the naturalness of translation. Although target-side context tokens, the sparsity within the context makes difficult context- aware neural machine translation (NMT) mod- els to discern them properly. In this paper, we introduce novel training method ex- plicitly inform the NMT model by pinpointing key informative tokens using a formality clas- sifier. Given target context, the formality classifier guides the model to concentrate on the formality-related tokens within the context. Additionally, modify the standard cross- entropy loss, especially toward the formality- related tokens obtained from the classifier. Ex- perimental results show that our approaches not only improve overall translation quality but also reflect the appropriate formality from the target context.
S2
Target sentence
T7
Tool
EHE
2|
214
contains formality-
T2
4
T
o
T
T
A
+
||
Table 1: Example from OpenSubtitles (En->Ko) Trans- lation of the current target sentence (72) depends on the target- context (Tz). When honorific context (T is given, it is natural that the subject(" =i) of the current target sentence is also translated as an honorific (T."). A subtle level of formality can be captured in the target-side context.
ending( "*
Since recent context-aware NMT models (Maruf et al., 2019; Zhang et al., 2020; Bao et al., 2021) incorporate the context into the NMT task, these models can be utilized infer the appropriate for- mality from the context and, accordingly, generate consistent translations. However, due the spar- sity of context (Lupo et al., 2021), it challenging to capture few formality-related words scattered within the context. Yin et al. (2021) guides the model to concentrate on the relevant contexts by collecting human- annotated contextual dataset, but this method resorts to manual annotations, which is time-consuming and labor-intensive. Additionally, post-editing is another approach for addressing the delicate formality, which needs cognitive efforts.
1 Introduction
Translation quality is determined not only by pre- serving semantics across languages but also by conforming to an appropriate syntax. Formality is one of the important syntactic properties, includ- ing honorifics. For example, as shown in Table 1, S2 can be translated into two variants: informal (TA)) and honorific (I: ). At the sentence level, the neural machine translation (NMT) models can generate translations conditioned on provided for- mality (Niu et al., 2017, 2018; Feely al., 2019). While this approach sheds some light on handling the stylistic variations in NMT, is still difficult to control the subtle differences within a class based on pre-defined formality class. To overcome this issue, we focus on the beyond sentence-level NMT setting, where the formality information remains intact in the target-side context. For instance, in Table 1, A and 7 have specific formality tokens highlighted in pink and blue, respectively.
Therefore, we propose a new training approach to encourage the NMT model to pinpoint infor- mative context tokens using formality classifier. We obtain importance distribution of each con- text from the classifier, which has more weight on formality
tokens. We then regularize the at- tention map of the NMT decoder toward Besides this instance-level guidance, we extract salient to- kens contributing to the formality class from the
"Equal Contributions.

---

classifier. We assign more weights to the loss for these tokens to strengthen formality-controlled gen- eration. Since the NMT model and classifier share vocabulary and tokenizer, the classifier can be flexi- bly integrated to generate formality-sensitive trans- lations.
Masked Multi-Head
Attention Regularization
Attention Map
0.31
0.27
0:42
yu [SEP) y
Classifier
Our contributions can be summarized as follows:
y
heads
We propose novel training strategy extract sparsely distributed rermted informa- tion using formality classifier.
R-1
[SEP]
yk
Augmentation
Figure I: Instance-level context attention regularization mechanism. We obtain a continuous distribution of the target context by making perturbations (5*-1) to the target context y²= We then regularize two attention heads the top decoder layer.
We emphasize that our approach is easily adaptable to the NMT model since the classi- fier and NMT model have the same vocabulary and tokenizer.
We prove that regularized attention aligns with human perception, and empirical results show that our approach improves translation quality in terms of BLEU and human evaluation.
{u' ........ ".....,!,, ). Basically, the probability of trans- lating st into y given the context C is defined as:
T II P weer C)
2 Related Work
As NMT models have reached a considerable level, handling of discourse phenomena (Bawden et al., 2018; Müller et al., 2018; Voita al., 2019b) has become crucial to improve translation quality. For- mality an important linguistic feature addressed by context-aware NMT models for coherent and fluent discourse.
where 3/ represents the 1hh token of hth sentence and y represents all the previous target tokens of h sentence. When the context size of the source and target is one, the context C' is {-^-1 1 "1 where the target context n is the previously gen- erated target sentence.
However, to generate consistent translation flecting the sophisticated formality of the target context y 1, training and inference strategies are modified. When training with teacher forcing, the cross-entropy loss is not applied to the target con- text. During the model inference, the target context is not generated but provided as condition to avoid exposure bias.
Since few tokens within sentence reveal the formality, context-aware NMT models have diffi- culty to training and evaluating consistent stylistic translation. First, due to this sparsity (Lupo et al., 2021) that also occurs within the context, a training alternative is needed amplify the weak training signal. addition, BLEU (Papineni et al., 2002), a standard translation quality metric, often fails to capture improvements, which bring subtle dif- ferences. To better perceive these differences in translation, human evaluation is sometimes per- formed in parallel (Voita et al., 2019a; Xiong et al., 2019; Freitag al., 2022). Our approach directs the model reach the heart of minor but deci- sive formality tokens. As result, our experiments include the performance in terms of BLEU and human evaluation.
3.1 Formality Classifier as Context Guider
We extract -nnoolee clues of each target context at the instance-level and formality-sensittee token set within a whole training corpus at the global level (Ramon al., 2020).
Formality Classifier We train a binary classi- fier to classify whether the sentence is honorific or not. The classifier consists of scaled dot-product attention (Vaswani et al., 2017), linear, and soft- max layer consecutively. Since formality-labeled monolingual data is unavailable for Koreans, we au- tomatically label portions of the pre-training corpus by identifying several honorific endings as heuris- tic rules. We hypothesize that the classifier, which
3 Methods
Given a source document X = {
xx? ,..,,,.. N..,,:NN
as sequence of N sentences, context-aware
NMT models translate a target document

---

extrapolates from few heuristic rules, learns abun- dant formality-related information. The classifier leverages the same vocabulary and tokenizer as the NMT model, resulting to control of the NMT generation immediately. detailed description of the training formality classifier can be found in Appendix A.I.
where wy 1.0 if yf is token belonging F. and w/ otherwise.
Experimental Settings
4.1 Data
We conduct experiments for two language pairs: Biggish--Koreen/aaaaneee.
Instance-level context attention regularization To guide the model distinguish the importance within the target context, we obtain importance dis- tribution for each target context from the classifier through LIME algorithm (Ribeiro et al., 201 As shown in Figure 1, the algorithm is used to generate token-level distribution based on the impact on pre- dicting formality class by removing random tokens from the target context. Therefore, the distribution has more weight the formality-involved tokens.
En We pre-train sentence-level bascline on a large parallel corpus of 1.5M sentence pairs in AI-HUB We use two fine-tuning datasets: OpenSubritle2001s 8 (Lison et al., 2018) and AI- HUB dialogue corpus. For OpenSubtitles dataset, noisy parallel sentences are filtered out through LABSE (Feng al., 2020) for sentence alignment. We split the refined dataset into 63K for training, 3.3K for validation, and 2.6K for testing.
We apply attention regularization encourage the attention of the NMT model to be aligned with the normalized token-level distribution generated by the classifier. We regularize the self-attention from two heads of the top decoder layer. The loss for attention regularization is:
En We use a publicly available pre-trained model trained on JParaCrawl (Morishita et al., 2020) of 22M sentence pairs. The Business Scene Dialogue (BSD) (Rikters et al., 2019), AMI Meeting (Rikters al., 2020) and OpenSubti- tles2018 (Lison et al., 2018) datasets are combined for fine-tuning. The dataset consists of 97K train- ing data, 63K validation data, and 67K test data.
CAR -aK LaK [P attn Pelasss fir fier-attn)
where a is a weight scalar to balance between trans- lation loss and attention regularization loss.
4.2 Models
Global-level token weighted loss To further in- duce the model to generate appropriate formality tokens, we propose training method that assigns more weights to a set of formality tokens extracted from the classifier. Inspired by Sun and Lu (2020), the salient formality token set F for each class is obtained based on the attention score, which means global importance within a corpus.
We consider three models as baselines; Sent-level is an abbreviation for the sentence-level model with- out any context. Concat-level has both previous source and target sentences as context, but for the previous target sentence, its loss is not applied dur- ing the training phase. For the inference, the refer- ential target context sentence is provided as prefix tokens. Tag context model is given a formality tag of the classifier as target-side context. Concat- level Tag model is given both formality tag and previous target sentence as target-side context.
The attention score for an arbitrary token i is:
h V
a
where h; is the embedded representation of token i, V is learnable query vector, and X is scaling scalar.
In experiments, we focus on the single-encoder architecture that concatenates the context and the current sentences (Bao et al., 2021); based on the findings conducted by Lopes et al. (2020) that this simple approach demonstrates competitiveness even surpasses its more complex counterparts.
To prevent the sparse distribution of formality tokens from being diluted during training, the loss for these tokens emphasized by applying more weights.
The modified cross-entropy of translating x² into t is as follows:
4.3 Results
Table shows the performance of our approaches and baselines. We achieve moderate improvements
CMT walogp(utlat 2L C)
www. aihub. or. kr (Data usage permission is required.)

---

En En-Ko Ko
En-tJa
Non-regularized Heads (Concat)
AI-HUB
OpenSub
AMI+BSD +OpenSub
210
211
Howwrtffe
Non-regularized Heads (Ours)
Sent-level
26.15
12.38
7/0
7/4 2
Concat-level
27.64
20.14
12.77
Regularized Heads (Ours)
Tag context
26.98
19.46
12.07
Concat-level + Tag
27.82
19.93
13.47
[SEP]
Ours (WL)
27.85
20.44
13.53
Non-regularized Heads (Comcat)
Ours (AmmReg)
27.86
20.38
12.91
01 21
81
64210) L-C1+
Informal
Ours (WL+ArtnReg)
27.90
20.70
13.67
Non-regularized Heads (Owrs)
0198 PM
U/7 04710l 1/EH [SSP]] Hi
Table Experimental results reported in BLEU. AunReg denotes model with instance-level context attention regularization. WL indicates model with global-level token weighted loss. For AI-HUB and AMI-BSD+Opennubb datasets, the improvement of our model (WL+ArmReeg) is statistically sigmificant at < 0.05 compared to the Concat- level baseline. For Open- Sub dataset, it is statistically significant at < O.
Regularized Heads (Ours)
Ss my busband recehring an organ transplant or And what about my older sister? Hlow dare you show up here! You ruined my life.
Figure 2: Qualitative examples are visualized based on attention weights. Higher intensity of the color indi- cates higher attention weight. The honorific example highlighted in purple, while the informal example is highlighted in green.
for each of our models in BLEU (Post, 2018) even though the Concat-level is known as a dom- inant baseline (Bao et al., 2021), where the target context itself is provided as a condition. When both instance-level context attention regularization and global-level token weighted loss are combined, the BLEU score was the highest.
Analysis
Attention Visualization Figure 2 illustrates how attention weights from regularized and non- regularized heads are distributed when generating formality-related tokens, which marked bold. Regularized heads (ours) in each example are vi- sualized with the averaged attention weights over two regularized heads in AtmReg. Non-regularized heads (concat) and non-regularized heads (ours) are highlighted using the averaged attention weights over all non-regular heads in concat-level and At- tnReg, respectively. We observe that regularized heads pay proper attention to formality-related to- kens within the target context, such as endings ("cll _²" or nominative markers ("7P"); on the other hand, non-regularized heads seem to focus mostly on the previous few tokens. This experiment shows that the model is able to learn various end expres- sions via the classifier, and the classifier can control the formality beyond rule-based learning.
We also conduct human evaluation on each of 50 examples from the Concat-level baseline and our final model (WL+AtnReg) where both methods are applied. Following Rao and Tetreault (2018); Niu and Carpuat (2020), we adopt two eval- uation criteria: formality consistency and mean- ing/content preservation.
Formality consistency We asked 11 human par- ticipants the question, "Choose the sentence that reflects the formality of the previous contextual sentence.". Our model acquires 93% of the total votes, meaning that our model better reflects the subtle stylistic differences. The kappa coefficient, which indicates agreement between 11 annotators, is 0.70.
Meaning/content preservation We asked the other human participants the question, "Given reference translation, which one of the two gen- erated translations, regardless of formality, better preserves its meaning?". An option "It's hard to say which better preserves the reference meaning." added. As a result, 62% of examples were considered both are comparable, 28% were judged better translated by our model.
"Given the
A Alignment between Regularized Heads and Hu- man Perception Attention regularization makes our model explicitly refer to key syntactic tokens that determine the formality of contextual sen- tences. We investigate how the attention weights of regularized heads are aligned with human per- ception. The attention weights of regularized heads in our model are shown to be adequately aligned with human perception showing the highest similar-
WW measure SacreBLEU scores with the signature BLEU Inrefs:1 Icasermixed leff:no Itok: (ko,jjjaa mocjjj//mccabssmoth...y Iversion:2.2.0.

---

ity score of 0.669 3, while non-regularized heads hardly align with human perception: 0.033 (non- regularized heads for our model) and 0.029 (non- regularized heads for concat-level baseline).
grant funded by the Korea government (MSIT) (No. 1RR-22022R|A22555000001111)
References
6 Conclusion
Guangsheng Bao, Yue Zhang. Zhiyang Teng, Boxing Chen, and Weihua Luo. 2021. G-transformer for document-level machine translation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 11th International Joint Conference on Natural Language Processing (Volume I: Long Papers), pages 3442-3455, Online. Association for Computational Linguisties.
To generate consistent translations reflecting the formality of the target-side context, we propose a new training approach to focus on formality- involved tokens by leveraging the formality classi- fier. Our approach results in convincing improve- ments in translation quality as measured by BLEU and human evaluation. As a result, we suggest a les- son that the formality classifier easily applicable to NMT model as a guider.
Rachel Bawden, Rico Sennrich, Alexandra Birch, and Barry Haddow. 2018. Evaluating discourse phenom- ena in neural machine translation. In Proceedings of the 2018 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, Volume / (Long Pa- pers), pages 1304-1313, New Orleans, Louisiana. Association for Computational Linguistics.
Limitations
We aim to generate appropriate translations in terms of syntactic formality. The improvements achieved are likely to be subtle; however, these sub- le differences are not properly captured by BLEU. perform
Weston Feely, Eva Hasler, and Adrià de Gispert. 2019. Controlling Japanese honorifics in English- to-Japanese neural machine translation. In Proceed- ings of the 6th Workshop on Asian Translation, pages 45 53, Hong Kong. China. Association for Computa- tional Linguistics.
To better identify these differences, human evaluation, which incurs an additional ex- pense. Although we appreciate the importance of introducing an automatic metric achieve scalabil- ity, we focus more on generating formality-relate changes in translation for now, leaving it to future work.
Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Ari- vazhagan, and Wei Wang. 2020. Language-agnostt bert sentence embedding.
Markus Freitag, Ricardo Rei, Nitika Mathur, Chi-kiu Lo, Craig Stewart, Eleftherios Avramidis, Tom Koemi, George Foster, Alon Lavie, and André F. T. Martins. 2022. Results of WMT22 metrics shared task: Stop using BLEU neural metrics are better and more robust. In Proceedings of the Seventh Conference Machine Translation (WMT), pages 46-68, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
Ethical Consideration
Since our work is in the field of controlled text generation, if an undesirable formality of target- side context is given, our model is amenable to leveraging However, we propose our work in anticipation of positive applications such as help- ing business conversation or academic writing. We also make sure that our academic use of AI-HUB dataset is consistent with the intended use demon- strated in its license.
Karthik Gopalakrishnan, Behnam Hedayatnia, Qin- lang Chen, Anna Gottardi, Sanjeev Kwatra, Anu Venkatesh, Raefer Gabriel, and Dilek Hakkani-Tine 2019. Topical-Char Chat: Towards Knowledge-Grounat Grounded Open-Domain Conversations. In Proc. Interspeech 2019, pages 1891-1895.
Acknowledgement
Taku Kudo and John Richardson. 2018. SentencePiece: A simple and language independent subword tok- enizer and detokenizer for neural text processing. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 66-71, Brussels, Belgium. Association for Computational Linguistics.
This work was supported by SAMSUNG Research, Samsung Electronics Co., Ltd., Institute of Infor- mation & communications Technology Planning & Evaluation (ITPP) grant funded by the Korea gov- emmeet (MSIT) (No. 20002019-0-00075, Artificial In- telligence Graduate School Program (KAIST)), and the National Research Foundation of Korea (NRF)
Pierre Lison, Jörg Tiedemann, and Milen Kouylekov. 2018. OpenSubriiles2018:: Statistical rescoring of sentence alignments in large, noisy parallel corpora. In Proceedings of the Eleventh International Confer- ence on Language Resources and Evaluation ( (LREC
We measure dat-product as a similarity score. Details of human perception are provided in appendix A.7.

---

2018), Miyazaki, Japan. European Language Re- sources Association (ELRA).
In Findings of the Association for Computational Lin- guisties: NAACL 2022, Seattle, USA. Association for Computational Linguistics.
António Lopes, M. Amin Farajian, Rachel Bawden, Michael Zhang, and André F. T. Martins. 2020. Document-level neural MT: A systematic compar- ison. In Proceedings of the 22nd Annual Conference of the European Association for Machine Translation, pages 225-234 Lisboa, Portugal. European Associa- tion for Machine Translation.
Myle Oit, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng. David Grangier, and Michael Auli. 2019. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of the 2019 Con- ference the A American Chapter of the Associa- tion for Computational Linguistics (Demonstrations), pages 48-53, Minneapolis, Minnesota. Association for Computational Linguistics.
Lorenzo Lupo. Marco Dinarelli, and Laurent Besacier. 2021. Divide and rule: Training context-aware multi-encoderr translation models with little resources. arXiv preprint 210i:22103.7151.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. Bleu: a method for automatic evalu- ation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Compu- tational Linguistics, pages 311 311-318, Philadelphia. Pennsylvania, USA. Association for Computational Linguistics.
Sameen Maruf, André F. T.. Martins, and Gholamreza Haffari. 2019. Selective attention for context-aware neural machine translation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, Volume I (Long and Short Pupers), pages 3092-3102, Minneapolis, Min- nesota. Association for Computational Linguistics.
Matt Post. 2018. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186- 191, Brussels, Belgium. Association for Computa- tional Linguistics.
Makoto Morishita, Jun Suzuki, and Masaaki Nagata, 2020. JParaCrawl: A large scale web-based English- Japanese parallel corpus. In Proceedings of The 12th Language Resources and Evaluation Confer- ence, pages 3603-3609, Marseille, France. European Language Resources Association.
Yanou Ramon, David Martens, Foster Provost, and Theodoros Evgeniou. 2020. A comparison of instance-level counterfactual explanation algorithms for behavioral and textual data: Sede, lime-c and shap-c. Adv. Data Anal. Classif, 14(4):801-81,
Mathias Müller, Annette Rios, Elena Voita, and Rico Sennrich. 2018. A large-scale test set for the evalua- tion of context-aware pronoum translation in neural machine translation. In Proceedings of the Third Conference Machine Translation: Research Pa- pers, pages 61-72, Brussels, Belgium. Association for Computational Linguistics.
Sudha Rao and Joel Tetreault. 2018. Dear sir or madam, may introduce the GYAFC dataset: Corpus, bench- marks and metrics for formality style transfer. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 129-140, New Or- leans, Louisiana. Association for Computational Lin- guisties.
Xing Niu and Marine Carpuat. 2020. Controlling neu- ral machine translation formality with synthetic su- pervision. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05):8568-851)
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "why should trust you?": Explain- ing the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference Knowledge Discovery and Data Mining, San Fran- cisco, CA, USA, August 13-17, 2016, pages 1135- 1144.
Xing Niu, Marianna Martindale, and Marine Carpuat. 2017. A study of style in machine translation: Con- trolling the formality of machine translation output. In Proceedings of the 2017 Conference on Empiri- cal Methods in Natural Language Processing, pages 281 2814-2819, Copenhagen, Denmark. Association for Computational Linguistics.
Matiss Rikters, Ryokan Ri, Tong Li, and Toshiaki Nakazawa. 2019. Designing the business conversa- tion corpus. In Proceedings of the 6th Workshop on Asian Translation, pages 54-61, Hong Kong, China. Association for Computational Linguistics.
Xing Niu, Sudha Rao, and Marine Carpuat. 2018. Multi- task neural models for translating between styles within and across languages. In Proceedings of the 27th International Conference on Computational Lin- guistics. pages 1008-1021, Santa Fe, New Mcxico, USA. Association for Computational Linguistics.
Matiss Rikters, Ryokan Ri. Tong Li, and Toshi- aki Nakazawa. 2020. Document-alignn aligned japanese- english conversation parallel corpus. In Proceed- ings of the Fifth Conference Machine Translation, pages 637-643, Online. Association for Computa- tional Linguistics.
Maria Nädejde, Anna Currey, Benjamin Hsu, Xing Niu, Marcello Federico, and Georgiana Dinu. 2022. CoCoA-MT: A dataset and benchmark for Con- trastive Controlled MT with application to formality.

---

Xiaobing Sun and Wei 2020. Understanding atten- tion for text classification. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3418-344 8-3428, Online, Association for Computational Linguistics.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, f. ukasz Kaiser, and Illia Polosukhin. 2017. Attention all you need. In Advances in Neural Information Pro- cessing Systems, volume 30. Curran Associates, Inc.
Elena Voita, Rico Sennrich, and Ivan Titov. 2019a. Context-aware monolingual repair for neural ma- chine translation. In Proceedings of the 2019 Confer- ence on Empirical Methods in Natural Language Pro- cessing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 877-886, Hong Kong. China. Association for Computational Linguistics.
Elena Voita, Rico Sennrich, and Ivan Titov. 2019b. When good translation is wrong in context: Context- aware machine translation improves deixis, ellip- sis, and lexical cohesion. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1198 1212, Florence, Italy. Asso- ciation for Computational Linguistics.
Hao Xiong. Ruiqing Zhang, Chuanqiang Zhang. Zhongjun He. Hua Wu, and Haifeng Wang. 2019, Dutongchuan: Context-aware translation model for simultaneous interpreting.
Kayo Yin, Patrick Fernandes, Danish Pruthi, Aditi Chaudhary, André F. T. Martins. and Graham Neu- big. 2021. Do context-aware translation models pay right attention? In Proceedings of the 59th nual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer- ence on Natural Language Processing (Volume Long Papers), pages 788-801, Online. Association for Computational Linguistics.
Pei Zhang, Boxing Chen, Niyu Ge, and Kai Fan. 2020. Long-short term masking transformer: A simple but effective baseline for document- level neural machine translation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Process- ing (EMNLP), pages 1081-1087, Online. Association for Computational Linguistics.

---

A Appendix
Extracting global-level formality token set The attention score, which is the standard for ex- tracting honorific tokens, is more than 0.1 for Open- Subtitles dataset and more than 0.2 for AI-HUB dialog and combined Japanese dataset, respectively. Conversely, the attention score is less than -0.2 and -0.1 extract the informal tokens.
A.5
A.1 Training Formality Classifier
For English-Korean language pair, we automat- ically label monolingual pre-training corpus by identifying some pre-defined honorific endings as heuristic rules ("22", "X|="", "4 ">|="" "á 4 ")"". The Korean corpus is randomly sampled and split into 6.4K for training, 1.6K for validation, and 1.6K for testing.
A.6 Implementation Details
We conduct all our experiments on fairseq frame- work (Ott et al., 2019). For English lan- guage pair, we use 32K shared vocabulary. For English-JJaanese language pair, vocabulary is provided with sentencepiece models (Kudo and Richardson, 2018). All context-aware NMT mod- els are an encoder-decoder Transformer architec- ture (Vaswani et al., 2017), which is concretely transformer base (60,528, 128 training parameters). We train the models with the optimization details described in (Vaswani et al., 2017). We clip the norm of the gradient to be no more than 0.1, since gradient overflow is detected without clipping.
For English-Japanese language pair, we use Nädejde et al. (2022) dataset to train the binary formality classifier. The dataset consists of 2K training data, 0.6K validation data and 0.6K test data, which comes from two domain: Telephony and Topical-Chat (Gopalakrishnan et al., 2019). To use the same tokenizer and vocabulary as the NMT model, we use the sentence-pieee model and vocabulary provided by Morishita et al. (2020), which are used for pre-training. The accuracies of Korean and Japanese classifiers are 98.7% and 94.8%, respectively.
For model selection, we save the model check- point per epoch, and the model with the validation loss in a single run is selected our final model. It takes about 6 hours single NVIDIA A100 machine train our model with AI-HUB dialog dataset.
A.2 LIME algorithm
We conduct experiments varing the number of per- turbations (1²-1 in Figure 1): we choose 100 after experimenting with 20, 50, 100, 500 and 1000.
A.3 Global-level token loss weight
A.7 Acquisition of Human Perception We derive human perception on informative formality-related tokens from annotations made by 13 native Koreans. We ask them to annotate 50 sampled sentences. Given tokenized sentence in Korean, annotators are asked to select tokens that determine its formality. Majority tokens in the sentence are considered ground-truth la- bels. Then, each ground-trute token is assigned an importance score proportion to the votes cast, We obtain the similarity between this human per- ception and the attention weights of the models by calculating their dot product.
The hyperparameter wy was tuned in the range [1.1, 1.25, 1.4, 1.5, 1.7, 1.75, 2.0, 2.5, 3.0].
A.4 Analysis of Global-level token weighted loss
To demonstrate the effectiveness of the global-level token weighted loss (WL) qualitatively, we examine how confidently model (WL) generates appro- priate formality token compared concat-level baseline. As shown in Table 3, our model succeeds to predict formality-related token "Q" accurately, Since the target-side context (T1) represents hon- orific context colored as blue, the current target sen- tence should maintain an honorific tone. However, after "H""" the concat-level model (Tyoncut) yields "?" with 66.1% %% in place of "Q" (29.5%), which completes an informal sentence. However, our model (I)uu)) produces an honorific token "Ω" with 58.6%, instead of generating "?" with 38.3%. It implics that WL helps the NMT model to gener- ate formality-relaterr tokens.

---

Source sentence
S1
There's an older daughter, well.
S2
What?
Target sentence
Ti
otoo 4477
Trnnu
#
T
+
Table Example from OpenSubtitles (En=+Ko). Trans- lation of the current target sentence (T2) would depend on the target-side context (Ti).


Reviews:
[{"review_id": "qqJAJOinA9", "reviewer": "Reviewer_9XVS", "summary": "", "questions": "- Do human annotators prioritize formality consistency over generic quality? What if they conflict?\n- Are you going to release code and data?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the thoroughness invested in your review. We extend our gratitude for all constructive comments. To enhance readability, we have noted the comments in block quotes along with the corresponding responses below:\n\n> The proposed method does not use generated target context but the reference translation, which dramatically limits the practical use case. When will the preceding reference translation be available for an MT task? Computer-assisted translation is one possibility but it's a specially use case which is not discussed in the paper. \n\nWe clarify that **our proposed approach does not demand  an accurate reference translation as a target-side contextual input for model inference**. Our model’s emphasis lies predominantly in capturing syntactic formality that can only be comprehended in target language, rather than semantics of the context. In light of this, reference translation can be replaced with contextual information that adheres to formality considerations.\n\n> (1) There is no intrinsic evaluation of the formality classifier and the contribution of context tokens. It's not clear if the model receives reliable formality signals in the first place.\n\nWe further evaluated the formality classifier to ensure that the formality-related token set is properly extracted to send a reliable formality signal to the NMT model. We used token-level formality-annotated test set, which was recommended by the reviewer 2 (https://github.com/amazon-science/contrastive-controlled-mt/tree/main/IWSLT2023/data/test/en-ko).\n\nWe extracted all of the human-annotated formality-related tokens and measured the attention score of the formality classifier for each token. Then we examined where it is greater than 0.2 or less than -0.2, which is the threshold for formality-token set extraction, as described in appendix A.4 Extracting global-level formality token set. As a result, we obtained 0.81 recall, which is reasonable.\n\n> (2) The BLEU score differences are small and the confidence intervals are not provided. We don't know if the improvements are significant.\n\nFor AI-HUB and AMI+BSD+OpenSub datasets, the improvement of our model (WL+AttnReg) is **statistically significant at p < 0.05** compared to the Concat-level baseline. For OpenSub dataset, it is statistically significant at p < 0.1.\n\n\n> (3) There are no direct/clear evidences showing that the improvement comes from more consistent formality. Human annotators are just asked to \"consider the formality of the given target context\", but we don't know how often do they make decisions based on formality consistency.\n\n> Do human annotators prioritize formality consistency over generic quality? What if they conflict?\n\nWe qualitatively confirmed that the generic qualities of the baseline and our model were sufficiently acceptable on the sampled test set. Therefore, the improvements in human evaluation were focused on how well our model generated translations that reflect the formality of the context.\n\nWe intentionally did not specify the formality class of the context to ask whether it was correctly reflected in the generated translation, recognizing that nuances in formality can vary even with a single formality class. Instead, we presented a few translation examples that were generated consistently with the formality of the context. Then we asked annotators to “Choose the sentence that is more likely to be generated if it reflects the formality of the previous context.”.\n\n> (4) The quality of regularized attention weights are better to be quantitatively evaluated than qualitatively exemplified.\n\nWe will measure the average cosine similarity between the token-level distribution of target-side context derived from the LIME algorithm of the formality classifier and the regularized context attention distribution from the NMT model. We will derive the distribution when the token of the decoding time step of the NMT model is included in the global-level formality-related token set. We will provide the results before the end of the discussion period.\n\n> Are you going to release code and data?\n\nYes\n\nWe will revise all the typos and presentation improvements in the camera-ready.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Improving translation consistency of formality w.r.t. target context is an interesting and useful research topic.\n- The proposed method is new and seems effective under their setting.", "reasons_to_reject": "- The proposed method does not use generated target context but the reference translation, which dramatically limits the practical use case. When will the preceding reference translation be available for an MT task? Computer-assisted translation is one possibility but it's a specially use case which is not discussed in the paper.\n- The evaluation protocol is not well-designed thus the results are not convincing. (1) There is no intrinsic evaluation of the formality classifier and the contribution of context tokens. It's not clear if the model receives reliable formality signals in the first place. (2) The BLEU score differences are small and the confidence intervals are not provided. We don't know if the improvements are significant. (3) There are no direct/clear evidences showing that the improvement comes from more consistent formality. Human annotators are just asked to \"consider the formality of the given target context\", but we don't know how often do they make decisions based on formality consistency. (4) The quality of regularized attention weights are better to be quantitatively evaluated than qualitatively exemplified.", "questions_to_authors": "- Do human annotators prioritize formality consistency over generic quality? What if they conflict?\n- Are you going to release code and data?", "paper_topic_and_main_contributions": "This paper proposes a new method to improve translation consistency of formality w.r.t. target context. It first classifies the formality of one preceding reference translation and measures the contribution of context tokens to the prediction. Then, it regularizes context attention to match the contribution of formality-involved tokens and re-weight salient formality tokens for target prediction during training. BLEU scores on En->Ko and En->Ja test sets are slightly improved over baselines. A small scale human evaluation shows a larger improvement.", "excitement": 2, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "1. Controlling Politeness in Neural Machine Translation via Side Constraints. NAACL 2016\n2. Multi-Task Neural Models for Translating Between Styles Within and Across Languages. COLING 2019\n3. Controlling Japanese Honorifics in English-to-Japanese Neural Machine Translation. WAT 2019\n4. Controlling neural machine translation formality with synthetic supervision. AAAI 2020\n5. Controlling Machine Translation for Multiple Attributes with Additive Interventions. EMNLP 2021", "Typos_Grammar_Style_and_Presentation_Improvements": "- L33: \"it is still difficult to control the subtle differences based on pre-defined formality class.\" This statement is inaccurate. Sentence-level formality control is reflected in subtle word-level changes."}}, {"review_id": "OdxW1e1Nfl", "reviewer": "Reviewer_7DuG", "summary": "", "questions": "A. Why wasn't the formality classifier used during training also used to obtain an evaluation of the goodness of the improved models in terms of style accuracy?\nB. Why was a custom subset of data used for the evaluation of the proposed approach, when gold test sets with annotations were available?\nC. Does the LIME method you employed guarantee that total importance would be an importance distribution that sums to 1 over the full input? If not, this would be problematic for the attention regularization loss.\nD. How was the choice of the specific attention heads used for attention regularization performed? Was it based on some heuristic, or previous literature?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the thoroughness invested in your review. We extend our gratitude for all constructive comments. To enhance readability, we have noted the comments in block quotes along with the corresponding responses below:\n\n> As pointed out by authors in the Related Work and Limitations section, BLEU is not the most appropriate metric to evaluate the quality of translations on granular phenomena like formality markers, which are the core focus of this work. While human evaluation is also conducted on a small subset of examples to assess the overall quality of translations, important details were omitted (exact question posed to annotators, agreement between the 11 annotators).\n\nWe intentionally did not specify the formality class of the context to ask whether it was correctly reflected in the generated translation, recognizing that nuances in formality can vary even with a single formality class. Instead, we presented a few translation examples that were generated consistently with the formality of the context. Then we asked annotators to “Choose the sentence that is more likely to be generated if it reflects the formality of the previous context.”. The kappa coefficient, which indicates agreement between 11 annotators, is 0.70.\n\n> Authors report scores mentioning explicitly the 13a BLEU tokenizer, despite evaluation for Japanese and Korean using Sacrebleu (https://github.com/mjpost/sacrebleu) normally involves the usage of Mecab tokenizers for these languages. Sacrebleu authors report how the default 13a tokenizer produces poor results for Japanese and Korean, so the results reported by the authors will be hardly comparable to those available in literature for the two language pairs.\n\nWe express our regret for the misstatement in the paper. We measured sacrebleu after tokenizing both detokenized generated translation and reference translation using mecab tokenizer for Korean and Japanese respectively. Therefore, we clarify that the evaluation was performed correctly and we will revise bleu signature(13a →ko-mecab, ja-mecab) in the camera-ready.\n\n\n> The field of machine translation has been slowly but steadily moving towards neural metrics such as BLEURT and COMET, which have been shown in multiple instances to align better with human judgments of translation quality (https://aclanthology.org/2022.wmt-1.2/). Further results using established neural MT metrics, ideally including some statistical testing to ensure the significance of improvements brought by the proposed approach, would ensure the robustness and applicability of the proposed method in this context.\n\nIn accordance with the reviewer’s comment, we will evaluate our models on COMET metric.\nFor AI-HUB and AMI+BSD+OpenSub datasets, the improvement of our model (WL+AttnReg) is **statistically significant at p < 0.05** compared to the Concat-level baseline. For OpenSub dataset, it is statistically significant at p < 0.1.\n\n> The proposed approach effectiveness is tested by merging together several document-level translation datasets which are not specific to formality phenomena, even though gold MT test sets with formality annotations are available both for En -> Ko (https://github.com/amazon-science/contrastive-controlled-mt/tree/main/IWSLT2023/data/train/en-ko) and En -> Ja (https://github.com/amazon-science/contrastive-controlled-mt/tree/main/CoCoA-MT/test/en-ja). A more granular evaluation of formality accuracy on those would have been important to truly grasp the impact of the improved training procedure. Since the En -> Ja corpus with gold formality annotations was already used by authors to train the formality classifier, it could have also served as a test set to evaluate the formality accuracy of the proposed MT systems, while the classifier could have served as an indicator of style correctness.\n\n> B. Why was a custom subset of data used for the evaluation of the proposed approach, when gold test sets with annotations were available?\n\nWe appreciate the reviewer’s efforts for recommending En →Ko evaluation dataset that we hadn’t noticed. For the En → Ja language pair, we used a subset of CoCoA-MT dataset as a test set, and obtained 94.8% of formality accuracy, which is described in the appendix A.1. Training Formality Classifier of the paper. For the En → Ko language pair, we further evaluated formality accuracy on the test set provided by the reviewer(https://github.com/amazon-science/contrastive-controlled-mt/tree/main/IWSLT2023/data/test/en-ko). It was 0.96. \n\n> A. Why wasn't the formality classifier used during training also used to obtain an evaluation of the goodness of the improved models in terms of style accuracy?\n\nWe will report the formality accuracy of the generated translations using the formality classifier in the camera-ready. \n\n> C. Does the LIME method you employed guarantee that total importance would be an importance distribution that sums to 1 over the full input? If not, this would be problematic for the attention regularization loss.\n\nYes, we normalized the importance distribution to sum to 1 for the input.\n\n> D. How was the choice of the specific attention heads used for attention regularization performed? Was it based on some heuristic, or previous literature?\n\nWe experimented with different numbers of heads, from 1 to 4, and determined the number of regularized heads to be 2, which resulted in the highest BLEU performance, along with qualitative evaluation of formality control.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper is well written and contextualizes well the proposed procedure\n- The idea of leveraging model rationales to complement and regularize the training procedure is interesting, especially for lexical phenomena such as formality.\n- The upweighting of specific lexical items in the global-level token weighted loss is a novel and interesting approach to further condition the training process.\n- Human evaluation results summarized by authors are promising and provide preliminary evidence of the effectiveness of the proposed method.\n- The plausibility evaluation with the attention patterns extracted by the proposed model suggests a strong alignment with human intuition, which is desirable to identify failure cases of the model.", "reasons_to_reject": "While the approaches proposed by this work are interesting, the evaluation approaches lacks rigor and makes me doubt of the actual effectiveness of the proposed method for formality-controlled MT. In particular:\n\n1. As pointed out by authors in the Related Work and Limitations section, BLEU is not the most appropriate metric to evaluate the quality of translations on granular phenomena like formality markers, which are the core focus of this work. While human evaluation is also conducted on a small subset of examples to assess the overall quality of translations, important details were omitted (exact question posed to annotators, agreement between the 11 annotators).\n\n2. Authors report scores mentioning explicitly the 13a BLEU tokenizer, despite evaluation for Japanese and Korean using Sacrebleu (https://github.com/mjpost/sacrebleu) normally involves the usage of Mecab tokenizers for these languages. Sacrebleu authors report how the default 13a tokenizer produces poor results for Japanese and Korean, so the results reported by the authors will be hardly comparable to those available in literature for the two language pairs.\n\n3. The field of machine translation has been slowly but steadily moving towards neural metrics such as BLEURT and COMET, which have been shown in multiple instances to align better with human judgments of translation quality (https://aclanthology.org/2022.wmt-1.2/). Further results using established neural MT metrics, ideally including some statistical testing to ensure the significance of improvements brought by the proposed approach, would ensure the robustness and applicability of the proposed method in this context.\n\n4. The proposed approach effectiveness is tested by merging together several document-level translation datasets which are not specific to formality phenomena, even though gold MT test sets with formality annotations are available both for En -> Ko (https://github.com/amazon-science/contrastive-controlled-mt/tree/main/IWSLT2023/data/train/en-ko) and En -> Ja (https://github.com/amazon-science/contrastive-controlled-mt/tree/main/CoCoA-MT/test/en-ja). A more granular evaluation of formality accuracy on those would have been important to truly grasp the impact of the improved training procedure. Since the En -> Ja corpus with gold formality annotations was already used by authors to train the formality classifier, it could have also served as a test set to evaluate the formality accuracy of the proposed  MT systems, while the classifier could have served as an indicator of style correctness.", "questions_to_authors": "A. Why wasn't the formality classifier used during training also used to obtain an evaluation of the goodness of the improved models in terms of style accuracy?\nB. Why was a custom subset of data used for the evaluation of the proposed approach, when gold test sets with annotations were available?\nC. Does the LIME method you employed guarantee that total importance would be an importance distribution that sums to 1 over the full input? If not, this would be problematic for the attention regularization loss.\nD. How was the choice of the specific attention heads used for attention regularization performed? Was it based on some heuristic, or previous literature?", "paper_topic_and_main_contributions": "The paper focuses on improving the quality of machine translations in the presence of explicit formality markers, which are typically quite sparse and poorly leveraged by MT models. To do so, authors propose a new training procedure leveraging a formality classifier to regularize the attention mechanism of the NMT model, effectively encouraging the model to rely more heavily on formality markers while generating translations. Moreover, authors propose a new global loss to promote the importance of lexical items that are relevant to formality-controlled translation. The effectiveness of this procedure is tested empirically over two translation directions (En -> Ja, En -> Ko), observing BLEU improvements and a large share of human preferences in favor of models trained with the proposed procedure. Finally, the plausibility of attention patterns produced after training a NMT model with the proposed mechanism is evaluated, finding a much higher similarity between human formality rationales and model attention patterns compared to regular NMT models.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "- The IWSLT 2022 (https://iwslt.org/2022/formality, https://aclanthology.org/2022.iwslt-1.10/) and IWSLT 2023 (https://iwslt.org/2022/formality, https://aclanthology.org/2023.iwslt-1.1/) formality-controlled translation shared tasks should be mentioned since they covered both the En -> Ja and En -> Ko directions, with several works being relevant to this paper's aims.\n- The RAMP methodology (https://aclanthology.org/2023.acl-short.126/) could be a relevant to this paper's related works, since it also aims to improve formality-controlled MT by promoting formality-relevant lexical items.\n- Entropy-based Attention Regularization (https://aclanthology.org/2022.findings-acl.88/) is another notable example of approaches regularizing the attention distribution of neural LMs for bias mitigation purposes.", "Typos_Grammar_Style_and_Presentation_Improvements": "53-55: It is unclear here how post-editing is related to the Yin et al. (2021) work you mention. It would be useful to clarify the usage of post-editing as a mean to obtain implicit annotations for formality, which is still time-consuming and labor-intensive.\n\n117-121: This part can be clarified by specifying why the generation of target context is effectively not evaluated during training, and why then providing gold context to the model is necessary at inference time.\n\n131-135: The claim that formality-labeled data is not available for Korean is false, see discussion above.\n\n147-151: The described removal-based feature importance approach is referred to as occlusion or blank-out. The LIME method you mention uses the coefficients of a surrogate linear models as importance measure for different inputs. The choice for this method should be justified, provided the existence of several feature attribution methods that would be fit for this purpose.\n\n178:  The link points to a landing page, but a direct reference to the dataset used should be preferable in this case. If not publicly available, this should be clearly mentioned in the footnote."}}, {"review_id": "2UidwMgDi9", "reviewer": "Reviewer_P8XY", "summary": "", "questions": "- **Question A**: Is the annotated formality dataset going to be released? I believe it would be beneficial for subsequent researchers and improve reproducibility.\n- **Question B**: In l. 533 the paper states that in the experiments gradient clipping is applied for gradient norms $> 0.1$ which seems to be a really tight clipping, how often do gradients get clipped and how big is the impact of having a looser bound e.g. $>1.0$ ?\n- **Question C**: In. l. 537 no averaging of the checkpoints is mentioned, any reason why this isn't applied as it is standard practice to improve robustness and translation quality?\n ", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> [medium]: The proposed approach barely outperforms the straightforward concat-level + tag baseline (see Table 2 for En→Ko it improves +0.08 BLEU on AI-Hub and +0.2 BLEU on AMI+BSD+OpenSub). I'm not necessarily convinced that the improvements are statistically significant. To improve this aspect, I'd suggest A) running statistical significance testing with bootstrap resampling which is already integrated into sacreBLEU as well as B) re-training with a few different seeds and reporting standard deviation for both baselines and the proposed method.\n\nA) For AI-HUB and AMI+BSD+OpenSub datasets, the improvement of our model (WL+AttnReg) is statistically significant at p < 0.05 compared to the Concat-level baseline. For OpenSub dataset, it is statistically significant at p < 0.1. \n\nB) We will report the average of the 5 run checkpoints with their standard deviation before the discussion period ends.\n\n> [medium]: Building onto the previous point -- It would be beneficial to create a dedicated formality test set (e.g. using the formality classifier) that could better highlight the achieved improvements.\n\nWe are grateful for the reviewer's suggestion. However, we performed human evaluation over creating document-level formality test set, recognizing that nuances in formality can vary even with a single formality class.\n\n> [medium]: While BLEU is still widely used, there are now better metrics to use for Machine Translation that correlate better with human judgment [1], specifically I'd recommend including chrF and COMET scores.\n\nAs recommended by the reviewer, we evaluated our models on chrF metric. We measured chrF2++ in terms of word 2-gram order. In accordance with the reviewer’s comment, we will evaluate our models on COMET metric. \n\n|Method|AIHUB|OpenSub|AMI+BSD+OpenSub|\n|------|---|---|---|\n|Sent-level|34.64|23.83|-|\n|Concat-level|36.02|28.64|24.44|\n|Tag context|35.73|28.02|22.92|\n|Concat-level + Tag|36.17|28.54|24.37|\n|Ours (WL)|36.18|29.06|24.44|\n|Ours (AttnReg)|36.21|29.10|23.80|\n|Ours (WL+AttnReg)|36.25|29.27|24.60|\n\n> [medium]: Extracting the token-level formality scores from the formality classifier using the augmented samples poses overhead during the training procedure in terms of wall-clock time and memory consumption. This aspect needs more analysis and reporting in the paper as well as compute-matched baselines.\n\nBefore training the NMT model, we inferred the token-level formality scores of the target-side context and stored them in a file. Hence, we avoided the increased memory consumption due to the additional parameters of the formality classifier and increased training time due to the inference time of the formality classifier. \n\n> [small]: no analysis is presented how sensitive the training convergence is to the hyperparameter choice \n, which value is used in the experiments, and how it was tuned (i.e. tuning ranges & distributions, number of samples, etc.)\n\nThe hyperparameter $\\alpha$ was tuned in the range [1.1, 1.25, 1.4, 1.5, 1.7, 1.75, 2.0, 2.5, 3.0]. If it is less than 2.0, the training converges within 15 epochs, but if it is greater than 2.0, it converges between 15 and 25 epochs. We will quantitative analysis of hyperparameter tuning in the camera-ready.\n\n> Question A: Is the annotated formality dataset going to be released? I believe it would be beneficial for subsequent researchers and improve reproducibility.\n\nYes, we will release the annotated formality dataset.\n\n> Question B: In l. 533 the paper states that in the experiments gradient clipping is applied for gradient norms > 0.1\n which seems to be a really tight clipping, how often do gradients get clipped and how big is the impact of having a looser bound e.g. > 1.0\n ?\n\nWe found that gradient clipping occurs in the first epoch whether the threshold of the gradient norms is 0.1 or 1.0 and there is no performance difference when it is 1.0.\n\n> Question C: In. l. 537 no averaging of the checkpoints is mentioned, any reason why this isn't applied as it is standard practice to improve robustness and translation quality?\n\nWe will report the average of the 5 run checkpoints with their standard deviation before the discussion period ends.\n\nWe will revise all the typos and presentation improvements and include recommended references in the camera-ready.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- interesting approach to use augmented context in the formality classifier to extract formality scores that are used for regularizing two attention heads\n- small scale verification of the proposed approach with human evaluation (50 samples)\n- interesting experiments on the correlation with human perception showing that the proposed regularization improves the alignment of attention weights with human perception to $0.669$ from $0.033$ (non-regularized)\n- uses sacreBLEU and provides hash for reproducible scores\n- illustrative examples and well written making it easy to follow and understand", "reasons_to_reject": "- **[medium]**: The proposed approach barely outperforms the straightforward concat-level + tag baseline (see Table 2 for En$\\rightarrow$Ko it improves $+0.08$ BLEU on AI-Hub and $+0.2$ BLEU on AMI+BSD+OpenSub). I'm not necessarily convinced that the improvements are statistically significant. To improve this aspect, I'd suggest A) running statistical significance testing with bootstrap resampling which is already integrated into sacreBLEU as well as B) re-training with a few different seeds and reporting standard deviation for both baselines and the proposed method. \n- **[medium]**: Building onto the previous point -- It would be beneficial to create a dedicated formality test set (e.g. using the formality classifier) that could better highlight the achieved improvements.\n- **[medium]**: While BLEU is still widely used, there are now better metrics to use for Machine Translation that correlate better with human judgment [1], specifically I'd recommend including chrF and COMET scores.\n- **[medium]**: Extracting the token-level formality scores from the formality classifier using the augmented samples poses overhead during the training procedure in terms of wall-clock time and memory consumption. This aspect needs more analysis and reporting in the paper as well as compute-matched baselines.\n-  **[small]**: no analysis is presented how sensitive the training convergence is to the hyperparameter choice $\\alpha$, which value is used in the experiments, and how it was tuned (i.e. tuning ranges & distributions, number of samples, etc.)", "questions_to_authors": "- **Question A**: Is the annotated formality dataset going to be released? I believe it would be beneficial for subsequent researchers and improve reproducibility.\n- **Question B**: In l. 533 the paper states that in the experiments gradient clipping is applied for gradient norms $> 0.1$ which seems to be a really tight clipping, how often do gradients get clipped and how big is the impact of having a looser bound e.g. $>1.0$ ?\n- **Question C**: In. l. 537 no averaging of the checkpoints is mentioned, any reason why this isn't applied as it is standard practice to improve robustness and translation quality?\n ", "paper_topic_and_main_contributions": "The presented paper proposes using a formality classifier to extract token-level formality importance scores using augmented samples which are then used for regularizing two attention heads in the topmost decoder layer. The converged model exhibits better formality properties in the presented experiments. ", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "- [1]: [Results of WMT22 Metrics Shared Task: Stop Using BLEU – Neural Metrics Are Better and More Robust](https://aclanthology.org/2022.wmt-1.2) (Freitag et al., WMT 2022)\n- [2]: [A Call for Clarity in Reporting BLEU Scores](https://aclanthology.org/W18-6319) (Post, WMT 2018)\n- [3]: [Jointly Learning to Align and Translate with Transformer Models](https://aclanthology.org/D19-1453) (Garg et al., EMNLP-IJCNLP 2019) also supervises attention heads for additional task", "Typos_Grammar_Style_and_Presentation_Improvements": "- Please include all figures as vector graphics e.g. Figure 1 such that they scale well and look good in print\n- The title in the paper pdf has a typo \"Leaveraging\" -> \"Leveraging\"\n- Figure 1 description: \"We then regularize the attention map from two heads of top decoder layer\" might be better to be reformulated as \"We regularize two attention heads of the topmost decoder layer\""}}]

Author rebuttal:

Meta-review: The reviewers are in consensus that the work is quite sound but the excitement level was low. The author responses went a long way to improve the soundness scores overall and addressing the reasons to reject from the reviewers.

While the overall assessment is higher than the numerical scores assigned by the reviewers, this is mainly due to the extra information that was provided during the author response, so it is very important that all of the extra experimental results and clarifications are included as part of the paper by the authors including the missing references to previous work. The additional context provided by the information in the author response is a big reason for the final evaluation of this submission in this meta-review.
Decision: Accept-Findings