Improving Multilingual Neural Machine Translation by Utilizing Semantic and Linguistic Features (2024)

Abstract: The many-to-many multilingual neural machine translation can be regarded as the process of integrating semantic features from the source sentences and linguistic features from the target sentences. To enhance zero-shot translation, models need to share knowledge across languages, which can be achieved through auxiliary tasks for learning a universal representation or cross-lingual mapping. To this end, we propose to exploit both semantic and linguistic features between multiple languages to enhance multilingual translation. On the encoder side, we introduce a disentangling learning task that aligns encoder representations by disentangling semantic and linguistic features, thus facilitating knowledge transfer while preserving complete information. On the decoder side, we leverage a linguistic encoder to integrate low-level linguistic features to assist in the target language generation. Experimental results on multilingual datasets demonstrate significant improvement in zero-shot translation compared to the baseline system, while maintaining performance in supervised translation. Further analysis validates the effectiveness of our method in leveraging both semantic and linguistic features.1

heading: 1 Introduction
text: The many-to-many multilingual neural machine translation (NMT) enables translation across multiple languages in a single model (Firat et al., 2016; Fan et al., 2021; Siddhant et al., 2020). This process involves integrating the semantic information from the source sentence and the linguistic features from the target sentence. Specifically, the encoder captures the semantic information of the source sentence and maps it to the representation space, when
*Corresponding author: Yang Feng. †This paper was done when Shuhao Gu studied at ICT/CAS. 1The code is available at https://github.com/ictnlp/ SemLing-MNMT.
the decoder integrates this representation with the inherent features of the target language to generate the target language sentences. By sharing parameters, multilingual NMT models enable knowledge transfer across languages, which significantly benefits low-resource machine translation (Aharoni et al., 2019), especially zero-shot translation. However, previous multilingual NMT models do not explicitly differentiate between semantic and linguistic features, resulting in the entanglement of knowledge and linguistics within the model. This entanglement negatively affects the performance of zero-shot translation in two main aspects. Firstly, the linguistic features of the source language interfere with the encoder’s ability to learn shared semantic information, causing the encoder to encode different languages into different representation subspaces (Kudugunta et al., 2019), which hinders knowledge transfer across multiple languages. Secondly, during the generation of the target language, the decoder is influenced by spurious correlations due to insufficient target language features, leading to off-target issues (Zhang et al., 2020).
Numerous methods have been proposed to address these challenges. Some methods focus on learning a universal cross-lingual representation space at the encoder (Arivazhagan et al., 2019; Pan et al., 2021; Pham et al., 2019; Gu and Feng, 2022), while others aim to facilitate target language generation at the decoder (Yang et al., 2021a; Gao et al., 2023). For example, Pan et al. (2021) employ contrastive learning to minimize the distance between parallel sentence pairs and maximize the distance between irrelevant sentence pairs. Gu and Feng (2022) leverage optimal transport theory to learn a universal representation and introduce an agreement-based training approach to make consistent predictions. Gao et al. (2023) promote consistent cross-lingual representation by cross-lingual consistency regularization. However, these methods primarily concentrate on learning
1 10410
language-agnostic semantic features while neglecting the language-specific linguistic features. Such incompleteness of information can lead to less accurate semantic mapping and alignment degradation, which can harm the supervised and zero-shot translation performance.
In this paper, we propose to utilize both semantic and linguistic features for multilingual NMT. Our insight is to learn a semantic representation space non-destructively and utilize linguistic features to guide target language generation, thereby improving the performance of zero-shot translation and maintaining supervised translation performance lossless. Specifically, at the encoder, we introduce a disentangler and disentangling learning task to facilitate harmless cross-lingual semantic alignment. At the decoder, we utilize a linguistic encoder to integrate low-level linguistic features from lower layers and high-level semantic features from higher layers. This fusion provides implicit guidance for the target language generation.
Experiments on multilingual datasets show that our method brings an average of 0.18+ BLEU in the supervised translation direction and an average of 3.74+ BLEU in the zero-shot translation direction compared with the baseline system. Furthermore, the analysis demonstrates that our method can alleviate the problems caused by the entanglement of semantic and linguistic features, obtain a better semantic representation, and reduce the off-target rate in zero-shot translation.

heading: 2 Background
text: In this section, we will give a brief introduction to the Transformer (Vaswani et al., 2017) model for NMT and many-to-many multilingual NMT.

heading: 2.1 Transformer for NMT
text: Define the set D of parallel corpus with input sentence xui and reference y v i , where u and v denote language type and i denotes the sentence id, which is the same for a sentence pair. The model first maps the source sentence xui from the token representation to the token vector representation embed(xui ). After that, the encoder encodes the token vector sequence into the hidden state representation h(xui ). Similarly, the model maps the predicted k − 1 target words into a sequence of token vectors, and the decoder decodes the kth target word based on the sequence of hidden states h(xui ) from the encoder and the predicted sequence of k − 1 target words. The model repeats this process until the ⟨eos⟩ token is predicted. The model is optimized by minimizing the cross-entropy (CE) loss between the predicted sentence and the reference as follows:
Lce = − ∑
xui ,y v i ∈D
log(Pθ(y v i |xui )) (1)
where θ denotes the model parameters.

heading: 2.2 Many-to-Many Multilingual NMT
text: The many-to-many multilingual NMT model supports translation between multiple languages. For-
2
mally, we define the set of languages L = {L1, ..., LM}, where M is the number of languages. We prepend a language token at the beginning of the source sentence and the target sentence to indicate the language type respectively. For example, the following sentence pair translated from English to French: "Hello world! → Bonjour le monde!" is transformed into "⟨en⟩ Hello world! → ⟨fr⟩ Bonjour le monde!".

heading: 3 Method
text: The core idea of our approach is to improve multilingual NMT by exploiting semantic and linguistic features at the same time. To achieve this, we disentangle semantic and linguistic features for the encoder and utilize low-level linguistic features for the decoder. Specifically, we propose a disentangler at the encoder to facilitate lossless semantic alignment. Meanwhile, we introduce a linguistic encoder at the decoder to fuse low-level features to guide target language generation. Figure 1 illustrates the framework of our approach.

heading: 3.1 Disentangling Semantic and Linguistic Features
text: Empirically, sentences possess both semantic and linguistic properties, which can be expressed as "sentence = semantics + linguistics". Semantic features encompass the meanings of language units that are shared across languages, while linguistic features comprise the rules for constructing sentences, such as lexical labels and morphological information. By disentangling linguistic features from sentences, we can obtain a universal semantic representation shared by different languages. Referring to the design of Tiyajamorn et al. (2021) on multilingual language understanding tasks, we
extract semantic and linguistic features separately and implement interactive decomposition using reconstruction constraints.
The structure of our disentangler is shown in Figure 2. The disentangler consists of two feed-forward networks (FFN), a semantic FFN and a language FFN, respectively. For the encoder representations, the former extracts languageagnostic semantic information, and the latter extracts language-specific linguistic features. Then, the two outputs are summed to reconstruct the original encoder representations. The extracted semantic features are fed into the decoder.
We define disentangling loss Ldet to optimize disentangler by multi-task learning:
Ldet = Lsem + Llang + λ1Lrecons (2)
where Lsem assists the semantic FFN to extract semantic information, Llang assists the language FFN to extract linguistic features, Lrecons is the reconstruction constraint, and λ1 is the hyperparameter to balance the losses.
Semantic Learning To learn a universal representation space across multiple languages, we explicitly minimize the distance of parallel sentence pairs and maximize the distance of irrelevant sentence pairs. For the multilingual parallel dataset D, we make the parallel sentence pair (xui ,y v i ) ∈ D a positive example and randomly choose a sentence ywj from language Lw as a negative example (xui ,y w j ), where Lu can be the same as Lw. We optimize by minimizing the following loss function:
Lsem =Exui ,yvi ∈D(1− sim +(R(xui ),R(yvi ))+
λ2Exui ,ywj ∈D(1 + sim −(R(xui ),R(ywj )))
(3) where sim(·) calculates the similarity of different sentences, and we use cosine similarity to evaluate the similarity. + and − denote positive and negative samples, respectively. R(·) calculates the average-pooled hidden state representation. λ2 is the hyperparameter that balances positive and negative samples. To simplify the implementation in the training, we sample the positive and negative samples within every training batch.
Language Learning Empirically, different languages have different linguistic features, such as lexical and syntactic patterns. Compared with sentences in different languages, the linguistic features are more similar between sentences in the same
3
language. To extract linguistic features of multiple languages, we design the training objective similar to semantic learning. We minimize the distance of sentences in the same language and maximize the distance of sentences in different languages. We make the same language sentence pair (xui ,y u j ) ∈ D a positive example and choose a random sentence yvk from any other language Lk as a negative example (xui ,y v k). We optimize by minimizing the following loss function:
Llang =Exui ,yuj ∈D(1− sim +(R(xui ),R(yuj ))+
λ2Exui ,yvk∈D(1 + sim −(R(xui ),R(yvk)))
(4) Where sim(·), +, − and R(·) are defined the same as semantic learning. λ2 is a shared hyperparameter. We sample the positive and negative samples within every training batch.
Reconstruction Constraint To make the semantic and linguistic features fully interact and learn a more reasonable semantic representation, we design the reconstruction constraint to utilize the linguistic features. We input the encoded hidden state h(xui ) from the encoder into the disentangler, extract the semantic information hsem(xui ) and the language features hlang(xui ) respectively, and reconstruct the original encoder representation by summation. We perform optimization by minimizing the following loss function:
Lrecons =Exui ∈D( 1 d ||h(xui ))−
hsem(x u i )− hlang(xui )||2)
(5)
Where d denotes the average sequence length.

heading: 3.2 Integrating Linguistic Features
text: In multilingual NMT models, decoders need to map the semantic representation to the target language representation. For zero-shot translation, since the model has not seen the translation direction during training, it is difficult to model the mapping relationships well, leading to the off-target phenomenon (Gu et al., 2019). Previous works show that the off-target rate is strongly correlated with the zero-shot translation performance (Zhang et al., 2020; Wu et al., 2021; Wang et al., 2022). Therefore, we hope to reduce the off-target rate by fusing additional information to guide the decoder to generate the target language.
Zhang et al. (2021) show that the lower encoder layers will focus more on linguistic information,
which is crucial to distinguishing different languages. Therefore, we introduce a linguistic encoder with two encoder layers, which has the same input as the decoder. We utilize a fusion layer to integrate the low-level representations containing more linguistic features and the high-level representations containing more semantic features. We denote the linguistic encoder output as hlingEnc and the decoder output as hdec. We concatenate these features in the feature dimension and utilize a two-layer FFN for fusion:
h = W1(ReLU(W2([hdec;hlingEnc]))) (6)
Where W1 ∈ Rd×dh and W2 ∈ R2dh×d are two feed-forward layers.

heading: 3.3 Joint Training Strategy
text: We optimize the final loss function using a joint training strategy, including cross-entropy loss and disentangling loss:
L = Lce + λdLdet (7)
where λ is the hyperparameter that controls the contribution of Ldet. Since Lce is calculated at the token level and Ldet is calculated at the sentence level, we multiply Ldet by the average sequence length d.

heading: 4 Experiment
text: We evaluate the performance of our model on the multilingual benchmarks, including supervised and zero-shot translation performance.

heading: 4.1 Dataset Description
text: We conduct experiments on the IWSLT2017, OPUS-7, and PC-6 datasets. The brief information about these datasets is shown in Appendix A.
IWSLT2017 The IWSLT2017 benchmark (Cettolo et al., 2017) contains five languages: German (De), English (En), Dutch (Nl), Romanian (Ro), and Italian (It). We experiment on the Englishcentric dataset consisting of eight supervised translation directions and twelve zero-shot translation directions. The training set contains 0.22M∼0.26M parallel sentence pairs per language direction. We use the official validation set and test set.
OPUS-7 The OPUS-7 dataset is a subset of the OPUS-100 dataset (Pan et al., 2021). The OPUS-7 dataset is English-centric and contains seven languages: Arabic (Ar), German (De), English (En),
4
French (Fr), Dutch (Nl), Russian (Ru), and Chinese (Zh). This dataset contains twelve supervised translation directions and thirty zero-shot translation directions. The training set contains 1M parallel sentence pairs per language direction. We use the official validation set and test set.
PC-6 We construct the PC-6 dataset following (Gu and Feng, 2022). PC-6 is a dataset of six languages extracted from PC32 (Pan et al., 2021), including Czech (Cs), Kazakh (Kk), Romanian (Ro), Russian (Ru), and Turkish (Tr). The PC-6 dataset contains ten supervised translation directions and twenty zero-shot translation directions. The amount of data in each language direction varies in size from 0.12M to 1.84M sentence pairs. For the supervised translation, we use the official WMT2016∼2019 validation set and test set. For the zero-shot translation, we extract from WikiMatrix (Schwenk et al., 2021) to obtain the test set.
We directly apply the Unigram Model algorithm to preprocess the original multilingual corpus using Sentencepiece toolkit (Kudo and Richardson, 2018). We build a shared dictionary across multiple languages, which contains 32K tokens.

heading: 4.2 Model Configuration
text: We use the base Transformer configuration (Vaswani et al., 2017) for both the encoder and decoder. For the disentangler and the fusion layer, we set ffn_dim = 2048. For the linguistic encoder, we apply two encoder layers of the base Transformer configuration. We use the Adam optimizer and set β = (0.9, 0.98). We set warmup = 4000 and use the inverse square root learning scheduler with the learning rate set to 7e−4. We apply label-smoothed cross-entropy loss with a smoothing rate of 0.1. We set dropout = 0.3 for the IWSLT2017 dataset and dropout = 0.1 for the OPUS-7 and PC-6. We set λ = 0.05, λ1 = 0.2 and λ2 = 0.2 for our system. We train the models on 4 RTX3090 GPUs.
In the inference stage, we use beam search with beam size 5 and set the length penalty to 1. We report the BLEU (Papineni et al., 2002), ChrF (Popović, 2015) and COMET (Rei et al., 2022)2 scores for comprehensive evaluation. The BLEU and ChrF scores are computed using the SacreBLEU toolkit (Post, 2018). We select the best-performing three checkpoints on the valida-
2https://huggingface.co/Unbabel/ wmt22-comet-da
tion sets from the last six checkpoints and report the average scores on the test sets.

heading: 4.3 Contrast Systems
text: We compare our model with the following multilingual NMT systems. We prepend a language token at the beginning of the sentences for NMT systems.
m-Transformer (Johnson et al., 2017) This method utilizes cross-entropy loss to train a Transformer model on multilingual datasets.
Pivot Translation (Cheng et al., 2017) This method transforms zero-shot translation into English-centric two-stage translation. The translation model is m-Transformer.
Contrastive Learning (mRASP2 w/o AA) (Pan et al., 2021) This method introduces contrastive learning to learn a universal representation space for the encoder. For a fair comparison, we do not use the aligned augmentation (AA).
Target Language Prediction (TLP) (Yang et al., 2021a) This method leverages a target language prediction task to help the decoder retain information about the target language.
Denoising Training (DT) (Wang et al., 2021) This method introduces a denoising auto-encoder task during training. Specifically, they utilize all English sentences to construct the denoising corpus via text infilling operation.
CrossConST (Gao et al., 2023) This approach adds KL divergence loss to the Softmax layer to promote consistent cross-lingual representation:
Lkl = γKL(Pθ(x,y)||Pθ(y,y)) (8)
Where γ is the hyperparameter controlling the proportion of Lkl. We set γ = 0.25 on IWSLT2017, γ = 0.06 on OPUS-7 and γ = 0.1 on PC-6. We jointly train Lce and Lkl.
Alpaca (Taori et al., 2023) Alpaca is fine-tuned by Stanford from LLaMA-7B (Touvron et al., 2023) with 52k instruction data. We reproduce Alpaca using the LoRA (Hu et al., 2022) setting.
ChatGPT We use the GPT-3.5-Turbo API3 for translation experiments. Our translation prompt for Alpaca and ChatGPT is reported in Appendix B.
3The API version is GPT-3.5-Turbo-0613.
5

heading: 4.4 Main Results
text: The results of the main experiment are shown in Table 1. For OPUS-7 and PC-6, we report the average BLEU scores for the same target language for convenience, and detailed results for the zeroshot translation directions are shown in Appendix C. The ChrF and COMET results are presented in Appendix D.
The results show that our system significantly improves zero-shot translation performance and maintains supervised translation performance. Specifically, in the supervised translation direction, the performance of our system is comparable to that of m-Transformer, which suggests that our disentangling approach mitigates alignment degradation and maintains the supervised translation perfor-
mance. In the zero-shot translation direction, our system significantly outperforms all the contrast models, indicating that our additional modeling of linguistic features can assist the model in learning better semantic representation and language generation. Moreover, our method even outperforms the Pivot system on IWSLT2017 and PC-6, which we believe is probably because Pivot accumulates too many errors on these two datasets.
The results of large language models (LLMs) indicate that our task-specific model still has advantages in the era of LLMs. Our model significantly outperforms Alpaca-7B in supervised and zero-shot translation. Compared to the ChatGPT, our model achieves more than 90% performance with just about 0.1% of the parameters. Furthermore, our model employs significantly less data
6
and inference is much faster.

heading: 5 Analysis
text: In this section, we will analyze what contributes to the performance. We first analyze the effects of each module via an ablation study. Then, we evaluate the off-target rate and visualize the encoder and decoder sentence representations, demonstrating that our approach learns universal representations and generates target language more accurately. Finally, we examine some translation cases to show the usefulness of our approach.

heading: 5.1 Ablation Study
text: To further analyze the effectiveness of our approach, we conducted experiments on different variants of our system on the IWSLT2017. We briefly report the experimental results in Table 2.
Compared with the m-Transformer, ③ can significantly improve the zero-shot translation. The gap in zero-shot translation performance remains even when scaling up the m-Transformer (mTransformer†). This indicates that it is effective to leverage the linguistic encoder to learn low-level linguistic features and integrate different levels of representations. We present detailed results and analysis of the scaling experiments in Appendix E for parameter comparisons.
Compared with ①, ② provides a significant improvement in zero-shot translation while maintaining supervised translation performance. This indicates that the disentangling learning task is effective. Disentangling learning can significantly improve the performance of zero-shot translation while mitigating the performance loss in supervised translation caused by semantic alignment. The analysis between ④ and ⑤ similarly supports this view.
Compared with ② and ③, ⑤ shows that the dis-
entangler and the linguistic encoder can collaborate to improve the zero-shot translation performance.

heading: 5.2 Off-target Issue
text: The many-to-many multilingual NMT models face the off-target issue (Gu et al., 2019) in zero-shot translation, which seriously impacts the zero-shot translation performance. The off-target issue refers to the problem that the model simply copies the source sentences or generates incorrect target sentences. Related work shows that multilingual NMT models have difficulty establishing mapping relations for the zero-shot translation directions, which may be overfitted to other language directions. To test whether our approach can alleviate the offtarget issue, we use langid toolkit (Lui and Baldwin, 2012) to identify the target language and define the off-target rate as the proportion of off-target sentences. We conduct experiments on the contrast systems and our method. Table 3 shows that our method can effectively reduce the off-target rate compared to other systems, proving that our method can provide an effective guide for target language generation.
7
Furthermore, we test the performance of intarget sentences to demonstrate that our approach improves the in-target translation quality. Considering that there is no comparability between the BLEU scores of different test sets, we extract the part where all model outputs are in-target for testing. Table 4 presents the results. Compared with Table 1, the performance gap between different models in the in-target part is smaller, which indicates the importance of mitigating the off-target issue. Meanwhile, the zero-shot performance of our models in the in-target part still significantly outperforms m-Transformer, which implies that our approach achieves better knowledge transfer and improves the zero-shot translation quality.

heading: 5.3 Visualization
text: We visualize the sentence representations of the encoder and decoder to verify, respectively, that our disentangler aids semantic alignment and our linguistic encoder achieves more accurate target language generation. We conduct experiments on a subset of IWSLT2017 test sets. We use tSNE (Van der Maaten and Hinton, 2008) to reduce the 512-dim representations to 2-dim.
Figures 3(a) - 3(d): The m-Transformer cannot
align different languages well, which is caused by the entanglement of semantic and linguistic features at the representation level. For semantic features, our semantic FFN pulls the semantic representation closer, achieving a similar effect as mRASP2 w/o AA. For linguistic features, our language FFN pushs the different language representations apart, which proves that the language FFN can extract linguistic features.
Figures 3(e) - 3(h): The representation after decoder embedding is language-mixed, while the linguistic encoder can encode different languages into different subspaces, even though no explicit task is designed to guide this. By introducing low-level linguistic features, our system can better distinguish the target language and thus generate the target sentences more accurately.

heading: 5.4 Case Study
text: To further demonstrate the usefulness of our system, we analyze some translation cases and compare the outputs of our model with that of the mTransformer in Figure 4. While the m-Transformer translates whole or partial sentences to the wrong language, our model can translate accurately, proving the advantages of our method.
8

heading: 6 Related Work

heading: 6.1 Zero-shot Machine Translation
text: To improve the zero-shot translation capability of multilingual NMT, researchers have proposed a series of auxiliary training objectives. These works can be divided into two categories. The first category is designed directly at the encoder so that the encoder outputs language-agnostic representations (Pham et al., 2019; Wei et al., 2021). These works are motivated by the hypothesis that a universal semantic representation can realize better knowledge transfer. Pan et al. (2021) propose a contrastive learning task to encourage the model to learn a universal representation space. Gu and Feng (2022) use optimal transport theory to close the representation of sentence pairs. However, these works focus only on semantic features and neglect linguistic features, which leads to performance degradation in the supervised translation direction. By integrating both semantic and linguistic features, our approach significantly improves the performance of zero-shot translation while effectively maintaining the supervised direction performance.
The second category is designed at the decoder to provide explicit guidance for target language generation. These works are motivated to improve multilingual representation. Yang et al. (2021a) propose the target language prediction task at the sequence level and the target gradient regularization at the gradient level. Gao et al. (2023) introduce the KL regularization term for the Softmax layer to promote cross-lingual consistent representation. In contrast to these approaches, we do not explicitly design auxiliary tasks, but utilize a lin-
guistic encoder to fuse low-level representations containing more linguistic features and high-level representations containing more semantic features.

heading: 6.2 Disentangling for Cross-lingual Alignment
text: Cross-lingual alignment is a classic problem in multilingual tasks. Although the "pretrainingfinetuning" paradigm can improve the cross-lingual ability of the model (Liu et al., 2020; Xue et al., 2021), the entangling of semantics and linguistics can impair the model performance. To address this problem, many researchers have improved it from the perspective of disentangling. Tiyajamorn et al. (2021) design auto-encoder to disentangle semantic and linguistic representation for natural language understanding tasks. Yang et al. (2021b) remove language identity information from pre-trained multilingual representations by singular value decomposition (SVD) and orthogonal projection. Zhao et al. (2021) explore three settings for removing language identity signals from multilingual representations, including vector space re-alignment, vector space normalization and input normalization. Wu et al. (2022) use SVD to extract linguistic features, use language-agnostic semantic information for task-specific training, and re-entangle semantic and linguistic features in the generation phase. We refer to this disentangling idea and utilize disentangling to achieve lossless semantic alignment for multilingual NMT.

heading: 7 Conclusion
text: In this paper, we propose to exploit both semantic and linguistic features to improve multilingual NMT. We design a disentangler and disentangling learning task at the encoder side to achieve lossless semantic alignment. Meanwhile, we introduce a linguistic encoder at the decoder side to fuse the low-level representation containing more linguistic features and the high-level representation containing more semantic features to improve the target language generation. Experiments on multilingual datasets indicate that our approach can significantly improve the performance of zero-shot translation while effectively maintaining the supervised translation performance. The analysis further demonstrates that utilizing both semantic and linguistic features can help the model learn semantic representation non-destructively and achieve more accurate target language generation.
9
Limitations
Although our approach can significantly improve zero-shot translation performance while maintaining supervised translation performance, the performance improvement in the supervised direction is limited. On the one hand, the datasets of our experiments contain only several languages, and the baseline model can already model the supervised translation well. On the other hand, our approach to modeling linguistic features is still preliminary and implicit. Future work could consider using larger multilingual datasets as well as designing better methods to model linguistic features.

heading: Acknowledgments
text: We thank all the anonymous reviewers for their insightful and valuable comments. This work was supported by a grant from the National Natural Science Foundation of China (No. 62376260).

heading: A Statistics of the Datasets
text: Table 5 summarizes information about the multilingual training datasets.

heading: B Translation Prompt for Alpaca and ChatGPT
text: For Alpaca, we follow the official template. We set Instruction to "Translate input from <src> to <tgt>." and set Input to the source sentence. For ChatGPT, we use the following format to structure the inputs: "Translate the sentences from <src> to <tgt>: <input>.". <src> denotes the source language, <tgt> denotes the target language and <input> denotes the source sentence.

heading: C Detailed Results on the OPUS-7 and PC-6 Test Sets
text: The detailed results on the OPUS-7 and PC-6 test sets are shown in Table 6 and Table 7 respectively.

heading: D ChrF and COMET Metrics
text: For the ChrF and COMET metrics, we evaluate the m-Transformer, mRASP2 w/o AA and our model. The results are presented in Table 8 and Table 9 respectively.

heading: E Scaling Experiments on the IWSLT2017 Test Set
text: Considering that our approach introduces additional parameters, we scale the model sizes of the m-Transformer and mRASP2 w/o AA for a fair comparison. As shown in Table 10, our model outperforms these variant models. We note a significant performance degradation of the m-Transformer when the model size reaches 100.4M. We attribute this to the limited size of the IWSLT2017 training set, thus the model performance largely depends on the method, rather than the model size. This further demonstrates the effectiveness of our method.
12
13
14

heading: Future Work
text: Future work could consider using larger multilingual datasets as well as designing better methods to model linguistic features