# A Mongolian-Chinese Neural Machine Translation Method Based on Semantic-Context Data Augmentation
## 2 Semantic-Context Data Augmentation  
Semantic-context DA is a technique that merges source and target data for DA. It can simultaneously utilize the semantic information of the source and target during training to generate diverse semantic information-rich vectors. These vectors can be fused with the vector generated by the encoder and then translated to improve translation performance. This method augments the original sentences without requiring additional training samples. A shared semantic space exists between the source and target languages of NMT, established by a semantic encoder, which contains many vector variants with the same semantics as the original training sample. This method can alleviate the problems of data sparsity and semantic inconsistency in Mongolian-Chinese NMT. Therefore, for data augmentation within the semantic space, this paper utilizes Transformer as the primary network and the pre-trained model XLM-R (Cross-lingual Language Model with RoBERTa architecture) [23] as an additional semantic encoder. The XLM-R model is based on the RoBERTa (Robustly optimized BERT approach) architecture, but in essence, the XLM-R model is a deep learning model based on Transformer's self-attention mechanism. By pre-training the model on large-scale text data, the semantic representation and features learned by the XLM-R model can capture the semantic information and language structure in the text. This general semantic representation is beneficial for various downstream natural language-processing tasks, because it provides a general representation that can be applied to different tasks and fields. A specific structure diagram of XLM-R is shown in Figure 1. XLM-R is used as a semantic encoder, which can dynamically allocate attention weights according to different positions and semantic relationships in the input sequence. At the same time, the model can better understand the semantic content of the input text, and take into account the relationships between and importance of words in the text when encoding, resulting in more accurate and rich semantic coding. The pre-trained semantic encoder is proficient in capturing both grammar and semantic information within the text. Within the semantic encoder, a forward function \(\phi(. :\theta^{\prime})\) parameterized by \(\theta^{\prime}\) is defined, enabling the mapping of discrete sentences into continuous vector representations [19]. This semantic encoder aims to convert the source sentence \(x\) and the target sentence \(y\) into real-valued vectors, \(r_{x}=\phi(x:\theta^{\prime})\) and \(r_{y}=\phi(y:\theta^{\prime})\), respectively. By mapping the semantic information of Mongolian and Chinese into the same vector space, the encoder achieves semantic representation and comprehension of these two languages. For the generation of semantic regions, the following method is used. Assuming \((x^{(i)},y^{(i)})\) and \((x^{(i)},y^{(i)})\) are two randomly selected training instances from the training corpus, the adjacent semantic region \(v(r_{x^{(i)}},r_{y^{(i)}})\) is defined as the union of two closed balls centered on \(r_{x^{(i)}}\) and \(r_{y^{(i)}}\), respectively. The radius of the two balls is \(d=||r_{x^{(i)}}-r_{y^{(i)}}||_{2}\), which is the Euclidean distance between \(r_{x^{(i)}}\) and \(r_{y^{(i)}}\). \(d\) is also considered a relaxation variable for determining semantic equivalence, meaning that a vector whose distance from \(r_{x^{(i)}}\) or \(r_{y^{(i)}}\) does not exceed \(d\) is semantically equivalent to \(r_{x^{(i)}}\) or \(r_{y^{(i)}}\). Figure 1: XLM-R structure diagram. Using the method of comparative learning to optimize the semantic space range, each vector generates a semantically similar region in a continuous space, that is, an adjacent semantic region that contains variants of the same semantic representation as the original sentence.