# A Mongolian-Chinese Neural Machine Translation Method Based on Semantic-Context Data Augmentation
## 3 Mongolian-Chinese Translation with Semantic-Context Data Augmentation Technique  
This paper combines the architecture approach of the Transformer model with semantic-context data augmentation, and the model is named Sem-NMT. The specific structure is shown in Figure 2.  
The Transformer model consists of multiple encoders and decoders. When the Transformer model is working, the encoder encodes the input sentence and transfers the encoded information to the decoder. The decoder translates Mongolian into Chinese based on encoding information. Each encoder in this model establishes communication with all decoders, allowing each decoder to access the necessary information. The internal structure of each encoder layer on the encoder side of the model remains identical, with the self-attention  
Figure 2: Semanticcontext data enhancement structure figure. layer being the most crucial component. During translation, the self-attention layer serves to guide the model on which words to prioritize, calculating the attention value for each word. The processing of the self-attention layer can better extract the features of the data. The calculation formula of the self-attention mechanism is as follows:  
\[Attention(Q,K,V)=softmax(\frac{QK^{\top}}{\sqrt{d_{k}}})V \tag{9}\]  
where \(Q\), \(K\), and \(V\) are obtained by multiplying three different parameter matrices, \(W^{Q}\), \(W^{K}\), \(W^{V}\), with the embedding vectors of the words. These three vectors describe the information contained in the vocabulary from different perspectives. In this paper, \(d_{k}=64\) and \(\sqrt{d_{k}}=8\). Both the encoder and decoder sides of this model employ a multi-head attention mechanism. After embedding the words, a word can be input into different self-attention layers, producing multiple distinct self-attention output values. Subsequently, these values are concatenated, and a parameter matrix of appropriate dimensions is used to reduce the dimension of the concatenated self-attention output. Finally, they are fed into a fully connected layer for further computation. Through this operation, we can obtain semantically rich vectors, the structure of which is shown in Figure 3.  
However, the multi-head self-attention layer alone cannot capture positional information within Mongolian sentences. This article employs positional encoding to capture structural information within sentences. The formula for positional encoding is as follows:  
\[PE(POS,2i)=sin(\frac{pos}{1000^{\frac{2i}{n_{model}}}}) \tag{10}\]  
\[PE(POS,2i+1)=cos(\frac{pos}{1000^{\frac{2i}{n_{model}}}}) \tag{11}\]  
In the formula, \(d_{model}\) is the dimension of the embedding vector for Mongolian words, and \(i\) is the component index of the embedding vector for Mongolian words, with a value range of 1 to \(d_{model}\). \(pos\) is the positional index of words in Mongolian. Formulas (10) and (11), respectively, encode the parity component index in the Mongolian word embedding vector. The utilization of trigonometric functions is attributed to their unique properties, enabling the transformation of Mongolian word encoding from the right side of the sentence to  
Figure 3: Multi-headed self-attention mechanism. reflect linear changes in the word encoding on the left side of the sentence. This coding method can capture the absolute position information and relative position information of Mongolian words. Figure 4 illustrates the utilization of positional encoding for Mongolian words. This involves acquiring the positional encoding of Mongolian words, adding it to the corresponding term of the word embedding vector, and subsequently inputting it into the translation model for training. Mongolian is a low-resource language with relatively few parallel corpora. Nowadays, machine translation requires a large number of corpora to train an excellent translation model. For Mongolian, we need to use DA methods to improve the translation performance of the model.