# A Mongolian-Chinese Neural Machine Translation Method Based on Semantic-Context Data Augmentation
## 2 Semantic-Context Data Augmentation
The adjacent semantic region \(v(r_{x},r_{y})\) in the semantic space contains multiple semantic variants of the described sentence pairs \((x,y)\), and contains many vector representations of similar semantics in Mongolian and Chinese. We construct negative samples in comparative learning by applying convex interpolation between the current instance and other instances in the same training batch for instance comparison. The negative sample and design are as follows:  
\[r_{x^{\prime}(i)}=r_{x^{(i)}}+\lambda_{x}\big{(}r_{x^{(i)}}-r_{x^{(i)}}\big{)},\lambda_{x}\in\bigg{(}\frac{d}{d^{{}^{\prime}}_{x}},1\bigg{]} \tag{1}\]  
\[r_{y^{\prime}(i)}=r_{y^{(i)}}+\lambda_{y}\Big{(}r_{y^{(i)}}-r_{y^{(i)}}\Big{)},\lambda_{y}\in\bigg{(}\frac{d}{d^{{}^{\prime}}_{y}},1\bigg{]} \tag{2}\]  
Here, \(d^{{}^{\prime}}_{x}=||r_{x^{(i)}}-r_{x^{(i)}}||_{2}\), \(d^{{}^{\prime}}_{y}=||r_{y^{(i)}}-r_{y^{(i)}}||_{2}\) in the two formulas above, where \(d^{{}^{\prime}}_{x}\) and \(d^{{}^{\prime}}_{y}\) are numbers larger than \(d\), or \(r^{{}^{\prime}}_{x^{(i)}}=r_{x^{(i)}}\) and \(r_{y^{(i)}}=r_{y^{(i)}}\). Following this design, through the interpolation of multiple instances within the same training batch, the adjacent semantic region of the \(i\)-th training instance can be comprehensively established. We dynamically adjust the values of \(\lambda_{x}\) (or \(\lambda_{y}\)) during the training process, as referenced in [24]. We optimize this vector space using contrastive learning methods and sample the source language vectors within the vector space. The sampling method employs normal distribution sampling to generate random semantic representations, which are subsequently fused with the original training samples to enhance the contextual semantic data of the training set. When conducting comparative learning, we generate a semantic space with more similar semantic information by optimizing the loss of the difference between the semantic similarity between the negative sample and the original sentence vector and the semantic similarity between the positive sample and the original sentence vector. \(s(x,y)\) represents the similarity function between the positive and negative samples, thereby deriving the loss function for contrastive learning, as depicted in Formula (3):  
\[Loss_{CL}(x^{(i)})=max(s(r_{x^{\prime}(i)},r_{x^{(i)}})-s(r_{y^{(i)}},r_{x^{(i )}})+\eta,0) \tag{3}\]  
Similarly, we also expect \(r_{y^{i}}\) and \(r_{x^{i}}\) to be as similar as possible, but different from all other instances in the same training batch. Therefore, \(Loss_{CL}(y^{(i)})\), as shown in Formula (4):  
\[Loss_{CL}(y^{(i)})=max(s(r_{y^{(i)}},r_{y^{(i)}})-s(r_{x^{(i)}},r_{y^{(i)}})+ \eta,0) \tag{4}\]  
The comparative loss of training batches can be expressed as  
\[Loss_{CL}=Loss_{x^{(i)}}+Loss_{y^{(i)}} \tag{5}\]  
The loss function is established based on the Max-Margin Loss Function, where the constant \(\eta\) represents the margin. Specifically, the aim is to enhance the similarity score of positive samples while diminishing the similarity score of negative samples. In essence, \(\eta\) represents the maximum allowable difference between the two scores; any further increase in the difference would not yield any reward, thereby ensuring the model's generalization ability. Moreover, if the similarity score of negative examples is less than the constant \(\eta\), the loss function is adjusted to bring their distance closer to \(\eta\). The loss function for the translation model is the cross-entropy loss function, depicted in Formula (6):  
\[Loss_{trans}=CrossEntropy(y,\hat{y}) \tag{6}\]When integrating semantic-context DA models with translation models, the fusion loss function is  
\[TotalLoss=w_{1}*Loss_{CL}+w_{2}*Loss_{trans} \tag{7}\]  
where \(w1\) and \(w2\) are the weights assigned to \(Loss_{CL}\) and \(Loss_{trans}\), respectively. These weights can be arbitrary positive numbers, and their sum does not have to equal 1. Their values depend on the relative importance you assign to each loss function during training.