## 3 System Design
### Multi Language Neural Machine Translation Model Based on BART  
Machine translation generally means the utilization of computers to achieve automatic translation from one language to another [34]. For the input source language \(S\), the machine translation system can convert it to the target language \(T\). The entire language conversion process can be modeled as the probability of obtaining the target language under the condition of inputting the source language \(P(Y)\) as follows:  
\[P(Y)=P(T|S) \tag{1}\]  
Due to the contextual semantic links between the words in the target sentence, the above probability Equation (1) can be further decomposed into the individual conditional probabilistic factors of the words in the target utterance [35; 36]:  
\[P(Y)=P(T|S)=\prod_{i=1}^{N}p(T_{i}|S;\theta) \tag{2}\]  
In Equation (2), \(N\) is the length of the target sentence and \(\theta\) is a parameter of the machine translation model. BART [29] is a pre-trained language model based on the overall structure of the Transformer, which has demonstrated significant improvements in natural language generation tasks. In this study, an NMT model is built by pre-training and fine-tuning parameters using a BART combination adaptor [37; 38; 39]. Two BART models, pre-trained as the encoder and decoder, are employed in tandem. The encoder adaptor is installed immediately following the encoder BART layer, and similarly, the decoder adaptor is located right after the decoder BART layer. Only the two adaptors are fine-tuned during training, which not only saves computational resources but also drastically reduces the model training time. The model structure for NMT is presented in Figure 3, where \(M\) and \(N\) represent the same structure for \(M\) and \(N\) stacks, respectively. Specifically, two pre-trained BART models, \(X_{BART}\) and \(Y_{BART}\), are used as the source-side language encoder and the target-side language decoder, respectively. The two adaptors are fine-tuned to target the sequence generation framework on a parallel Chinese-other lan  
Figure 3: Multilingual Neural Machine Translation Model Architecture. guage corpus. The loss function of the whole process is modeled as the following equation:  
\[L(Y|X;\theta_{AE},\theta_{AD})=-\sum_{i=1}^{N}logP(y_{i}|y_{i-1},x;\theta_{AE}, \theta_{AD}) \tag{3}\]  
\(L\) in Equation (3) is the loss function for the entire model, \(N\) is the length of the target sequence, \(x\) is the input language of the source, \(\theta_{AE}\) and \(\theta_{AD}\) are the parameters of the encoding adaptor and decoding adaptor, respectively, and \(y\) is the output language of the target. During the training period, the pre-trained models \(X_{BART}\) and \(Y_{BART}\) in Equation (3) remain unchanged. The procedure for fine-tuning the encoder is as follows. Initially, the input undergoes layer normalization and enters the first layer of the FFN. The _Tanh_ activation function between the two layers of the FFN assigns a non-linear activation to the current hidden state. Subsequently, the input enters the second FFN, where the initial input is combined as the output hidden state of the whole adaptor. The final encoder output is the final representation of the source language after passing through multiple encoder stacks:  
\[H_{l+1}^{E}=AE(X_{BART}(H+W_{2}\cdot(Tanh(W_{1}\cdot(LN(H)))))) \tag{4}\]  
In Equation (4), \(H\) denote the input hidden state of the encoder adaptor. \(LN\) represents normalization, while \(W_{1}\) and \(W_{2}\) stand for parameters of two FFN. _Tanh_ denotes the nonlinear activation function. \(H_{l}^{E}\) represents the final outcome of the entire representation of the source-side language encoder. The attention layer of the decoder adaptor will model the interdependence of languages:  
\[H_{l+1}^{D}=AD\Big{(}Y_{BART}\Big{(}H_{l}^{D}\Big{)},H^{E},H^{E}\Big{)} \tag{5}\]  
In Equation (5), \(H_{1}^{D}\) represents the output hidden state of the \(1 th\) decoder.