## 3 Materials and Methods
### Seq2 Seq+Attention  
**Seq2 Seq:** The Seq2 Seq model framework is a sequence training method proposed by [PERSON] et al. in 2014, which can map sequences to sequences. This method solves the problem that traditional deep learning networks can only input and output with fixed dimensional encoding. It has been widely used in machine translation, speech recognition, and question-answering systems. Later machine-translation methods were also designed based on this structure.  
The basic Seq2 Seq model is designed based on RNNs, consisting of two parts: an encoder and a decoder. The initial Seq2 Seq model was composed of LSTM cells [18], where one LSTM is utilized to read input sequences time step by time step and acquire a large fixed-dimensional vector representation, and another LSTM is used to extract output sequences from the vector. The second LSTM's input sequence is conditioned since the decoder will take the previous output as the current input. If the previous output is not sufficiently accurate, it will affect subsequent predictions. Therefore, a technique called teacher forcing is applied during training by using the correct form of each sentence as the input to the decoder, thus enforcing the correct input during training.  
**Attention Mechanism:** Although the Seq2 Seq model already closely mimics the behaviour and thinking process of human translation, which is to read a sentence in the source language, understand its meaning, and then translate it into the target language in one sentence, there are still two drawbacks to Seq2 Seq. Firstly, the encoder struggles to compress a long sentence into a fixed-dimensional vector. Secondly, the decoder forgets information in the output vector generated by the encoder during decoding.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Language** & **Original Sentence** & **Back-Translation** \\ \hline Chinese & \"\(\backslash\)\" & \"\(\backslash\)\" & \"\(\backslash\)\" \\ \hline English & \
\begin{tabular}{c} A civil aviation plane carrying 43 people crashed on \\ Monday morning while flying over high mountains \\ north of the Afghan capital Kabul. The weather \\ conditions at that time were extremely harsh. \\ \end{tabular} &
\begin{tabular}{c} This 43 person civil aircraft crashed from a high mountain north of \\ the Afghan capital Kabul on Monday morning. The climate \\ conditions at that time were very harsh. \\ \end{tabular} \\ \hline \hline \end{tabular}
\end{table}
Table 3: The effect of back-translation of this article.  
Figure 2: Back-translation flowchart. The written language dataset is translated into an intermediate language dataset through A Translator, and then the intermediate language dataset is translated into the corresponding colloquialism dataset through B Translator.  
[PERSON] et al. proposed a new approach in 2014 [19] that used the context information vector generated by the encoder as an additional input at each time step during decoding. In 2015, [PERSON] et al. proposed a global attention mechanism for translation [20]. It provides three simple algorithms for calculating attention weights, as shown in Equation (1).  
\[score(h_{t},\bar{h_{s}})=\left\{\begin{array}{ll}{{h_{t}}^{\top}\bar{h_{s}}},& \mathit{dot},\\ {{h_{t}}^{\top}{{W_{a}}\bar{h_{s}}}},&\mathit{general},\\ {{v_{a}}^{\top}{\tanh({W_{a}}[{h_{t}};\bar{h_{s}}])}},&\mathit{concat}.\end{array}\right. \tag{1}\]  
In this Equation, \(h_{t}\) represents the hidden state at the current time step, \(\bar{h_{s}}\) represents the hidden state of the source language, and \({W_{a}}\) and \({V_{a}}\) are both parameter matrices that need to be learned.  
This paper replaces LSTM with the bidirectional gated recurrent unit (GRU) [21] in the Seq2 Seq model. The Seq2 Seq+attention de-colloquialism model architecture used in this paper is shown in Figure 3.