## 4 Results and Discussion
### Performance Results  
#### 4.2.1 Training Accuracy  
We train models on an NVIDIA RTX 2080 Ti and NVIDIA Titan RTX. Since their memory is limited (11 GB and 24 GB), we use different batch sizes of 8 and 16, respectively, to avoid the out-of-memory (OOM) error. At the same time, to ensure the stability of the model, we use the gradient accumulation method to expand the batch size in another form. Gradient accumulation is to accumulate the gradients of several batches and then update  
\begin{table}
\begin{tabular}{l r r} \hline \hline
**Parameters** & **Notes** & **Defaults** \\ src\_vocab\_size & English vocabulary size & 32,000 \\ tgt\_vocab\_size & Chinese vocabulary size & 32,000 \\ batch\_size & Number of samples selected for one training & 32 \\ epoch\_num & Number of epochs & 40 \\ d\_model & Feature dimensions of the model & 512 \\ max\_len & Max length of sentence & 60 \\ beam\_size & Beam size for bleu & 3 \\ early\_stop & Early stop for loss increase & 5 \\ optimizer & Use an optimizer for training & Adam \\ use\_gpu & Whether or not to use a GPU & 0 \\ \hline \hline \end{tabular}
\end{table}
Table 1: The configurable training parameters.  
\begin{table}
\begin{tabular}{l l l l l l} \hline \hline
**CPUs** & **Freq.1\#Core** & **GPUs** & **Category** & **Linux Kernel** & **OpenMP** \\ \hline Intel Core 1/-7500U & 2.7 GHz 1 2 & N/A & Laptop & v\_15.0 & GCC v7.5.0 \\ Intel Core 1/-7700K & 4.2 GHz 1 4 & N/A & Desktop & v\_15.0 & GCC v7.5.0 \\ Intel Xeon Platinum 9242 & 2.3 GHz 196 & N/A & Server & v\_310.0 & GCC v4.8.5 \\ Intel Xeon Silver 4210 & 2.2 GHz 1 40 & NVIDIA Titan RTX & Server & v\_18.0 & GCC v8.4.1 \\ Intel Xeon Silver 4210 & 2.2 GHz 140 & NVIDIA GeForce & Server & v\_18.0 & GCC v8.4.1 \\ Intel Xeon Silver 4210 & 2.2 GHz 140 & RTX 2080 Ti & NVIDIA GeForce & Server & v\_18.0 & GCC v8.3.1 \\ Intel Xeon Silver 4210 & 2.2 GHz 140 & RTX 2060 SUPER & Server & v\_18.0 & GCC v8.3.1 \\ \hline \hline \end{tabular}
\end{table}
Table 2: The hardware and system software.  
the network parameters. The number of batches accumulated in each update is called the accumulation step. Figure 7 shows the training process on these two GPUs, respectively. Each curve describes the trend of loss (of the training set and validation set) and BLEU value of the validation set. We see that the training process on both GPUs is basically consistent. As the number of epochs increases, the train loss continues to decrease. In addition, the loss and BLEU values of the verification set are stable at half way. As a matter of fact, too many training iterations would lead to the problem of overfitting, which has a negative impact on the translation quality. Thus, we run the verification process for every five iterations to avoid the overfitting issue.  
We compare the test performance of the trained models on two GPUs in Table 3. The batch_size (bs) and the accumulation step (step) used in the gradient accumulation method for each GPU are indicated in the table. The different GPU devices basically have no impact on the model training effect. The gradient accumulation method can facilitate us in achieving the same performance with small batches as with large batches.