## 5 Conclusions and Future Outlook  
With the advancements in pre-trained models and the development of attention mechanisms in natural language processing, the BERT model, built on the Transformer encoder layer, has shown outstanding performance in various downstream tasks. It has become the mainstream approach to combine pre-training with downstream task fine-tuning. Thus, the focus of this work is on the popular BERT model.  
In this work, we deployed the BERT model onto a 64-core processor and improved its prediction accuracy on typical NLP downstream tasks. Leveraging the abundant resources of the multi-core processor, we can parallelize the fine-tuning process to reduce the required training time. First, we have successfully deployed the BERT model onto an ARM-based multi-core processor by migrating the necessary environments and installing the required libraries. Second, to accelerate the fine-tuning training process, we have employed both external model and internal acceleration techniques to reduce training time. This includes setting the right number of CPUs and configuring the inter- and intra-thread counts to minimize training time. The experimental results have shown that setting the number of CPUs as equal to the number of inter and intra threads can achieve the shortest training time. Furthermore, increasing the number of CPUs can reduce the training time when sufficient resources are available. By analyzing the BERT model's source code, we found that the main acceleration part of the model lies in the self-attention computation, which involves matrix multiplications. Through a multi-threading configuration, the matrix multiplication computations can be accelerated. Finally, we have fine-tuned the BERT model on several typical NLP downstream tasks by setting three hyperparameters: batch size, learning rate, and number of epochs. By optimizing these hyperparameters, we can improve the model's prediction accuracy and reduce the training time. The experimental results have shown that increasing the batch size can significantly reduce the training time for downstream tasks, and for larger datasets, achieving the same training accuracy required increasing the number of epochs. Additionally, choosing the right learning rate is key. Our empirical results have shown that we can achieve the best performance when the initial learning rate is set around \(1\times 10^{-5}\). Different downstream tasks require different optimal learning rates, however, and there is no one-size-fits-all learning rate setting that maximizes accuracy for all tasks. Regarding the number of epochs, increasing it within a certain range can result in better performance on downstream tasks, but exceeding a certain range can lead to overfitting and a decline in performance on the test set.  
Our work lays the foundation for future research on deploying the BERT model on the ARM-based multi-core processor. The BERT model can be applied to various tasks in natural language processing, such as information extraction, intelligent question answering, document retrieval, and machine translation. In the field of software engineering, training a BERT model can be employed for tasks such as code clone detection, code generation, and code correction.  
Conceptualization, L.Z. and J.F.; methodology, L.Z., J.F. and W.G.; validation, W.G. and J.F.; writing--original draft preparation, L.Z. and J.F.; writing--review and editing, L.Z. and J.F. All authors have read and agreed to the published version of the manuscript.  
This work was partially funded by the Social Science Fund of Hunan Province, China (Grant No. 22 YBA305) and the National Natural Science Foundation of China (Grant No. 61972408).  
Not applicable.  
The raw data supporting the conclusions of this article will be made available by the authors on request.  
The authors declare no conflicts of interest.