## 4 Experiments and Analysis
### Experiment on the Impact of Training Epochs on Model Performance  
After each training epoch, the loss of the model is recorded, and true positives (TP) and true negatives (TN) are recorded every 300 training epochs for the subsequent calculation of performance metrics. The relationship between the number of training epochs and the loss function value is illustrated in Figure 4.  
Figure 4: The impact of fine-tuning the number of training epochs on the loss function.  
In this paper, the LLaMA3 model, with versions including LLaMA3 8B, LLaMA3 8B 4 bit, LLaMA3.1 8B Instruct, LLaMA3.1 8B Instruct 4 bit, and Qwen2.5 14B, was fine-tuned on the training set for 1500 epochs, taking 4.5 h, 2.6 h, 4.8 h, 2.8 h, and 6.5 h respectively. For large language models with massive parameters, this time overhead is considered acceptable [31]. During the fine-tuning phase, the loss function of the model after each iteration was tracked and recorded, and a nonlinear function was used for fitting. As shown in the experimental results, the loss function value rapidly decreased with the increase in the number of iterations. After exceeding 1120 iterations, the loss function value of the model remained stable, indicating that the model has good convergence.  
As can be seen from Figure 4, the Qwen2.5 14B model exhibits the fastest decrease in the loss function. The LLaMA3 8B and LLaMA3.1 8B Instruct versions outperform their 4-bit quantized counterparts in terms of the speed of the loss function reduction. This may be due to the fact that quantization may introduce some information loss, affecting the model's learning efficiency.  
The experimental results show that as the number of training epochs increases, the loss function value of the model decreases significantly, indicating that the model gradually adapts to the training data during the learning process. In the early stages, the model's performance improves rapidly, which may be because the model quickly adjusts from its pre-trained state to the specific task. As the number of iterations increases, the rate of performance improvement slows down until a stable state is reached, suggesting that the model has fully learned the patterns in the training data.  
During the fine-tuning process, the model is evaluated every 300 epochs. The evaluation criteria encompass five metrics in total: the discriminative ability of sensitive data on the validation set, precision, recall, F1-score, and false positive rate. An iteration epoch vs. model performance evaluation curve is illustrated in the Figure 5 below.  
The model's accuracy is approximately 70%. As the number of iteration epochs increases, this ability improves significantly. Overall, the LLaMA3 8B and LLaMA3.1 8B Instruct versions perform better than the LLaMA3 8B 4 bit and LLaMA3.1 8B Instruct 4 bit versions in terms of model performance. After completing 2100 epochs of fine-tuning, this capability metric approaches 100%. The convergence speed of the accuracy of the Tongyi Qianwen2.5 14B model is faster as the number of iterations increases, reaching close to 100% after completing 1800 iterations.  
Figure 5: The impact of the number of iterations on model accuracy.  
As shown in Figure 9. The prototype model's false alarm rate was approximately 30%, but as its capabilities improved, the false alarm rate decreased significantly. After 2500 iterations of fine-tuning, the false alarm rate was reduced to 0. The experimental results indicate that more iterations can lead to better model performance. To balance computational cost and model performance, it is recommended to perform fine-tuning for more than 2500 iterations.