# MIRA-ChatGLM: A Fine-Tuned Large Language Model for Intelligent Risk Assessment in Coal Mining
## 4 Experiments and Results
### LoRA Versus QLoRA  
On a server equipped with two RTX 4090 (48 G) GPUs, we evaluated three parameter-efficient fine-tuning (PEFT) methods to investigate their impact on the performance of the MIRA-ChatGLM model. We utilized the Rouge_chinese [32] and NLTK [36] toolkits to compute ROUGE and BLEU scores and assessed the model's Predict_runtime, Predict_samples_per_second, and Predict_steps_per_second using the evaluation module in the LLaMA-Factory framework [37]. As shown in Figure 6, when fine-tuning with smaller datasets (fewer than 3000 entries), the models exhibited varying performance on BLEU and ROUGE metrics; however, the overall results were suboptimal. Conversely, when the data volume reached 3000 entries, the MIRA-ChatGLM models fine-tuned using the three different methods showed consistent performance on the BLEU metric (and similarly for ROUGE), with relatively high scores. As the data volume continued to increase, improvements in model performance became gradual, while the required hardware resources and training time continued to rise. At approximately 10,000 data points, models that were fine-tuned using all three methods achieved optimal performance across all metrics. To evaluate the performance of different models more comprehensively, we employed three key metrics: Predict_runtime, Predict_samples_per_second, and Predict_steps_per_second to assess the inference speed of the fine-tuned models. The specific data are presented in Table 3. The observations indicate that, with comparable performance in key metrics such as text generation quality and accuracy, there were significant differences in inference speed based on whether the model was fine-tuned using quantization. In particular, models fine-tuned with LoRA required the shortest time to generate a batch of samples. In contrast, the 4-bit quantized QLoRA method took 35% longer than LoRA, while the 8-bit quantized QLoRA method took nearly 60% longer than LoRA. Additionally, in terms of the number of samples generated per second and the number of steps executed per second, LoRA fine-tuning also demonstrated a leading advantage. These data indicate that LoRA fine-tuning not only maintains the quality of generated text but also preserves the model's inference efficiency. The fine-tuned models, once quantized, exhibited a loss in inference speed, which became more pronounced as the quantization level increased. Although the QLoRA method has advantages in reducing model parameters and saving memory usage, its inference speed is significantly lower than that of LoRA fine-tuning. In practical applications, this suggests that when pursuing efficient inference, LoRA fine-tuning may be the better choice, while QLoRA is more suitable for resource-sensitive scenarios that require model size reduction. We randomly provided the model with a common gas alarm dataset and inquired about the reasons for the alarm and corresponding measures from the MIRA-ChatGLM fine-tuned on a 3 k dataset. Table 4 presents the responses of the models that were fine-tuned using the three methods. Figure 6: Scores of BLEU under different fine-tuning methods and data scales. The horizontal axis represents the parameter scale, and the vertical axis represents the score. \begin{table}
\begin{tabular}{c c c c} \hline \hline  & **Predict\_Runtime** & **Predict\_Samples\_per\_Second** & **Predict\_Steps\_per\_Second** \\  & **(/min)** & **(pps)** & **(sps)** \\ \hline LoRA & 25.93 & 1.294 & 0.324 \\ QLoRA-8 bit & 41.60 & 0.806 & 0.202 \\ QLoRA-4 bit & 35.40 & 0.947 & 0.237 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Evaluation parameters for model inference speed. Experimental results indicate that, under the same data volume, the model fine-tuned with LoRA excels in learning deeply from the dataset, which enhances its professionalism and accuracy in answering questions. In contrast, the quantized fine-tuned models may contain more redundant information in their responses, and the proposed handling measures may show some discrepancies.