# MIRA-ChatGLM: A Fine-Tuned Large Language Model for Intelligent Risk Assessment in Coal Mining
## 4 Experiments and Results  
### Impact of Dataset Size  
Our study investigates the impact of dataset size on the performance of the MIRA-ChatGLM model, particularly across eight different data scales, using LoRA and various quantization levels (8-bit and 4-bit) for fine-tuning. These datasets originate from historical alarm information uploaded by gas sensors during the coal mining production process. The performance comparisons of the MIRA-ChatGLM fine-tuned under different data scales and methods are illustrated in Figure 5.  
Generally, model performance improves with increasing data volume [33, 34, 35]. This is because large-scale data provides richer information, helping the model to learn features and patterns more effectively while reducing the risk of overfitting. Our research confirms this: as the data volume increases, the MIRA-ChatGLM model can learn more data patterns and features, significantly enhancing performance until approximately 3000 data points. However, as shown in Figure 3, when the data volume exceeds 3000, the model's generalization ability to use new data diminishes, and the performance indicators change slowly despite the continued increase in training data. Specifically, the performance indicators  
Figure 5: Line graph of the performance of MIRA-ChatGLM under three fine-tuning methods with different data scales. As the data volume increases, the performance of MIRA-ChatGLM gradually improves. The horizontal axis represents the parameter scale, and the vertical axis represents the score. of models fine-tuned with 5000, 7000, and 10,000 data points are nearly identical. This phenomenon may stem from information redundancy, model capacity limitations, and diminishing marginal returns. As the data volume increases, the effective information provided by additional data decreases; the model has already mastered most of the important features, and excessive data may introduce noise, leading to a gradual reduction or saturation of performance gains. When fine-tuning with QLoRA using 8-bit quantization, we observed a slight decline in performance indicators as the data volume increased from 500 to 700, followed by a steady improvement as the data volume continued to rise to 1000. We hypothesize that this performance fluctuation may be related to the variability in results when evaluating quantized models, especially in cases with smaller dataset sizes, where this variability significantly impacts the comparative performance of the quantized model. This variability may arise from various factors, including the diversity of samples in the dataset, randomness during model training, and the model's sensitivity to specific data. To mitigate the impact of this variability on performance comparisons, we implemented measures such as conducting multiple repeated experiments and reporting average performance and standard deviation, as well as employing techniques like cross-validation to more accurately estimate model performance on limited data. Specifically, the base model GLM-4-9B-chat without fine-tuning demonstrates significant limitations in processing alarm data assessments, analyzing alarm causes, and providing relevant mitigation measures. It can only offer general information regarding common situations of gas alarm events in coal mines and general response measures, failing to conduct in-depth risk assessments or propose specialized handling methods for the specific alarm data uploaded by sensors. After fine-tuning small-scale datasets (particularly those composed of 300, 500, and 700 data points), the MIRA-ChatGLM model is generally able to conform to the required format when answering questions and can accurately identify alarm data. However, its ability to analyze alarm signals precisely, determine the causes of alarms, and provide corresponding mitigation measures still requires further enhancement. This performance shortcoming may be attributed to the small dataset size and insufficient sample diversity, leading to high variability in the model's training results.