# Research and Application of a Multi-Agent-Based Intelligent Mine Gas State Decision-Making System
## 5 Experiments
### Function Evaluation  
The experiment focuses on evaluating the three core functional modules of the system: intelligent Q&A, intelligent judgment, and report generation. ChatGPT, with versions GPT-3.5-turbo, GPT-4, and GPT-4o, was utilized as the benchmark LLM model for this assessment. The corresponding website for accessing ChatGPT is [[https://chat.openai.com/](https://chat.openai.com/)]([https://chat.openai.com/](https://chat.openai.com/)), accessed on 19 December 2024. To reduce the impact of randomness on the results, the model's temperature parameter was set to 0, ensuring output consistency. A specific scenario was selected to test the practical application effectiveness of the intelligent Q&A module. The test question was: \"The gas sensor is installed at the return corner of the coalface, with the sensor number T0. The current gas concentration is 1.6%. Should an alarm be triggered? If an alarm is required, please provide corresponding handling suggestions\". Figure 5 shows the responses from different models to this question, with key information highlighted using black, bold underlines. The gas sensor To detect a sensor of the cell rating sensor of 1.55.444 at the return corner of the cell rating sensor of 1. The intelligent assessment module obtains real-time data through sensors, including key information such as the name of the coal mine, type of measurement point, installation location, and maximum concentration values. The acquired data can consist of single or multiple entries. Figure 6 shows the output results of the intelligent assessment in JSON format. The performance evaluation results are shown in Table 3. In both the intelligent question answering and intelligent assessment modules, the BLEU and BERT scores of the three models all performed excellently, indicating a high degree of matching between the system-generated answers and the standard answers. This result confirms the system's consistency in understanding and responding to complex safety production issues. Additionally, the report generation module performs excellently in terms of wait time; the error rate for Grammar Errors is maintained at an extremely low level, reflecting high textual accuracy; and the Diversity Score meets the expected standards, indicating good diversity in content generation. These data suggest that the system exhibits high efficiency and quality in report generation. Notably, the GPT-4o model demonstrates superior performance across the three key modules of intelligent Q&A, intelligent judgment, and report generation. To evaluate the performance of the alarm prediction module, accuracy, recall, and F1 score were used as assessment metrics. The comparison results between the system's alarm prediction module and traditional time series prediction models are presented in Table 4. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Model** & **Accuracy** & **Recall** & **F1** \\ \hline Intelligent Mine Gas State Decision-Making & & & & & \\ System Prediction & & & & & \\ Traditional Time Series Prediction & 78.21\% & 20.93\% & 50.94\% \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Evaluation Results of the Alarm Prediction Module. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & **Intelligent Q\&A** & **Intelligent Judgment** & & **Report Generation** & \\ \hline
**LLM-** & **BLEU** & **BERT** & **BLEU** & **BERT** & **Waiting** & **Grammar** & **Diversity** \\
**Backbone** & **Score** & **Score** & **Score** & **Score** & **Time (s)** & **Errors** & **Score** \\ \hline GPT3-turbo & 0.852 & 0.834 & 0.985 & 0.984 & 57.38 & 0.046 & 0.79 \\ GPT4 & 0.927 & 0.934 & 0.986 & 0.989 & 62.92 & 0.038 & 0.84 \\ GPT-4o & 0.957 & 0.968 & 0.993 & 0.991 & 41.28 & 0.024 & 0.87 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: System performance score table. Figure 6: Intelligent judgment output results. The alarm prediction model of the Intelligent Mine Gas State Decision-Making System outperforms traditional time series prediction models in overall performance, particularly in terms of accuracy.