# Research on Large Language Model for Coal Mine Equipment Maintenance Based on Multi-Source Text
## 4 Results and Discussion
### Experimental Results Analysis  
#### 4.2.1 Coal Mine Equipment Dialog Consultation  
An experiment in which prior historical dialogs were randomly used as input in sequence in light of evaluating the performance of dialog consultation for coal mining equipment was conducted. Sample responses are shown in Table 2. BLEU [36] and ROUGE [37] were used to assess the quality of the dialog. BLEU is a commonly used metric that compares a candidate translation to one or more reference texts based on n-element syntactic precision. The larger the value, the better. On the other hand, ROUGE is a particularly important index for evaluating automatic summarization and machine translation. This is useful as it focuses on the recall aspect of the generated abstracts by comparing them with the references. As with the former, the larger the value, the better.  
The results of the experiment are illustrated in Table 3, showing that XCoalChat achieved excellent ratings on all metrics in the BLEU and ROUGE scores. It revealed that the dataset constructed in this paper contains more diverse and rich knowledge about coal mine equipment maintenance, and the knowledge density of the model was improved by the method of knowledge graph enhancement, which allows the model to cover the information provided by the reference text more effectively by generating responses under the consideration of the constraints between the knowledge triples.  
\begin{table}
\begin{tabular}{l l} \hline \hline
**Dialog Example** \\ \hline
**User:** At present, the main way of coal mine equipment full life cycle management \\
**XCoalChat:** The existing coal mine machinery and equipment full life cycle management system \\ stays at the level of the sensing data visual display and simple threshold analysis. The maintenance method primarily adopts the point inspection and patrol method, and takes the maintenance decision of regular and post-disposal. Taking the existing mining hoist monitoring system as an example, it is able to monitor the operation process information of the hoist through PLC data, but is unable to identify the early failure hidden danger of the motor and other key points. \\
**User:** What are the pain point issues faced? \\
**XCoalChat:** 1. There are many problems in coal mine equipment data such as multi-source heterogeneity, noise pollution, and the lack of effective labels, which lead to low data value density. It is essential to study the modeling method of equipment status knowledge to improve the knowledge expression ability of on-site monitoring data. 2. A large amount of normal \\ operation repetitive data has been accumulated while equipment failure samples are missing. In the existing various bearings, motors, gearboxes and other subsystems, and various types of test bench fault sample data, the study of transfer learning and other technical means to achieve a single sub-system to the complex system of knowledge generalized learning. 3. Strengthen the research on the knowledge characteristics and failure mechanisms of equipment systems, combine with data-driven methods, and establish a three-sphere integrated equipment full life cycle model of knowledge, mechanism, and data. \\ \hline \hline \end{tabular}
\end{table}
Table 2: Example of coal mine equipment dialog.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model** & **BLEU** & **ROUGE-1** & **ROUGE-2** & **ROUGE-L** \\ \hline ChatGLM & 6.6201 & 15.1096 & 2.6145 & 11.1354 \\ Bloom & 6.9827 & 15.4548 & 1.1098 & 8.5373 \\ LLama & 5.7694 & 13.1431 & 2.2185 & 10.5258 \\ XCoalChat & 7.4353 & 26.5329 & 7.7447 & 20.8454 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Comparison of the dialog consultation of coal mine equipment.