# A Novel Dual-Strategy Approach for Constructing Knowledge Graphs in the Home Appliance Fault Domain
## 4 Experiments
### Experimental Results and Analysis  
In this experiment, models such as Bert-BiLSTM, Bert-BiLSTM-CRF, RoBERTa-zh-IDCNN, and RoBERTa-zh-IDCNN-CRF are introduced for comparative experiments. The table below shows the recognition results of each model in the entity recognition task. From Table 6, we can observe that the proposed RoBERTa-zh-BiLSTM-Attn-CRF model achieves a precision of 96.64% and an F1-score of 93.23% on the dataset. Compared to the basic Bert-BiLSTM model, significant improvements are seen after incorporating RoBERTa-zh pretraining, multi-head attention, and the CRF layer. First, RoBERTa-zh pretraining, compared to Bert, better handles the unique linguistic phenomena and grammatical structures of Chinese. Additionally, the dynamic masking strategy introduced in RoBERTa-zh improves the model's contextual understanding, compared to Bert's fixed masking. When compared to the RoBERTa-zh-BiLSTM-CRF model, the addition of multi-head attention leads to improvements in all three evaluation metrics. This is because the BiLSTM encounters the gradient vanishing problem when processing long-range dependencies in sentences, while the multi-head attention mechanism can directly connect distant positions in the sequence. Furthermore, due to its parallel processing ability, multi-head attention speeds up the training process by performing computations across multiple attention heads simultaneously. In the ontology linking and relation extraction section, this paper selects the Bert-BiLSTM-CRF model as the baseline for relation extraction experiments. Additionally, comparative experiments are conducted by incorporating large language models such as ChatGLM3, ChatGPT-3.5-turbo, ChatGPT-4o, ChatGLM-4-flash, and ChatGLM-4-plus. The following table presents the recognition results of each model in the relation extraction task. \begin{table}
\begin{tabular}{c c} \hline \hline
**Model Parameter** & **Model Parameter Values** \\ \hline BiLSTM hidden layers & 128 \\ Epoch & 10 \\ Learning rate & \(3\times 10^{-5}\) \\ CRF Learning rate & \(3\times 10^{-2}\) \\ Optimizer & AdamW \\ Attention Head Count & 16 \\ top\_p & 0.7 \\ temperature & 0.9 \\ max\_tokens & 2000 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Parameter settings and values. From Table 7, we can see that compared to ChatGLM3 and ChatGPT-3.5-turbo, both models of ChatGLM-4 show improvements in F1-score, accuracy, and recall rate. Specifically, ChatGLM4-plus achieved a 31.54%, 41.04%, and 18.01% improvement in F1-score, accuracy, and recall, respectively, over ChatGLM3. By analyzing the relation extraction results, it is evident that the output from ChatGLM3 contains a large amount of redundant data. For example, in the extraction process, entities such as \"outside unit\", \"compressor malfunction\", \"vibration\", and \"failure to start\" were mentioned. However, the original sentence did not involve the relationships between \"compressor malfunction\" and \"failure to start\". The result returned by ChatGLM3 erroneously associated these two entities. This is due to the combinatory pattern of triples in ChatGLM3, which leads to numerous redundant and unrealistic associations in the returned data. Consequently, this resulted in poor performance across the three evaluation metrics. This phenomenon reflects the issue of fidelity hallucination in the ChatGLM3 model when processing appliance fault data. In contrast, other models overcame this limitation. Additionally, the performance of large language models in extracting \"appliance types\" is relatively poor, such as with entities like \"range hood\", \"exhaust fan\", and \"washing machine\", \"Haier washing machine\". This is primarily because many appliance names in appliance fault repair cases are recorded in colloquial terms or include brand names. When faced with these non-standardized or brand-nested appliance types, the large models tend to extract all possible entities, resulting in many instances of entities with different names but the same meaning within the \"appliance type\" category.