# Hybrid Decision-Making-Method-Based Intelligent System for Integrated Bogie Welding Manufacturing
## 4 Results and Interpretability
### Intelligent Decision Results  
The 284 welding process data for bogie welding were used for system testing, including 107 side beams, 55 cross beams, 55 frames, and 22 brake hangers and beams. The data are mainly derived from WPS data that have been approved by experts and applied in engineering production. We defined design information as known data (type of base material, size of base material, quality grade, joint form, etc.) and process information (welding position, assembly gap, blunt edge, welding parameters, preheating temperature, etc.) as decision data. The model completes validates the decision information based on known information. The ratio of correct cases to the number of test cases, called Case Accuracy, is used to evaluate the accuracy of case matching. The accuracy of corrected cases (Corrected Accuracy) is calculated by the ratio of correct cases (Case Accuracy) to test cases (Test Cases). The test results are shown in Table 4.  
As shown in Table 4, the case accuracy is 0.792, and the corrected accuracy is 0.947. Compared with the case-matching results, the corrected case significantly improved accuracy. The low case matching accuracy is mainly due to the incomplete case base, especially the lack of \"Crossbeam\" and \"Brake hanger/beam\" cases. In addition, the correctness of the input known conditions also affects the matching accuracy. Even though the corrected case achieves high accuracy, the limitation that the logical representation is designed for most problems and those individual problems are easily ignored has a negative impact on accuracy. Therefore, the decision accuracy of the system may be improved by enhancing the iterative ability of the case base, monitoring the condition input, and improving the rule compatibility. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**No.** & **Part Name** & **Test Cases** & **Right Case** & **Case Accuracy** & **Corrected Case** & **Corrected Accuracy** \\ \hline
1 & Side Beam & 107 & 90 & 0.841 & 103 & 0.963 \\
2 & Vehicle Frame & 110 & 92 & 0.836 & 107 & 0.973 \\
3 & Crossbeam & 55 & 35 & 0.636 & 49 & 0.891 \\
4 & Brake hanger/beam & 12 & 8 & 0.667 & 10 & 0.833 \\ Total & & 284 & 225 & 0.792 & 269 & 0.947 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Structured document decision accuracy. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Model** & **Metrics** & **Technology** & **Manufacture** & **Design** & **Department** & **Standard** & **Quality** \\ \hline \multirow{3}{*}{HMM} & Precision & 0.512 & 0.569 & 0.593 & 0.556 & 0.905 & 0.713 \\  & Recall & 0.528 & 0.474 & 0.743 & 0.590 & 0.880 & 0.599 \\  & F1-score & 0.519 & 0.517 & 0.646 & 0.569 & 0.892 & 0.650 \\ \hline \multirow{3}{*}{CRF} & Precision & 0.783 & 0.686 & 0.715 & 0.722 & 0.935 & 0.758 \\  & Recall & 0.603 & 0.429 & 0.788 & 0.501 & 0.891 & 0.671 \\  & F1-score & 0.678 & 0.527 & 0.748 & 0.586 & 0.912 & 0.710 \\ \hline \multirow{3}{*}{BiLSTM} & Precision & 0.613 & 0.476 & 0.592 & 0.662 & 0.937 & 0.569 \\  & Recall & 0.386 & 0.463 & 0.542 & 0.497 & 0.818 & 0.497 \\ \cline{1-1}  & F1-score & 0.473 & 0.469 & 0.535 & 0.568 & 0.873 & 0.530 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Entity extraction model results metrics. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Models** & **Metrics** & **Belong\_to** & **Reference** & **Requirement** & **Applicable\_to** & **Unknown** & **Macro-Average** \\ \hline \multirow{3}{*}{BiLSTM + Attention} & Precision & 0.838 & 0.568 & 0.806 & 0.857 & 0.872 & 0.788 \\  & Recall & 1.000 & 0.875 & 0.625 & 0.878 & 0.850 & 0.846 \\  & F1-score & 0.912 & 0.688 & 0.704 & 0.867 & 0.861 & 0.816 \\ \hline \multirow{3}{*}{CR-CNN} & Precision & 0.912 & 0.704 & 0.670 & 0.818 & 0.850 & 0.791 \\  & Recall & 1.000 & 0.792 & 0.763 & 0.878 & 0.638 & 0.814 \\ \cline{1-1}  & F1-score & 0.954 & 0.745 & 0.713 & 0.847 & 0.729 & 0.802 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Relationship extraction model results metrics. For causal decision making, 10 decision questions were selected for testing the system.