# A Modified Bayesian Network Model to Predict Reorder Level of Printed Circuit Board
## 4 Model Evaluation
### Experimental Results
Figure 8 indicates that the difference between estimated values and actual values is small in most cases and almost all of them are less than 1. In order to verify the proposed ensemble approach in this research, the data preprocessing and modified BN prediction model were implemented and the performance was compared to other  
Figure 8: Observed and estimated reorder frequency with 95% upper limits by Monte Carlo simulations. (**a**) Training observed and estimated reorder frequency with 95% upper limits. (**b**) Test observed and estimated reorder frequency with 95% upper limits. classifiers including TAN, AdaBoost, and ANN. Among the four competing methods, TAN as a naive Bayesian network has been widely utilized in classification [17; 18]. AdaBoost is an ensemble method whose output is the weighted average of many weak classifiers and is the best-known and most widely applied boosting algorithm in both research and practice [34]. ANN has a strong learning ability and has also been widely used for prediction and classification [35; 36; 37]. TAN and ANN were implemented in the IBM SPSS Modeler and AdaBoost was developed by Matlab while the proposed modified BN was developed based on Matlab and package FullBNT. Confusion matrices of different approaches are given in Figure 9. These confusion matrices show that TAN achieved the highest sensitivity for observed reorder levels 2, 3, and 4. However, the ability to identify a reorder level 1 was weak and the unbalanced and biased distribution of reorder level 1 can greatly increase the production risk if a large amount of orders were taken as batch production in advance without customers' confirmation. On the other hand, the modified BN achieved better results with sensitivity 98.5% and 98.1% for training and test samples, respectively. It can reduce the number of samples with a reorder level 1 to be incorrectly predicted as 2, 3, or 4, which reduces the risk of batch production. In addition, the modified BN can correctly identify a higher amount of orders with an observed reorder level 2 and 3 compared with AdaBoost and ANN both for training and test samples. The sensitivity of the modified BN for observed reorder level 4 deteriorated for the test sample. However, many samples have been predicted as 2 or 3, which can also be taken for batch production. Overall indicators of confusion matrices also show that the modified BN obtained the highest accuracy (81.9%) both for training and test samples. The approaches were compared both for training and test samples according to indicators presented in Equations (10)-(12) and the comparison results can be referred to Table 10. It shows that the proposed modified BN obtained the lowest _MAE_ and _MAPE_ for training samples and the lowest _MSE_ and _MAE_ for test samples. The results in Figure 9 also illustrate that TAN obtained the maximum correctly classified instances with observed reorder levels of 2, 3, and 4 as well as the maximum incorrectly classified instances with an observed reorder level of 1 compared to the other classifiers. Therefore, the indicators show that TAN achieved the lowest _MSE_ for training samples as well as almost the largest _MAE_ and _MAPE_ both for training and test samples. This may be caused by redundant links between the evidence node and the missing strong links such as the causal relationships between _F2_, _F3_, _F6_, and _F1_. Yet, it is worth noting that TAN deteriorated greatly on the test dataset according to _MAE_ and _MAPE_, which indicates that the TAN considered in the current study lacked robustness and generalization ability. The ANN exhibited steady performance both for training and test samples but had no superiority according to the three indicators. The AdaBoost achieved slightly better performance for test samples for the indicator _MAPE_ but the worst performance for the indicators _MSE_ and _MAE_.