# A Modified Bayesian Network Model to Predict Reorder Level of Printed Circuit Board
## 4 Model Evaluation
### Evaluation Indicators  
The confusion matrix was taken to visualize the performance of different approaches in which each column of the matrix represents the instances in an actual class while each row represents the instances in a predicted class. All correct predictions are located in the diagonal of each table and errors can be visually inspected by values outside the diagonal. Related terminology and derivations are defined in Table 9 [33].  
\begin{table}
\begin{tabular}{c c c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Predicted Value**} & \multicolumn{8}{c}{**Overserved Value**} \\ \cline{2-13}  & \multicolumn{3}{c}{**Recency = 2–3**} & \multicolumn{3}{c}{**Recency = 4-5**} & \multicolumn{3}{c}{**Recency = 6–10**} \\ \cline{2-13}  & **1** & **2** & **3** & **4** & **1** & **2** & **3** & **4** & **1** & **2** & **3** & **4** \\ \hline
1 & 0 & 1.2 & 1.25 & 1.3 & 0 & 1.15 & 1.2 & 1.25 & 0 & 1.1 & 1.15 & 1.2 \\
2 & 1.3 & 0 & 0.8 & 0.9 & 1.35 & 0 & 0.75 & 0.85 & 1.4 & 0 & 0.7 & 0.8 \\
3 & 1.35 & 0.75 & 0 & 0.9 & 1.4 & 0.8 & 0 & 0.85 & 1.45 & 0.85 & 0 & 0.8 \\
4 & 1.45 & 0.85 & 0.75 & 0 & 1.5 & 0.9 & 0.8 & 0 & 1.55 & 0.95 & 0.85 & 0 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Loss matrix for different binned recency values.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Reorder Level** & **1** & **2** & **3** & **4** \\ \hline Mean value for training samples & 0 & 1.28 & 3.67 & 10.03 \\ Mean value for test samples & 0 & 1.27 & 3.67 & 10.28 \\ Mean value for all samples & 0 & 1.28 & 3.67 & 10.09 \\ \hline \hline \end{tabular}
\end{table}
Table 7: Mean reorder frequency for different reorder levels.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Reorder Level**} & \multicolumn{8}{c}{**Different Clusters**} \\ \cline{2-7}  & **C1** & **C2** & **C3** & **C4** & **C5** & **C6** & **C7** \\ \hline
1 & 89.67 & 91.88 & 65.87 & 58.87 & 89.60 & 42.60 & 92.49 \\
2 & 9.42 & 7.32 & 24.84 & 31.50 & 7.03 & 23.25 & 6.67 \\
3 & 0.78 & 0.64 & 7.06 & 7.99 & 2.18 & 17.79 & 0.65 \\
4 & 0.13 & 0.16 & 2.23 & 1.64 & 1.18 & 16.36 & 0.19 \\ \hline \hline \end{tabular}
\end{table}
Table 8: Average conditional probability of different reorder levels given different clusters (%).  
In order to evaluate the performance of the proposed model, the following mean squared error (_MSE_), mean absolute error (_MAE_), and mean absolute percentage error (_MAPE_) evaluation indicators were used. _MSE_ is the average of square sums between the predicted reorder level \(\alpha_{i}\) and the observed value \(\omega_{i}\)[14]. It defines the goodness of fit of the models and is given by the following equation.  
\[MSE=\frac{1}{n}{\sum\limits_{i=1}^{n}{{{(\alpha_{i}-\omega_{i})}^{2}}}} \tag{10}\]  
The _MAE_ is the average of the sum of the absolute difference between observed values and the predicted reorder level, which can be expressed below.  
\[MAE=\frac{1}{n}{\sum\limits_{i=1}^{n}{{{\left|{{(\alpha_{i}-\omega_{i})}}\right|}}}} \tag{11}\]  
The _MAPE_ is the average of the sum of the normalized absolute difference between observed values and estimated values. The formula is written below.  
\[MAPE=\frac{1}{n}{\sum\limits_{i=1}^{n}{{\left|{{\left|{{\omega_{i}-\alpha_{i}} }\right|}}\right.}}}\times 100 \tag{12}\]