## 5 Results  
In this study, we randomly split the entire dataset into 80% for training and the remaining 20% for testing. This process was repeated 30 times to secure more reliable results than a single experiment. Table 4 shows the classification performance of each model for the nine product groups and Figure 15 shows the box plots of the AUROC of three classification models for each product group. Bold numbers in the table are the highest values among the three classification models for each group. \begin{table}
\begin{tabular}{c c c c c c c c c c c c c c c c c c c} \hline \hline \multicolumn{1}{c}{\multirow{2}{*}{**Group**}} & \multicolumn{6}{c}{**Binary**} & \multicolumn{6}{c}{**Random RGB**} & \multicolumn{6}{c}{**Bin2 Vec**} \\ \hline \multicolumn{1}{c}{**Group**} & **Model** & **REC** & **PRE** & **ACC** & **F1** & **ERC** & **AUC** & **REC** & **F1** & **BCR** & **AUC** & **REC** & **PRE** & **ACC** & **F1** & **BCR** & **AUC** \\ \hline \multirow{1}{*}{1} & \multirow{5}{*}{RF} & 83.72 & 82.45 & 91.81 & 83.03 & 88.86 & 0.946 & 61.46 & 74.11 & 85.61 & 67.14 & 75.66 & 0.880 & 61.77 & 73.61  
Except for product Group 6, which will be discussed later, the proposed CNN model outperformed the MLP and RF with all WBM coloring schemes for all performance evaluation criteria. Not only did it yield the best performance for all product groups, it also resulted in the most stable performance; the standard deviations for 30 repetitions are much smaller than those of MLP and RF. The BCR of the CNN model improved by 8.13% on average (min: \(-\)0.44% for product Group 6 with binary, max: 22.6% for product Group 9 with random RGB), and the F1-measure of the CNN model improved by 15.14% on average (min: 0.98% for Group 6 with random RGB, max: 36.8% for product Group 6 with binary). In addition, the AUC of the CNN model improved by 0.04 on average (min: \(-\)0.029 for product Group 6 with binary, max: 0.12 for Group 8 with binary). These improvements are all statistically significant at significance level of 0.01. Between the MLP and RF, RF generally yielded better bad wafer classification performance than MLP. The average differences between BCR, F1-measure, and AUC are 3.24%, 0.72%, and 0.02, respectively. Figure 15: Box plot of DNN, RF, CNN, AUC results based on 1\(\sim\)9 groups. With respect to the WBM coloring scheme, both random RGB and Bin2 Vec generally worked well for bad wafer classification. Their AUROC values are greater than 0.99 for all product groups. There are no significant differences between the random RGB and Bin2 Vec in terms of bad wafer classification performance. In other words, although the bin codes on the wafer are represented by randomly assigned colors, different bin codes are distinguishable such that the bad wafer classification model can learn the difference between bad and good wafers with high accuracy. This implies that although the relationship between bin codes was are not configured, the weights of convolution operations can be appropriately learned for the purpose of accurate bad wafer classification. An interesting observation is that when binarization was used to represent the WBM color, its bad wafer classification performance significantly decreased for product Group 6 irrespective of classification models. Its AUROC is 0.292 although that of random RGB and Bin2 Vec for the same product group is 0.979. It can be concluded that for a certain wafer group, it is not sufficient to know whether each die passes all EDS tests or not, but additional information, i.e., which combination of EDS tests fails, must be provided for an improved classification performance. The experimental results show that both random RGB and Bin2 Vec provided satisfactory performance in the bad wafer classification task. However, Bin2 Vec has the advantage of more in-depth analysis for discovering significant patterns among bad WBMs. Figure 16 shows all bad WBMs in a two-dimensional space reduced by the \(t\)-SNE method [40].