## 4 Experiments
### Hyperparameter Tuning  
In this subsection, we perform hyperparameter tuning for ensemble models. In order, the five hyperparameters are the layer size (number of perceptrons), the dropout rate, the learning rate, the optimization function, and the FC residual block depth. Table 6 shows the entire ENN model's prediction accuracy with varying layer sizes of the ensemble DNN model. The dropout and learning rates were fixed at 0.4 and 0.005, respectively. AdamW was used as an optimization function, and the FC residual block depth was set to six. The learning rate is low with large layers, but more information is learned. On the other hand, with small layers, the learning rate is high, but less information is learned. The ENN model performed best with the layer size set to 1024 as Top-1 and Top-3 accuracies were 98.374% and 99.410%, respectively. Table 7 shows the prediction accuracy of the ENN model according to the dropout rate of the FC residual block. Dropout prevents overfitting in the learning process, and a high dropout rate causes more forgetting in the propagation process. For this experiment, we chose AdamW for optimization. The layer size was set to 1024. The learning rate and the depth of the FC residual block were fixed at 0.005 and 6, respectively. The dropout rate varied between 0.1 and 0.5. We found that the ENN model performed the best with the dropout set to 0.4. However, the difference between other dropout settings was not significant. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c}{**2048**} & \multicolumn{2}{c}{**1024**} & \multicolumn{2}{c}{**512**} & \multicolumn{2}{c}{**256**} \\ \cline{2-9}  & **Top-1** & **Top-3** & **Top-1** & **Top-3** & **Top-1** & **Top-3** & **Top-1** & **Top-3** \\ \hline (2,21) AB & 98.861 & 99.827 & 98.930 & 99.966 & 98.689 & 99.827 & 98.930 & 99.827 \\ (3,32) ABC & 98.483 & 99.502 & 98.256 & 99.592 & 98.120 & 99.457 & 98.211 & 99.389 \\ (4,42) ABCD & 98.585 & 99.500 & 98.344 & 99.362 & 98.568 & 99.431 & 98.413 & 99.465 \\ (5,53) ABCDE & 97.963 & 99.330 & 98.154 & 99.289 & 98.031 & 99.180 & 97.744 & 99.180 \\ (6,63) ABCDEF & 98.068 & 99.160 & 98.321 & 99.149 & 98.033 & 99.103 & 98.137 & 99.275 \\ (7,74) ABCDEFG & 98.051 & 99.158 & 98.296 & 99.315 & 98.267 & 99.119 & 98.228 & 99.128 \\ (8,84) ABCDEFGH & 98.361 & 99.163 & 98.318 & 99.198 & 98.137 & 99.129 & 98.102 & 99.180 \\ \hline Average & 98.339 & 99.377 & 98.374 & 99.410 & 98.264 & 99.321 & 98.252 & 99.349 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Top-1 and Top-3 accuracy(%) of the ENN model with varying layer size. The letter IDs identify the group of design rights. For example, (2,21)AB means that 42 design rights are divided into groups ’A’ and ’B’. \begin{table}
\begin{tabular}{c c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c}{**0.1**} & \multicolumn{2}{c}{**0.2**} & \multicolumn{2}{c}{**0.3**} & \multicolumn{2}{c}{**0.4**} & \multicolumn{2}{c}{**0.5**} \\ \cline{2-11}  & **Top-1** & **Top-3** & **Top-1** & **Top-3** & **Top-1** & **Top-3** & **Top-1** & **Top-3** & **Top-1** & **Top-3** \\ \hline (2,21) AB & 98.965 & 99.896 & 98.930 & 99.966 & 98.723 & 99.827 & 98.999 & 99.896 & 98.965 & 99.793 \\ (3,32) ABC & 98.188 & 99.547 & 98.256 & 99.592 & 98.279 & 99.547 & 98.392 & 99.660 & 98.256 & 99.479 \\ (4,42) ABCD & 98.551 & 99.517 & 98.344 & 99.362 & 98.620 & 99.517 & 98.447 & 99.500 & 98.326 & 99.413 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Accuracy (%) of the ENN model according to the dropout rate.