## 4 Experiments
### Hyperparameter Tuning
\begin{table}
\begin{tabular}{c c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c}{**UP-DETR**} & \multicolumn{2}{c}{**Yolo**} & \multicolumn{2}{c}{**EfficientNet**} & \multicolumn{2}{c}{**ResNet**} & \multicolumn{2}{c}{**WideResNet**} \\ \cline{2-11}  & **Top-1** & **Top-3** & **Top-1** & **Top-3** & **Top-1** & **Top-3** & **Top-1** & **Top-3** & **Top-1** & **Top-3** \\ \hline (1,21) K & 99.965 & 100.000 & 96.308 & 98.689 & 99.931 & 100.000 & 99.068 & 100.000 & 99.482 & 99.965 \\ (1,21) L & 100.000 & 100.000 & 98.344 & 99.413 & 99.655 & 100.000 & 99.310 & 100.000 & 98.965 & 100.000 \\ (1,42) M & 99.879 & 100.000 & 97.981 & 98.689 & 99.948 & 100.000 & 99.586 & 100.000 & 99.620 & 100.000 \\ (1,42) N & 99.845 & 100.000 & 96.411 & 99.051 & 99.931 & 100.000 & 99.396 & 99.983 & 99.396 & 99.983 \\ (1,84) O (Single) & 99.784 & 99.957 & 90.709 & 96.299 & 99.905 & 100.000 & 99.569 & 99.983 & 99.681 & 99.983 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: _Cont._Table 8 shows the Top-1 and Top-3 accuracy of the ENN model according to the learning rate of the ensemble model. For this experiment, we used AdamW for optimization. The layer size, the dropout rate, and the FC residual block depth were set to 1024, 0.4, and 6, respectively. The learning rate is used in the learning process to limit how much it learns at a time. With a high learning rate, significant weight changes lead to quick learning. However, the learning result can be sub-optimal. With the low learning rate, more weight values are examined, which can lead to more optimal results. However, the low learning rate makes the whole learning process slower. Compared to the layer size and dropout rate, the ENN model was sensitive to the learning rate regarding the prediction accuracy. The best Top-1 and Top-3 accuracy was obtained with a learning rate of 0.005. Recently, [PERSON] et al. [38] suggested a method to adjust the learning degree in stages according to epochs to expedite learning without falling into local minimum. Table 9 shows the prediction accuracy of the ENN model according to the optimization function. As mentioned above, we fixed the layer size, the dropout rate, and the learning rate at the values that led the ENN model to perform the best. The depth of the FC residual block was fixed at six. The optimization function directs the learning process to find the global minimum of loss as quickly as possible without falling into the local minimum. We experimented with RMSprop [39], SGD [40], Adam [41], and AdamW [42]. Specifically, we set the momentum of both SGD and RMSprop to 0.9. As a result, the ENN model performed the best with AdamW. RMSprop showed a sharp drop in Top-1 accuracy as the number of classes and the backbone models increased. \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c}{**0.1**} & \multicolumn{2}{c}{**0.2**} & \multicolumn{2}{c}{**0.3**} & \multicolumn{2}{c}{**0.4**} & \multicolumn{2}{c}{**0.5**} \\ \cline{2-10}  & **Top-1** & **Top-3** & **Top-1** & **Top-3** & **Top-1** & **Top-3** & **Top-1** & **Top-3** & **Top-1** & **Top-3** \\ \hline (5,53) ABCDE & 97.935 & 99.221 & 98.154 & 99.289 & 97.935 & 99.262 & 98.113 & 99.439 & 98.195 & 99.330 \\ (6,63) ABCDEF & 98.068 & 99.034 & 98.321 & 99.149 & 97.930 & 99.045 & 98.263 & 99.218 & 98.091 & 99.114 \\ (7,74) ABCDEFG & 98.237 & 99.285 & 98.296 & 99.315 & 98.208 & 99.138 & 98.374 & 99.256 & 98.159 & 99.138 \\ (8,84) ABCDEFGH & 98.206 & 99.111 & 98.318 & 99.198 & 98.456 & 99.172 & 98.275 & 99.249 & 98.068 & 99.189 \\ \hline Average & 98.307 & 99.373 & 98.374 & 99.410 & 98.307 & 99.358 & 98.409 & 99.460 & 98.294 & 99.351 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: _Cont._