# Hyperparameter Optimization Using Successive Halving with Greedy Cross Validation
## 3 Results
### Experiment Results: Wall-Clock Time
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Dataset**} & **ML** & **Greedy Successive Halving** & **Standard Successive Halving** \\  & **Algorithm** & \(k=5\) & \(k=10\) & \(k=5\) & \(k=10\) \\ \hline \multirow{2}{*}{\begin{tabular}{c} California \\ Housing \\ \end{tabular} } & Passive & 0.034 *** & 0.034 *** & 0.057 & 0.063 \\  & \begin{tabular}{c} Tweedie \\ \end{tabular} & 0.093 *** & 0.074 *** & 0.157 & 0.141 \\ \hline \multirow{2}{*}{\begin{tabular}{c} Diabetes \\ \end{tabular} } & Passive & 0.228 *** & 0.121 *** & 0.659 & 0.634 \\  & \begin{tabular}{c} Tweedie \\ \end{tabular} & 0.315 *** & 0.152 *** & 1.028 & 0.890 \\ \hline \multirow{2}{*}{\begin{tabular}{c} Wine \\ Recognition \\ \end{tabular} } & Bernoulli & 0.259 *** & 0.178 *** & 0.971 & 0.970 \\  & \begin{tabular}{c} Decision Tree \\ Deep Neural \\ Network \\ \end{tabular} & 0.220 *** & 0.164 *** & 0.821 & 0.845 \\  & \begin{tabular}{c} Deep Neural \\ Network \\ \end{tabular} & 0.111 *** & 0.079 *** & 0.466 & 0.473 \\ \hline \multirow{2}{*}{\begin{tabular}{c} Wisconsin \\ Diagnostic \\ Breast Cancer \\ \end{tabular} } & Bernoulli & 0.335 *** & 0.259 *** & 0.863 & 0.892 \\  & \
\begin{tabular}{c} Decision Tree \\ Deep Neural \\ Network \\ \end{tabular} & 0.149 *** & 0.113 *** & 0.374 & 0.372 \\  &
\begin{tabular}{c} Deep Neural \\ Network \\ \end{tabular} & 0.091 *** & 0.063 *** & 0.249 & 0.231 \\ \hline \hline \end{tabular}  
* \(p<0.001\) for a Welch’s \(t\)-test comparing wall-clock times for the greedy and standard successive halving algorithms.  
\end{table} 
Table 4: Mean standardized wall-clock times for evaluation of 500 candidate models, relative to an exhaustive search. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Dataset**} & **ML** & **Greedy Successive Halving** & **Standard Successive Halving** \\  & **Algorithm** & \(k=5\) & \(k=10\) & \(k=5\) & \(k=10\) \\ \hline \multirow{2}{*}{\begin{tabular}{c} California \\ Housing \\ \end{tabular} } & Passive & 0.030 *** & 0.030 *** & 0.050 & 0.056 \\  & \begin{tabular}{c} Tweedie \\ \end{tabular} & 0.082 *** & 0.075 *** & 0.142 & 0.147 \\ \hline \multirow{2}{*}{\begin{tabular}{c} Diabetes \\ \end{tabular} } & Passive & 0.227 *** & 0.130 *** & 0.638 & 0.625 \\  & \begin{tabular}{c} Tweedie \\ \end{tabular} & 0.297 *** & 0.151 *** & 1.004 & 0.882 \\ \hline \multirow{2}{*}{\begin{tabular}{c} Wine \\ Recognition \\ \end{tabular} } & Bernoulli & 0.250 *** & 0.175 *** & 0.953 & 0.962 \\  & \begin{tabular}{c} Decision Tree \\ Deep Neural \\ Network \\ \end{tabular} & 0.224 *** & 0.166 *** & 0.834 & 0.841 \\  & \begin{tabular}{c} Network \\ \end{tabular} & 0.106 *** & 0.084 *** & 0.467 & 0.551 \\ \hline \multirow{2}{*}{\begin{tabular}{c} Wisconsin \\ Diagnostic \\ Breast Cancer \\ \end{tabular} } & Bernoulli & 0.322 *** & 0.250 *** & 0.852 & 0.874 \\  & \
\begin{tabular}{c} Decision Tree \\ Deep Neural \\ Network \\ \end{tabular} & 0.145 *** & 0.107 *** & 0.357 & 0.354 \\  &
\begin{tabular}{c} Deep Neural \\ Network \\ \end{tabular} & 0.082 *** & 0.068 *** & 0.241 & 0.272 \\ \hline \hline \end{tabular}  
* \(p<0.001\) for a [PERSON]’s \(t\)-test comparing wall-clock times for the greedy and standard successive halving algorithms.  
\end{table} 
Table 5: Mean standardized wall-clock times for evaluation of 1000 candidate models, relative to an exhaustive search. under otherwise identical conditions. The comparative overall average wall-clock times required by the competing successive halving algorithms to complete the ML model selection task for varying numbers of candidate models and folds is illustrated in Figure 6 below.