# Hyperparameter Optimization Using Successive Halving with Greedy Cross Validation
## 4 Discussion
### Limitations and Opportunities for Future Research  
Although careful and systematic efforts were taken in this study to investigate the performance of the proposed greedy successive halving algorithm in comparison to that of the standard successive halving algorithm, there nevertheless remain several limitations to this work that merit acknowledgement. First, the experiments described herein relied on four different datasets, two of which involved classification problems and two of which involved regression problems. These datasets are heterogeneous in terms of their number of cases and their numbers of independent and dependent variables, and are publicly available and easily accessible to aid in the reproducibility of the study's findings. Despite these advantages and the observed consistency of the results of the experiments across datasets, it is possible that the greedy successive halving algorithm would perform differently if used with other datasets. Future research should thus subject the greedy successive halving algorithm to other datasets with varying characteristics in order to ascertain the extent to which greedy successive halving performs consistently across a wide variety of datasets.  
In addition to relying on just four different datasets, the findings of the experiments reported herein are also limited insofar as they relied on only five different machine learning algorithms. Although care was taken to include three different ML classifier algorithms and two different ML regressors in the study, and although the ML algorithms used in the experiments were chosen because of their widely varying approaches to machine learning, there remain dozens of other ML algorithms that were not included in this investigation. Future research in this area should therefore endeavor to evaluate the performance of greedy successive halving when used as a basis for rapid hyperparameter optimization in AI/ML scenarios involving other popular machine learning algorithms.  
Next, the experiments described in the current study evaluated the comparative performance of the greedy successive halving algorithm at selecting a high-performing ML model from sets of 250, 500, and 1000 different candidate models. There is some evidence in the results from the experiments that suggests that the relative speed with which the greedy successive halving algorithm is able to complete the ML model selection task tends to improve in comparison to an exhaustive search as the number of candidate modelsincreases. The nature of the relationship between the cardinality of the set of candidate models and the advantage in speed that the greedy successive halving algorithm has over an exhaustive search was not formally studied in this paper, and this represents another opportunity for future research.  
Finally, the standard successive halving algorithm, the proposed greedy successive halving algorithm, and the exhaustive search algorithm against which the competing successive halving algorithms were compared were all implemented in this study as serial processes. Although doing so would not be trivial, it would certainly be feasible to develop versions of the greedy successive halving algorithm that take advantage of parallel processing. It is currently unknown how well the performance of a parallel implementation of the greedy successive halving algorithm would compare against the performance of a parallel implementation of the standard successive halving algorithm. Developing and evaluating the performance of a parallel version of the greedy successive halving algorithm is thus a potentially very fruitful opportunity for future research on state-of-the-art hyperparameter optimization.