Repository: autogluon/autogluon

Issue:
`hyperband` and `rl` search strategies are nut running for Tabular prediction. 
```
import autogluon as ag
from autogluon.task import TabularPrediction as task

import sklearn.datasets
import pandas as pd

cancer_data = sklearn.datasets.load_breast_cancer(return_X_y=False)
data = pd.DataFrame(cancer_data['data'], columns=cancer_data['feature_names'])
data['target'] = cancer_data['target']

gbm_options = {
    'num_boost_round': 100,
    'num_leaves': ag.space.Int(lower=26, upper=66, default=36),
}

hyperparameters = {
    'GBM': gbm_options,
}

hp_tune = True
time_limits = 2 * 60
num_trials = 3
output_directory = './agModels--tuningCancerDataset'
data = task.Dataset(data)
label_column = 'target'
search_strategy = 'hyperband'
predictor = task.fit(
    train_data=data,
    holdout_frac=0.3,
    label=label_column,
    output_directory=output_directory,
    time_limits=time_limits,
    num_trials=num_trials,
    hyperparameter_tune=hp_tune,
    hyperparameters=hyperparameters,
    search_strategy=search_strategy,
    auto_stack=False,
    stack_ensemble_levels=0
)
```

The error: 
```
File "/Users/rmelikbe/Desktop/repos/autogluon/autogluon/utils/tabular/ml/trainer/abstract_trainer.py", line 354, in train_single_full
    Y_train=y_train, Y_test=y_test, scheduler_options=(self.scheduler_func, self.scheduler_options), verbosity=self.verbosity)
  File "/Users/rmelikbe/Desktop/repos/autogluon/autogluon/utils/tabular/ml/models/lgb/lgb_model.py", line 285, in hyperparameter_tune
    scheduler = scheduler_func(lgb_trial, **scheduler_options)
  File "/Users/rmelikbe/Desktop/repos/autogluon/autogluon/scheduler/hyperband.py", line 136, in __init__
    reward_attr=reward_attr, visualizer=visualizer, dist_ip_addrs=dist_ip_addrs)
  File "/Users/rmelikbe/Desktop/repos/autogluon/autogluon/scheduler/fifo.py", line 91, in __init__
    self.searcher = searcher_factory(searcher, **kwargs)
  File "/Users/rmelikbe/Desktop/repos/autogluon/autogluon/searcher/searcher_factory.py", line 14, in searcher_factory
    raise AssertionError("name = '{}' not supported".format(name))
AssertionError: name = 'hyperband' not supported
```


Discussion:
What is the output of the assertion message?

hyperband is not a search strategy, but a scheduling strategy which can be run with different underlying searchers. Currently supported are 'random' (default), 'skopt', 'grid'. You probably want to keep it at 'random' (as this is what gives you *the* Hyperband method as published).

In the near future, there will be more searchers being supported here, but it may take a few months still.
I am personally not much familiar how a call to task.fit maps to the creation of a scheduler. When you create a scheduler (FIFOScheduler, HyperbandScheduler), you can pass searcher and search_options, and searcher -> name in searcher_factory. You seem to be using search_strategy above ...
Hi @mseeger yep, I was using `hyperband` as a `search_strategy`. I wanted to use `HyperbandScheduler` behind the scenes. If it is not supported at this moment, it would be correct to remove it from docstring: https://github.com/awslabs/autogluon/blob/a1347ea81ff678a6047cc1723c32fcf0dc81bbf2/autogluon/task/tabular_prediction/tabular_prediction.py#L179

Hello,

this is not the problem here.
If you look into tabular_predicition.py you see that towards the end, when scheduler_options are set, you have
   searcher = search_strategy

This is wrong for Hyperband. HB is not a searcher, but a scheduler, and you want searcher = 'random' if search_strategy is 'hyperband'. This would fix your issue.

I still think that code should be refactored, because it mixes searcher and scheduler, something we need to keep apart for future use cases. @Innixma ?

One complexity here is that now every searcher can be used with the two different schedulers we have, namely FIFO and Hyperband. Currently, all searchers can be used with FIFO, but only 'random' can be used with Hyperband. But this will change at some point ...
In fact, let me try and fix this for now. I'll do a pull request and notify you.

Resolving patch:
diff --git a/autogluon/task/tabular_prediction/tabular_prediction.py b/autogluon/task/tabular_prediction/tabular_prediction.py
--- a/autogluon/task/tabular_prediction/tabular_prediction.py
+++ b/autogluon/task/tabular_prediction/tabular_prediction.py
@@ -272,7 +272,7 @@ def fit(train_data, label, tuning_data=None, output_directory=None, problem_type
             dist_ip_addrs = []
 
         if search_options is None:
-            search_options = {}
+            search_options = dict()
 
         if hyperparameters is None:
             hyperparameters = {
@@ -355,7 +355,15 @@ def fit(train_data, label, tuning_data=None, output_directory=None, problem_type
         }
         if isinstance(search_strategy, str):
             scheduler = schedulers[search_strategy.lower()]
+            # This is a fix for now. But we need to separate between scheduler
+            # (mainly FIFO and Hyperband) and searcher. Currently, most searchers
+            # only work with FIFO, and Hyperband works only with random searcher,
+            # but this will be different in the future.
+            if search_strategy == 'hyperband':
+                # Currently, HyperbandScheduler only supports random searcher
+                scheduler_options['searcher'] = 'random'
         else:
+            # TODO: Check that search_strategy is a subclass of TaskScheduler
             assert callable(search_strategy)
             scheduler = search_strategy
             scheduler_options['searcher'] = 'random'


Test patch: