Repository: sktime/sktime

Issue:
Implement Multiplexer
**Is your feature request related to a problem? Please describe.**
Arising from the call on Feb 2 on forecasting model selection & evaluation.

The multiplexer is a component that allows easy tuning for selection over a list of estimators with potentially distinct parameters

**Describe the solution you'd like**
The high-behaviour is as follows.

The multiplexer takes a named list (dict) of components at initialization, `components`.
It also has a single `select` parameter that should be identical to one of the components´ names.

The behaviour of the estimator is identical to the component `components[select]`, the other components are ignored.

Why is this useful:
this is arising from a suggestion by @aiwalter that model selection should be possible over inhomogenous (not the same) estimators.

Better than the "naive" solution to write a custom grid search and random search is the solution to wrap a multiplexer in a tuner - this way, model selection can be done using any evaluation strategy, e.g., grid search, random search, Bayesian optimization, etc.

**Usage and implementation sketch**

```python
class Multiplexer
    def __init__(self, components, select=None):
        self.components = components
        self.select = select
        
    def fit(self, ...):
        self.components[select].fit(...)

    def predict(self, ...):
        self.components[select].predict(...)
        
    def update(self, ...):
        self.components[select].update(...)
```

To get model selection, one would then write the following:

```python
mpx = Multiplexer({"ARIMA": ARIMA(sth), "ExpS": ExpSmoothing(sth)])
grid = {"select": ["ARIMA", "ExpSmoothing"]}
gscv = ForecastingGridSearchCV(mpx, grid, ...)
```
A technical challenge might be to check whether this works well with nested `get_params`/`set_params`.
Easy solution might just be to "unpack" the list in the constructor.



Discussion:
thx @fkiraly  for writing this idea down here! I might work on it, but currently I have other topics with a higher priority which I would like to add to `sktime` before :)
Hey everyone, I can take on doing this one? @mloning 

yes sure @koralturkk 

Resolving patch:
diff --git a/sktime/forecasting/base/_meta.py b/sktime/forecasting/base/_meta.py
--- a/sktime/forecasting/base/_meta.py
+++ b/sktime/forecasting/base/_meta.py
@@ -27,7 +27,11 @@ def __init__(self, forecasters, n_jobs=None):
         super(_HeterogenousEnsembleForecaster, self).__init__()
 
     def _check_forecasters(self):
-        if self.forecasters is None or len(self.forecasters) == 0:
+        if (
+            self.forecasters is None
+            or len(self.forecasters) == 0
+            or not isinstance(self.forecasters, list)
+        ):
             raise ValueError(
                 "Invalid 'estimators' attribute, 'estimators' should be a list"
                 " of (string, estimator) tuples."
diff --git a/sktime/forecasting/compose/__init__.py b/sktime/forecasting/compose/__init__.py
--- a/sktime/forecasting/compose/__init__.py
+++ b/sktime/forecasting/compose/__init__.py
@@ -14,6 +14,7 @@
     "RecursiveTimeSeriesRegressionForecaster",
     "ReducedForecaster",
     "StackingForecaster",
+    "MultiplexForecaster",
 ]
 
 from sktime.forecasting.compose._ensemble import EnsembleForecaster
@@ -25,3 +26,4 @@
 from sktime.forecasting.compose._reduce import RecursiveTimeSeriesRegressionForecaster
 from sktime.forecasting.compose._reduce import ReducedForecaster
 from sktime.forecasting.compose._stack import StackingForecaster
+from sktime.forecasting.compose._multiplexer import MultiplexForecaster
diff --git a/sktime/forecasting/compose/_multiplexer.py b/sktime/forecasting/compose/_multiplexer.py
new file mode 100644
--- /dev/null
+++ b/sktime/forecasting/compose/_multiplexer.py
@@ -0,0 +1,143 @@
+#!/usr/bin/env python3 -u
+# -*- coding: utf-8 -*-
+# copyright: sktime developers, BSD-3-Clause License (see LICENSE file)
+
+from sktime.forecasting.base._meta import _HeterogenousEnsembleForecaster
+from sktime.forecasting.base._base import DEFAULT_ALPHA
+from sktime.forecasting.base._sktime import _OptionalForecastingHorizonMixin
+from sklearn.base import clone
+
+__author__ = ["Kutay Koralturk"]
+__all__ = ["MultiplexForecaster"]
+
+
+class MultiplexForecaster(
+    _OptionalForecastingHorizonMixin, _HeterogenousEnsembleForecaster
+):
+    def __init__(
+        self,
+        forecasters: list,
+        selected_forecaster=None,
+    ):
+
+        """
+        MultiplexForecaster facilitates a framework for performing
+        model selection process over different model classes.
+        It should be used in conjunction with ForecastingGridSearchCV
+        to get full utilization.
+
+        Single use of MultiplexForecaster with forecasters
+        and selected_forecaster parameter specified,
+        works just like the selected component.
+        It does not provide any further use in that case.
+
+        When used with ForecastingGridSearchCV, MultiplexForecaster
+        provides an ability to compare different model class
+        performances with each other, just like a model tournament.
+        When ForecastingGridSearchCV is fitted with a MultiplexForecaster,
+        returned value for the selected_forecaster argument of best_params_
+        attribute of ForecastingGridSearchCV, gives the best
+        performing model class among given models provided in forecasters.
+
+        Parameters
+        ----------
+        forecasters : list
+            List of (forecaster names, forecaster objects)
+            MultiplexForecaster switches between these forecasters
+            objects when used with ForecastingGridSearchCV to
+            find the optimal model
+        selected_forecaster: str
+            An argument to make a selection among forecasters.
+            MultiplexForecaster uses selected_forecaster
+            to choose which component to fit.
+            Important for using with ForecastingGridSearchCV as a
+            hyperparameter.
+
+        Attributes
+        ----------
+        _forecaster : Sktime forecaster
+            forecaster that MultiplexForecaster will currently
+            forecast with.
+        """
+
+        super(MultiplexForecaster, self).__init__(forecasters=forecasters, n_jobs=None)
+        self.selected_forecaster = selected_forecaster
+        self._forecaster = None
+
+    def _check_fit_params(self, fit_params):
+        forecaster_fit_params = {}
+
+        if self.selected_forecaster is None or fit_params == {}:
+            return forecaster_fit_params
+
+        for component in self.forecasters:
+            name, _ = component
+            if name not in fit_params.keys():
+                raise KeyError(
+                    "If you provide fit_params for models "
+                    "dictionary key of fit params need to "
+                    " match the associated component key."
+                )
+
+        if self.selected_forecaster in fit_params.keys():
+            forecaster_fit_params = fit_params[self.selected_forecaster]
+            return forecaster_fit_params
+        else:
+            return forecaster_fit_params
+
+    def _check_selected_forecaster(self):
+        component_names = [name for name, _ in self.forecasters]
+        if self.selected_forecaster not in component_names:
+            raise Exception(
+                "Please check the selected_forecaster argument provided "
+                " Valid selected_forecaster parameters: {}".format(component_names)
+            )
+
+    def _set_forecaster(self):
+        self._check_selected_forecaster()
+        if self.selected_forecaster is not None:
+            for name, forecaster in self.forecasters:
+                if self.selected_forecaster == name:
+                    self._forecaster = clone(forecaster)
+
+    def fit(self, y, X=None, fh=None, **fit_params):
+        """Fit to training data.
+
+        Parameters
+        ----------
+        y : pd.Series
+            Target time series to which to fit the forecaster.
+        fh : int, list or np.array, optional (default=None)
+            The forecasters horizon with the steps ahead to to predict.
+        X : pd.DataFrame, optional (default=None)
+            Exogenous variables are ignored
+        fit_params : dict
+            A dictionary composed of key-value pairs
+            of forecaster names and fit params to be
+            used for each forecaster.
+            Example: {"ARIMA": ..., "ETS": ...}
+        Returns
+        -------
+        self : returns an instance of self.
+        """
+
+        self._set_y_X(y, X)
+        self._set_fh(fh)
+        self._check_forecasters()
+        self._set_forecaster()
+        forecaster_fit_params = self._check_fit_params(fit_params=fit_params)
+        self._forecaster.fit(y, X=X, fh=fh, **forecaster_fit_params)
+        self._is_fitted = True
+        return self
+
+    def _predict(self, fh, X=None, return_pred_int=False, alpha=DEFAULT_ALPHA):
+        return self._forecaster.predict(
+            fh, X, return_pred_int=return_pred_int, alpha=alpha
+        )
+
+    def update(self, y, X=None, update_params=True):
+        """Call predict on the forecaster with the best found parameters. """
+        self.check_is_fitted()
+        self._update_y_X(y, X)
+        self._forecaster.update(y, X, update_params=update_params)
+        return self


Test patch:
diff --git a/sktime/tests/_config.py b/sktime/tests/_config.py
--- a/sktime/tests/_config.py
+++ b/sktime/tests/_config.py
@@ -74,6 +74,7 @@
 from sktime.transformations.series.adapt import TabularToSeriesAdaptor
 from sktime.transformations.series.detrend import Detrender
 from sktime.transformations.series.impute import Imputer
+from sktime.forecasting.compose._multiplexer import MultiplexForecaster
 from sktime.transformations.series.outlier_detection import HampelFilter
 
 
@@ -181,6 +182,14 @@
         "max_q": 2,
         "seasonal": False,
     },
+    MultiplexForecaster: {
+        "forecasters": [
+            ("Naive_mean", NaiveForecaster(strategy="mean")),
+            ("Naive_last", NaiveForecaster(strategy="last")),
+            ("Naive_drift", NaiveForecaster(strategy="drift")),
+        ],
+        "selected_forecaster": "Naive_mean",
+    },
     ShapeletTransformClassifier: {"n_estimators": 3, "time_contract_in_mins": 0.125},
     ContractedShapeletTransform: {"time_contract_in_mins": 0.125},
     ShapeletTransform: {