Repository: sktime/sktime

Issue:
[ENH] implement get_fitted_params for tuning forecasters
Tuning forecasters like `ForecastingGridSearchCV` should implement a `get_fitted_params` method.
Note that here, by applying a tuning method, the hyperparameters that are tuned become "normal" (fitted) parameters.

The `get_fitted_params` of tuning should return two things in its dictionary:

* the hyper-parameters that are tuned, with their tuned values
* either: the "best estimator" in its fitted state; or, better, parameters of the "best estimator" with a nesting structure like for `sklearn` `get_params`  (this might be slightly fiddly.

For implementation, it's fine to start with one of the two.

See also discussion in #1486, FYI @boukepostma and @asksteph.

FYI @aiwalter.


Discussion:
Sounds like a pretty interesting small project, can I claim the ticket?
all yours, @ZiyaoWei !
small note of caution: `get_fitted_params` is not subject to the "plumbing" layer, i.e., you still have to write boilerplate.

The boilerplate is simple:

```python
if not self.is_fitted:
    raise NotFittedError
```
So I finally got a bit of time to look into this and it seems `get_fitted_params()` has a implementation in `BaseGridSearch` which just returns the underlying best params - if we change the implementation should we worry about compatibility with existing usages?

Also the only tuning forecasters I see are the ones in _tune.py (`ForecastingGridSearchCV` and `ForecastingRandomizedSearchCV`) - do we have more?
>  if we change the implementation should we worry about compatibility with existing usages?

not if it returns everything in the expected `dict` format.

>Also the only tuning forecasters I see are the ones in _tune.py (`ForecastingGridSearchCV` and `ForecastingRandomizedSearchCV`) - do we have more?

These are the only ones I know.

Resolving patch:
diff --git a/sktime/forecasting/model_selection/_tune.py b/sktime/forecasting/model_selection/_tune.py
--- a/sktime/forecasting/model_selection/_tune.py
+++ b/sktime/forecasting/model_selection/_tune.py
@@ -13,6 +13,7 @@
 from joblib import Parallel, delayed
 from sklearn.model_selection import ParameterGrid, ParameterSampler, check_cv
 
+from sktime.exceptions import NotFittedError
 from sktime.forecasting.base._delegate import _DelegatedForecaster
 from sktime.forecasting.model_evaluation import evaluate
 from sktime.utils.validation.forecasting import check_scoring
@@ -68,6 +69,30 @@ def __init__(
 
     _delegate_name = "best_forecaster_"
 
+    def get_fitted_params(self):
+        """Get fitted parameters.
+
+        Returns
+        -------
+        fitted_params : dict
+            A dict containing the best hyper parameters and the parameters of
+            the best estimator (if available), merged together with the former
+            taking precedence.
+        """
+        if not self.is_fitted:
+            raise NotFittedError
+
+        if self._is_vectorized:
+            return {"forecasters_": self.forecasters_}
+
+        fitted_params = {}
+        try:
+            fitted_params = self.best_forecaster_.get_fitted_params()
+        except NotImplementedError:
+            pass
+        fitted_params = {**fitted_params, **self.best_params_}
+        return fitted_params
+
     def _run_search(self, evaluate_candidates):
         raise NotImplementedError("abstract method")
 
@@ -200,7 +225,7 @@ class ForecastingGridSearchCV(BaseGridSearch):
     """Perform grid-search cross-validation to find optimal model parameters.
 
     The forecaster is fit on the initial window and then temporal
-    cross-validation is used to find the optimal parameter
+    cross-validation is used to find the optimal parameter.
 
     Grid-search cross-validation is performed based on a cross-validation
     iterator encoding the cross-validation scheme, the parameter grid to
@@ -251,7 +276,7 @@ class ForecastingGridSearchCV(BaseGridSearch):
     cv_results_ : dict
         Results from grid search cross validation
     n_splits_: int
-        Number of splits in the data for cross validation}
+        Number of splits in the data for cross validation
     refit_time_ : float
         Time (seconds) to refit the best forecaster
     scorer_ : function


Test patch:
diff --git a/sktime/forecasting/model_selection/tests/test_tune.py b/sktime/forecasting/model_selection/tests/test_tune.py
--- a/sktime/forecasting/model_selection/tests/test_tune.py
+++ b/sktime/forecasting/model_selection/tests/test_tune.py
@@ -47,23 +47,18 @@ def _get_expected_scores(forecaster, cv, param_grid, y, X, scoring):
     return scores
 
 
-def _check_cv(forecaster, gscv, cv, param_grid, y, X, scoring):
-    actual = gscv.cv_results_[f"mean_test_{scoring.name}"]
+def _check_cv(forecaster, tuner, cv, param_grid, y, X, scoring):
+    actual = tuner.cv_results_[f"mean_test_{scoring.name}"]
 
     expected = _get_expected_scores(forecaster, cv, param_grid, y, X, scoring)
     np.testing.assert_array_equal(actual, expected)
 
     # Check if best parameters are selected.
-    best_idx = gscv.best_index_
+    best_idx = tuner.best_index_
     assert best_idx == actual.argmin()
 
-    best_params = gscv.best_params_
-    assert best_params == param_grid[best_idx]
-
-    # Check if best parameters are contained in best forecaster.
-    best_forecaster_params = gscv.best_forecaster_.get_params()
-    best_params = gscv.best_params_
-    assert best_params.items() <= best_forecaster_params.items()
+    fitted_params = tuner.get_fitted_params()
+    assert param_grid[best_idx].items() <= fitted_params.items()
 
 
 NAIVE = NaiveForecaster(strategy="mean")