Repository: sktime/sktime
Pull request #7268 (2024-10-15 11:41:21)

& talat-khattatov [ENH] box-cox bias adjustment for forecasters
I have added adjust_bias parameter in BoxCox transformation in accordance with issue #2391 

Closes #2391 by following option 1 in #7287.

fkiraly: I also note that the method used now is not the one described here:
https://otexts.com/fpp2/transformations.html

The formula in the text uses the *forecast variance*, but we are using the *sample variance*. This may make sense if we are dealing with residuals, but it is not the same algorithm, this should be explained.

For the bias adjustment as in the reference, we need access to forecast variances. This is only possible if we are wrapping a forecaster, so the type of the bias-adjusted box-cox-transform for forecasts for the algorithm in the reference needs to be a forecaster as well. (which may sound counterintuitive)
sanskarmodi8: > I also note that the method used now is not the one described here: https://otexts.com/fpp2/transformations.html
> 
> The formula in the text uses the _forecast variance_, but we are using the _sample variance_. This may make sense if we are dealing with residuals, but it is not the same algorithm, this should be explained.
> 
> For the bias adjustment as in the reference, we need access to forecast variances. This is only possible if we are wrapping a forecaster, so the type of the bias-adjusted box-cox-transform for forecasts for the algorithm in the reference needs to be a forecaster as well. (which may sound counterintuitive)

Hi @fkiraly ,

Thank you for the feedback and for pointing out the difference between using sample variance and forecast variance. I am sorry I did not realized that my approach wasn’t aligned with the method described in the reference, and I appreciate your clarification. 

As a beginner, I find working with forecast variances challenging at this stage. I’ll close the PR for now and take this as a learning opportunity.

Thank you again for your guidance.
fkiraly: @sanskarmodi8, no reason to close! This was a nice start! We could take the original algorithm for debiasing if you have a reference for it?
sanskarmodi8: Actually I don't have a reference for that.
I originally implemented the bias adjustment based on the mean of the exponential of the transformed values, thinking it would work well but then I understood that this approach might not be suitable for values of 𝜆 other than 0.
Then I researched several methods but didn't found one completely suitable to be used here. That's why I closed the PR.
fkiraly: There was another request for this feature on discord, I am trying to consolidate the discussion around how it should look like, here: https://github.com/sktime/sktime/issues/7287

It is quite interesting, it seems to be a hard problem to come up with a good design. Apologies for not noticing earlier that it did not fit the transformation API due to use of the `predict_var` output!
sanskarmodi8: > There was another request for this feature on discord, I am trying to consolidate the discussion around how it should look like, here: #7287
> 
> It is quite interesting, it seems to be a hard problem to come up with a good design. Apologies for not noticing earlier that it did not fit the transformation API due to use of the `predict_var` output!

Thanks for pointing me to #7287. I’ll definitely take a look at the discussion in #7287.
fkiraly: changed this to draft as it is not ready yet, but the code here might still be useful to find the right solution! As said, this PR shows that #7287 is not an easy problem to solve, design-wise.
sanskarmodi8: I have now added the code for BoxCoxBiasAdjustedForecaster as a forecaster wrapper based on the option 1 we discussed in #7287 
sanskarmodi8: > This is great!
> 
> Two blocking requests:
> 
> * the docstring above should move to the class, and also please add the new class to the forecasting API reference, `docs/source/api_reference/forecasting.rst`
> * I made a suggestion about making the limit lambda=0 more numerically stable. Let me know if you need help in getting the right decomposition (e.g., if naive Taylor expansion does not work)

Thanks for the review. I'm working on it.
fkiraly: oh, I also discovered something else. Due to the use of `_DelegatedForecaster` as a parent, the proba methods will not behave as expected. Also, the tags are not properly transferred in `__init__`.

I would suggest:

* do *not* inherit from `_DelegatedForecaster`, that will restore the proba methods to the usual defaulting which should be correct.
* in `__init__`, we can use `_DelegatedForecaster._set_delegated_tags`, with the `forecaster` as a target, which will take care of the tag cloning.

This is slightly messy, but the easiest way I can think of to take care of tags, and ensure the proba methods remain correct.
sanskarmodi8: > I also tried to check the Taylor expansion. I do get the derivatives, but I think the pre-factors are off by one inside the factorial. E.g., should it not be simply lambda, and lambda-squared divided by two, instead of lambda divided by two, and lambda-squared divided by six?

I'm sorry for this mistake. It should be like below right?

```python
def _apply_bias_adjustment(self, y, variance):
    """Apply bias adjustment using Taylor expansion around λ = 0.

    The bias adjustment is calculated using a Taylor series expansion of the
    Box-Cox transformation around λ = 0.

    For the Box-Cox transformation:
    g(x, λ) = (x^λ - 1)/λ  for λ ≠ 0
    g(x, λ) = log(x)       for λ = 0

    The Taylor expansion around λ = 0 gives:
    g(x, λ) ≈ log(x) + λ * (log(x))^2/2 + λ^2 * (log(x))^3/6 + O(λ^3)

    Parameters
    ----------
    y : pd.DataFrame
        Predictions in the original scale.
    variance : pd.DataFrame
        Variance of predictions in the transformed scale.

    Returns
    -------
    y_adjusted : pd.DataFrame
        Bias-adjusted predictions.
    """
    lmbda = self.boxcox_transformer_.lmbda_
    log_y = np.log(y)

    first_order = 0.5 * variance
    second_order = lmbda * variance * (1 - 2 * log_y)
    third_order = (lmbda**2 / 2) * variance * (log_y**2 - 2 * log_y + 1)

    adjustment_factor = 1 + first_order + second_order + third_order
    y_adjusted = y * adjustment_factor

    return y_adjusted
```
fkiraly: > It should be like below right?

Something like this, I think this is closer to the truth, but I feel this is also not correct - have you tried to verify with Wolfram, or do you have a piece of paper with your steps to check? Maybe I also made a mistake.
fkiraly: Instead of deriving the full decomposition, I would also suggest to use this numerically stable function from `scipy`, and find an expression for the bias adjustment that shoves the numerical instability in it - the Taylor expansion strategy should work for that, i.e., isolate the power transform expression and then use the scipy method for it:

https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.boxcox.html

Patch:
diff --git a/.all-contributorsrc b/.all-contributorsrc
index bd1a81718..325456df5 100644
--- a/.all-contributorsrc
+++ b/.all-contributorsrc
@@ -3155,6 +3155,25 @@
         "example",
         "tutorial"
       ]
-    }
+    },
+    {
+      "login": "jusssch",
+      "name": "Julius Schmid",
+      "avatar_url": "https://avatars.githubusercontent.com/jusssch",
+      "profile": "https://github.com/jusssch",
+      "contributions": [
+        "code"
+      ]
+    },
+    {
+      "login": "manolotis",
+      "name": "Manuel Muñoz Sánchez",
+      "avatar_url": "https://avatars.githubusercontent.com/manolotis",
+      "profile": "https://github.com/manolotis",
+      "contributions": [
+        "doc"
+      ]
+     }
+
   ]
 }
diff --git a/README.md b/README.md
index 480fa793f..5405d5974 100644
--- a/README.md
+++ b/README.md
@@ -1,6 +1,7 @@
-<a href="https://www.sktime.net"><img src="https://github.com/sktime/sktime/blob/main/docs/source/images/sktime-logo.svg" width="175" align="right" /></a>
 
-# Welcome to sktime
+## Welcome to sktime
+
+<a href="https://www.sktime.net"><img src="https://github.com/sktime/sktime/blob/main/docs/source/images/sktime-logo.svg" width="175" align="right" /></a>
 
 > A unified interface for machine learning with time series
 
diff --git a/docs/source/api_reference/detection.rst b/docs/source/api_reference/detection.rst
index 4558d4022..4fe40aac1 100644
--- a/docs/source/api_reference/detection.rst
+++ b/docs/source/api_reference/detection.rst
@@ -97,6 +97,14 @@ Time Series Segmentation
 
     ClusterSegmenter
 
+.. currentmodule:: sktime.annotation.bs.BinarySegmentation
+
+.. autosummary::
+    :toctree: auto_generated/
+    :template: class.rst
+
+    BinarySegmentation
+
 Time Series Anomaly Detection
 -----------------------------
 
diff --git a/docs/source/api_reference/forecasting.rst b/docs/source/api_reference/forecasting.rst
index d951367bb..8610df650 100644
--- a/docs/source/api_reference/forecasting.rst
+++ b/docs/source/api_reference/forecasting.rst
@@ -211,6 +211,18 @@ Wrappers that add prediction intervals to any forecaster.
     EnbPIForecaster
 
 
+Calibration and bias adjustment
+-------------------------------
+
+.. currentmodule:: sktime.forecasting.boxcox_bias_adjusted_forecaster
+
+.. autosummary::
+    :toctree: auto_generated/
+    :template: class.rst
+
+    BoxCoxBiasAdjustedForecaster
+
+
 Trend forecasters
 -----------------
 
@@ -337,6 +349,14 @@ All "ARIMA" and "Auto-ARIMA" models below include SARIMAX capability.
 
     VARMAX
 
+.. currentmodule:: sktime.forecasting.vecm
+
+.. autosummary::
+    :toctree: auto_generated/
+    :template: class.rst
+
+    VECM
+
 Auto-ARIMA models
 ~~~~~~~~~~~~~~~~~
 
diff --git a/docs/source/changelog.rst b/docs/source/changelog.rst
index 8b7180b15..ea8a7474b 100644
--- a/docs/source/changelog.rst
+++ b/docs/source/changelog.rst
@@ -90,7 +90,8 @@ Core interface changes
 ~~~~~~~~~~~~~~~~~~~~~~
 
 Performance metrics for probabilistic forecasting, for interval and quantile returns,
-can now optionally be constructed with an ``alpha`` (quantile forecasts) or ``coverage`` parameter.
+can now optionally be constructed with an ``alpha`` (quantile forecasts) or ``coverage`` (interval forecasts) parameter,
+for example ``EmpiricalCoverage(coverage=0.7)`` for "empirical coverage at nominal interval coverage 70 percent".
 
 These are for use with benchmarking or tuning interfaces, where a metric is provided,
 but ``predict_interval`` or ``predict_quantiles`` are not explicitly called.
@@ -98,6 +99,9 @@ but ``predict_interval`` or ``predict_quantiles`` are not explicitly called.
 In such a case, the parameter of the metric will be used by the tuner or benchmark utility to
 tune with the metric at that quantile ``alpha`` or interval ``coverage``.
 
+For example, ``ForecastingGridSearchCV(fcst, cv, scoring=PinballLoss(alpha=[0.3, 0.7]))``,
+to tune the forecaster ``fcst`` scored by the pinball loss at quantiles 0.3 and 0.7.
+
 Enhancements
 ~~~~~~~~~~~~
 
@@ -148,7 +152,7 @@ Transformations
 ^^^^^^^^^^^^^^^
 
 * [ENH] ``HOG1D`` Transformer add new test parameter set to ``get_test_params`` (:pr:`7183`) :user:`Humorloos`
-* [ENH] Added second parameters example to ``get_test_params ``of Filter (:pr:`7178`) :user:`KarlKolibri`
+* [ENH] Added second parameters example to ``get_test_params`` of ``Filter`` (:pr:`7178`) :user:`KarlKolibri`
 * [ENH] temporal radial basis function feature transformer (:pr:`7261`) :user:`phoeenniixx`
 
 Test framework
diff --git a/examples/02_classification.ipynb b/examples/02_classification.ipynb
index 00b2e0df0..0a5f05201 100644
--- a/examples/02_classification.ipynb
+++ b/examples/02_classification.ipynb
@@ -32,7 +32,7 @@
     "\n",
     "- Example: Daily energy consumption profile over time - Predict season, e.g., winter/summer, or type of consumer\n",
     "\n",
-    "Regression = try to assign one *category* per time series, after training on time series/category examples\n",
+    "Regression = try to assign one continuous *numerical value* per time series, after training on time series/category examples\n",
     "\n",
     "- Example: Temperature/pressure/time profile of chemical reactor - Predict total purity (fraction of 1)\n",
     "\n",
diff --git a/sktime/alignment/utils/utils_align.py b/sktime/alignment/utils/utils_align.py
index a0db08d78..cef8d68a4 100644
--- a/sktime/alignment/utils/utils_align.py
+++ b/sktime/alignment/utils/utils_align.py
@@ -25,8 +25,8 @@ def reindex_iloc(df, inds, copy=True):
         out of bound indices will result in np.nan values
         entries are references to original DataFrame if copy=False
 
-    Example
-    -------
+    Examples
+    --------
     >>> X = pd.DataFrame({'a' : [1,2,3,4]}, index=[-4,7,11,14])
     >>> reindex_iloc(X, [1, 2, 6])
          a
@@ -56,8 +56,8 @@ def convert_align_to_align_loc(align, X, align_name="align", df_name="X", copy=T
     pd.DataFrame in alignment format, with columns 'ind'+str(i) for integer i
         cols contain loc index of X[i] mapped to alignment coordinate for alignment
 
-    Example
-    -------
+    Examples
+    --------
     align_df = pd.DataFrame({'ind0' : [1,2,3], 'ind1' : [0,2,4]})
     X = [pd.DataFrame({'a' : [1,2,3,4]}, index=[-4,7,11,14]),
             pd.DataFrame({'a' : [1,2,3,5,6]}, index=[4,8,12,16,20])
diff --git a/sktime/annotation/igts.py b/sktime/annotation/igts.py
index c64ebd9b6..9f0df5add 100644
--- a/sktime/annotation/igts.py
+++ b/sktime/annotation/igts.py
@@ -152,8 +152,8 @@ class IGTS:
     activities.", Pervasive and Mobile Computing, 38, 92-109, (2017).
     https://www.sciencedirect.com/science/article/abs/pii/S1574119217300081
 
-    Example
-    -------
+    Examples
+    --------
     >>> from sktime.annotation.datagen import piecewise_normal_multivariate
     >>> from sklearn.preprocessing import MinMaxScaler
     >>> X = piecewise_normal_multivariate(lengths=[10, 10, 10, 10],
diff --git a/sktime/base/_meta.py b/sktime/base/_meta.py
index d6f6abc4e..f6ad21185 100644
--- a/sktime/base/_meta.py
+++ b/sktime/base/_meta.py
@@ -726,8 +726,8 @@ def flatten(obj):
     list or tuple, tuple if obj was tuple, list otherwise
         flat iterable, containing non-list/tuple elements in obj in same order as in obj
 
-    Example
-    -------
+    Examples
+    --------
     >>> flatten([1, 2, [3, (4, 5)], 6])
     [1, 2, 3, 4, 5, 6]
     """
@@ -752,8 +752,8 @@ def unflatten(obj, template):
         has element bracketing exactly as `template`
             and elements in sequence exactly as `obj`
 
-    Example
-    -------
+    Examples
+    --------
     >>> unflatten([1, 2, 3, 4, 5, 6], [6, 3, [5, (2, 4)], 1])
     [1, 2, [3, (4, 5)], 6]
     """
diff --git a/sktime/classification/deep_learning/cntc.py b/sktime/classification/deep_learning/cntc.py
index e7ed5b046..6238ac9c7 100644
--- a/sktime/classification/deep_learning/cntc.py
+++ b/sktime/classification/deep_learning/cntc.py
@@ -116,14 +116,12 @@ class CNTCClassifier(BaseDeepClassifier):
         """Construct a compiled, un-trained, keras model that is ready for training.
 
         In sktime, time series are stored in numpy arrays of shape (d,m), where d
-        is the number of dimensions, m is the series length. Keras/tensorflow assume
-        data is in shape (m,d). This method also assumes (m,d). Transpose should
-        happen in fit.
+        is the number of dimensions, m is the series length.
 
         Parameters
         ----------
         input_shape : tuple
-            The shape of the data fed into the input layer, should be (m,d)
+            The shape of the data fed into the input layer, should be (d,m)
         n_classes: int
             The number of classes, which becomes the size of the output layer
 
@@ -230,8 +228,6 @@ class CNTCClassifier(BaseDeepClassifier):
         if self.callbacks is None:
             self._callbacks = []
         y_onehot = self._convert_y_to_keras(y)
-        # Transpose to conform to Keras input style.
-        X = X.transpose(0, 2, 1)
 
         check_random_state(self.random_state)
         self.input_shape = X.shape[1:]
@@ -275,8 +271,6 @@ class CNTCClassifier(BaseDeepClassifier):
         """
         import numpy as np
 
-        # Transpose to work correctly with keras
-        X = X.transpose((0, 2, 1))
         X2 = self.prepare_input(X)
         probs = self.model_.predict([X2, X, X], self.batch_size, **kwargs)
 
diff --git a/sktime/classification/kernel_based/_arsenal.py b/sktime/classification/kernel_based/_arsenal.py
index 2021b6645..1ed36200a 100644
--- a/sktime/classification/kernel_based/_arsenal.py
+++ b/sktime/classification/kernel_based/_arsenal.py
@@ -462,12 +462,20 @@ class Arsenal(BaseClassifier):
             ``create_test_instance`` uses the first (or only) dictionary in ``params``.
         """
         if parameter_set == "results_comparison":
-            params = {"num_kernels": 20, "n_estimators": 5}
-        else:
-            params = {
-                "num_kernels": 10,
-                "n_estimators": 2,
-                "save_transformed_data": True,
-            }
-
-        return params
+            return {"num_kernels": 20, "n_estimators": 5}
+
+        params0 = {
+            "num_kernels": 10,
+            "n_estimators": 2,
+            "save_transformed_data": True,
+        }
+        params1 = {
+            "num_kernels": 23,
+            "n_estimators": 20,
+            "rocket_transform": "minirocket",
+            "max_dilations_per_kernel": 28,
+            "n_features_per_kernel": 2,
+            "contract_max_n_estimators": 113,
+            "save_transformed_data": True,
+        }
+        return [params0, params1]
diff --git a/sktime/datasets/_single_problem_loaders.py b/sktime/datasets/_single_problem_loaders.py
index 907b8f3c6..35741c506 100644
--- a/sktime/datasets/_single_problem_loaders.py
+++ b/sktime/datasets/_single_problem_loaders.py
@@ -1543,8 +1543,8 @@ def load_m5(
     - dept_id
     - date
 
-    Example
-    -------
+    Examples
+    --------
     >>> data = load_m5()
     >>> data.head()
     """
diff --git a/sktime/datatypes/_adapter/gluonts.py b/sktime/datatypes/_adapter/gluonts.py
index e89b95619..d01e2e996 100644
--- a/sktime/datatypes/_adapter/gluonts.py
+++ b/sktime/datatypes/_adapter/gluonts.py
@@ -14,7 +14,7 @@ def convert_pandas_to_listDataset(pd_dataframe: pd.DataFrame):
     gluonts.dataset.common.ListDataset
         A gluonTS ListDataset formed from `pd_dataframe`
 
-    Example
+    Examples
     --------
     >>> import pandas as pd
     >>> cols = ["instances", "timepoints"] + [f"var_{i}" for i in range(2)]
diff --git a/sktime/forecasting/boxcox_bias_adjusted_forecaster.py b/sktime/forecasting/boxcox_bias_adjusted_forecaster.py
new file mode 100644
index 000000000..1efd30565
--- /dev/null
+++ b/sktime/forecasting/boxcox_bias_adjusted_forecaster.py
@@ -0,0 +1,223 @@
+# copyright: sktime developers, BSD-3-Clause License (see LICENSE file)
+
+"""BoxCoxBiasAdjustedForecaster implementation."""
+
+__author__ = ["sanskarmodi8"]
+
+import pandas as pd
+from scipy.special import inv_boxcox
+
+from sktime.forecasting.base import BaseForecaster
+from sktime.forecasting.base._delegate import _DelegatedForecaster
+from sktime.transformations.series.boxcox import BoxCoxTransformer
+
+
+class BoxCoxBiasAdjustedForecaster(BaseForecaster):
+    r"""Box-Cox Bias-Adjusted Forecaster.
+
+    This module implements a forecaster that applies Box-Cox transformation
+    and bias adjustment to the predictions of a wrapped forecaster.
+
+    The bias adjustment is applied to both point predictions and prediction
+    intervals using the bias correction formula for inverse Box-Cox transforms:
+
+    .. math::
+
+        y_t = \text{inv\_boxcox}(w_t, \lambda) \cdot [1 + \sigma_t^2(1-\lambda)/
+        (2(\lambda w_t + 1)^2)]
+
+    where:
+    - :math:`y_t` is the back-transformed forecast at time t
+    - :math:`w_t` is the transformed data at time t
+    - :math:`\lambda` is the Box-Cox parameter
+    - :math:`\sigma_t^2` is the forecast variance on the transformed scale at time t
+
+    Parameters
+    ----------
+    forecaster : BaseForecaster
+        The wrapped forecaster to which Box-Cox transformation and
+        bias adjustment will be applied.
+    lambda_fixed : float, optional (default=None)
+        The Box-Cox transformation parameter. If None, it will be estimated.
+
+    Notes
+    -----
+    This forecaster applies only to univariate, non-hierarchical inner forecasters.
+
+    References
+    ----------
+    .. [1] Hyndman, R.J., & Athanasopoulos, G. (2018) "Forecasting:
+           principles and practice", 2nd edition, Section 2.7 - Box-Cox Transformations,
+           OTexts: Melbourne, Australia. OTexts.com/fpp2
+    """
+
+    _tags = {
+        "capability:pred_int": True,
+        "capability:pred_var": True,
+    }
+
+    def __init__(self, forecaster, lambda_fixed=None):
+        self.forecaster = forecaster
+        self.lambda_fixed = lambda_fixed
+        self.boxcox_transformer_ = None
+        _DelegatedForecaster._set_delegated_tags(self, forecaster)
+        super().__init__()
+
+    def _get_delegate(self):
+        """Return the delegate forecaster."""
+        return self.forecaster
+
+    def _fit(self, y, X=None, fh=None):
+        """Fit the forecaster to the training data.
+
+        Parameters
+        ----------
+        y : pd.Series
+            Target time series to which to fit the forecaster.
+        X : pd.DataFrame, optional (default=None)
+            Exogenous variables.
+        fh : ForecastingHorizon
+            The forecasting horizon with the steps ahead to predict.
+
+        Returns
+        -------
+        self : returns an instance of self.
+        """
+        self.boxcox_transformer_ = BoxCoxTransformer(lambda_fixed=self.lambda_fixed)
+        y_transformed = self.boxcox_transformer_.fit_transform(y)
+
+        self.forecaster_ = self.forecaster.clone()
+        self.forecaster_.fit(y=y_transformed, X=X, fh=fh)
+
+        if not self.forecaster_.get_tag("capability:pred_int"):
+            raise ValueError(
+                "The wrapped forecaster must support prediction intervals "
+                "(capability:pred_int) to enable bias adjustment."
+            )
+
+        return self
+
+    def _predict(self, fh, X=None):
+        """Forecast time series at future horizon.
+
+        Parameters
+        ----------
+        fh : ForecastingHorizon
+            The forecasting horizon with the steps ahead to predict.
+        X : pd.DataFrame, optional (default=None)
+            Exogenous variables.
+
+        Returns
+        -------
+        y_pred : pd.Series
+            Bias-adjusted point predictions.
+        """
+        y_pred_transformed = self.forecaster_.predict(fh, X)
+        variance = self.forecaster_.predict_var(fh, X)
+
+        y_pred = self._apply_bias_adjustment(y_pred_transformed, variance)
+
+        return y_pred
+
+    def _predict_interval(self, fh, X=None, coverage=None):
+        """Compute prediction intervals for the forecasts.
+
+        Parameters
+        ----------
+        fh : ForecastingHorizon
+            The forecasting horizon with the steps ahead to predict.
+        X : pd.DataFrame, optional (default=None)
+            Exogenous variables.
+        coverage : list of float, optional (default=None)
+            Confidence levels for prediction intervals.
+
+        Returns
+        -------
+        pred_int : pd.DataFrame
+            Prediction intervals.
+        """
+        pred_int_transformed = self.forecaster_.predict_interval(fh, X, coverage)
+        columns = pred_int_transformed.columns
+        variance = self.forecaster_.predict_var(fh, X).squeeze()
+
+        result = pd.DataFrame(0, index=pred_int_transformed.index, columns=columns)
+
+        coverage_idx = 0
+        var_type_idx = 1
+        coverage_levels = columns.levels[coverage_idx].unique()
+
+        for coverage_level in coverage_levels:
+            level_mask = columns.get_level_values(coverage_idx) == coverage_level
+
+            lower_mask = level_mask & (
+                columns.get_level_values(var_type_idx) == "lower"
+            )
+            if any(lower_mask):
+                lower = pred_int_transformed.loc[:, lower_mask]
+                lower_adjusted = self._apply_bias_adjustment(lower, variance)
+                result.loc[:, lower_mask] = lower_adjusted
+
+            upper_mask = level_mask & (
+                columns.get_level_values(var_type_idx) == "upper"
+            )
+            if any(upper_mask):
+                upper = pred_int_transformed.loc[:, upper_mask]
+                upper_adjusted = self._apply_bias_adjustment(upper, variance)
+                result.loc[:, upper_mask] = upper_adjusted
+
+        result = result.sort_index(axis=1)
+
+        return result
+
+    def _apply_bias_adjustment(self, y, variance):
+        """Apply bias adjustment for BoxCox Transformations.
+
+        Parameters
+        ----------
+        y : pd.Series
+            Transformed predictions or prediction intervals.
+        variance : pd.Series
+            Variance of the predictions.
+
+        Returns
+        -------
+        y_adjusted : pd.Series
+            Bias-adjusted predictions or prediction intervals.
+        """
+        y_values = y.values.reshape(-1, 1)
+        variance_values = variance.values.reshape(-1, 1)
+        lmbda = self.boxcox_transformer_.lambda_
+
+        denominator = 2 * (lmbda * y_values + 1) ** 2
+        adjustment = 1 + (variance_values * (1 - lmbda)) / denominator
+        adjusted_values = inv_boxcox(y_values, lmbda) * adjustment
+
+        result = pd.Series(adjusted_values.ravel(), index=y.index, name=y.name)
+        return result
+
+    @classmethod
+    def get_test_params(cls, parameter_set="default"):
+        """Return testing parameter settings for the estimator.
+
+        Parameters
+        ----------
+        parameter_set : str, default="default"
+            Name of the set of test parameters to return.
+
+        Returns
+        -------
+        params : dict or list of dict
+            Parameters to create testing instances of the class.
+        """
+        from sktime.forecasting.naive import NaiveForecaster
+
+        params = [
+            {
+                "forecaster": NaiveForecaster(strategy="mean"),
+            },
+            {
+                "forecaster": NaiveForecaster(strategy="mean"),
+                "lambda_fixed": 0.5,
+            },
+        ]
+        return params
diff --git a/sktime/forecasting/compose/_reduce.py b/sktime/forecasting/compose/_reduce.py
index 40491edc7..61f78c2e6 100644
--- a/sktime/forecasting/compose/_reduce.py
+++ b/sktime/forecasting/compose/_reduce.py
@@ -2634,8 +2634,8 @@ class YfromX(BaseForecaster, _ReducerMixin):
         if there are 2 or less levels, "global" and "panel" result in the same
         if there is only 1 level (single time series), all three settings agree
 
-    Example
-    -------
+    Examples
+    --------
     >>> from sktime.datasets import load_longley
     >>> from sktime.split import temporal_train_test_split
     >>> from sktime.forecasting.compose import YfromX
diff --git a/sktime/forecasting/vecm.py b/sktime/forecasting/vecm.py
index 12487758b..f5cfa65ea 100644
--- a/sktime/forecasting/vecm.py
+++ b/sktime/forecasting/vecm.py
@@ -62,8 +62,8 @@ class VECM(_StatsModelsAdapter):
         2D ndarray/pd.DataFrame of size (any, neqs)
         Forecasted value of exog_coint
 
-    Example
-    -------
+    Examples
+    --------
     >>> from sktime.forecasting.vecm import VECM
     >>> from sktime.split import temporal_train_test_split
     >>> from sktime.forecasting.base import ForecastingHorizon
diff --git a/sktime/libs/README.md b/sktime/libs/README.md
index 294dc6ad6..87e60e782 100644
--- a/sktime/libs/README.md
+++ b/sktime/libs/README.md
@@ -14,6 +14,10 @@ This folder contains libraries directly distributed with, and maintained by, `sk
   Unofficial fork of abandoned package from June 2024 onwards,
   see [pykalman issue 109](https://github.com/pykalman/pykalman/issues/109).
 
+* `uni2ts` - a package implementing the MOIRAIForecaster. Unofficial fork of
+ the package with minimal dependencies and code specific to the forecaster.
+ Official package available at [pypi](https://pypi.org/project/uni2ts/).
+
 * `vmdpy` - a package implementing Variational Mode Decomposition.
   Official fork, `vmdpy` is maintained in `sktime` since August 2023.
 
diff --git a/sktime/performance_metrics/forecasting/_classes.py b/sktime/performance_metrics/forecasting/_classes.py
index d739d8057..5265e54bd 100644
--- a/sktime/performance_metrics/forecasting/_classes.py
+++ b/sktime/performance_metrics/forecasting/_classes.py
@@ -44,9 +44,12 @@ from sktime.performance_metrics.forecasting._functions import (
     median_squared_scaled_error,
     relative_loss,
 )
+from sktime.performance_metrics.forecasting.sample_weight._types import (
+    check_sample_weight_generator,
+)
 from sktime.utils.warnings import warn
 
-__author__ = ["mloning", "tch", "RNKuhns", "fkiraly"]
+__author__ = ["mloning", "tch", "RNKuhns", "fkiraly", "markussagen"]
 __all__ = [
     "make_forecasting_scorer",
     "MeanAbsoluteScaledError",
@@ -133,7 +136,11 @@ class BaseForecastingErrorMetric(BaseMetric):
         "reserved_params": ["multioutput", "multilevel"],
     }
 
-    def __init__(self, multioutput="uniform_average", multilevel="uniform_average"):
+    def __init__(
+        self,
+        multioutput="uniform_average",
+        multilevel="uniform_average",
+    ):
         self.multioutput = multioutput
         self.multilevel = multilevel
 
@@ -187,7 +194,7 @@ class BaseForecastingErrorMetric(BaseMetric):
             Must be of same format as ``y_true``, same columns if indexed,
             but not necessarily same indices.
 
-        sample_weight : optional, 1D array-like, default=None
+        sample_weight : optional, 1D array-like, or callable, default=None
             Sample weights for each time point.
 
             * If ``None``, the time indices are considered equally weighted.
@@ -198,6 +205,10 @@ class BaseForecastingErrorMetric(BaseMetric):
               individual time
               series must be the same, and equal to the length of ``sample_weight``,
               for all instances of time series passed.
+            * If a callable, it must follow ``SampleWeightGenerator`` interface,
+              or have one of the following signatures:
+              ``y_true: pd.DataFrame -> 1D array-like``,
+              or ``y_true: pd.DataFrame x y_pred: pd.DataFrame -> 1D array-like``.
 
         Returns
         -------
@@ -219,6 +230,18 @@ class BaseForecastingErrorMetric(BaseMetric):
         """  # noqa: E501
         return self.evaluate(y_true, y_pred, **kwargs)
 
+    def _apply_sample_weight_to_kwargs(self, y_true, y_pred, **kwargs):
+        """Apply sample weight to kwargs.
+
+        Sample weight is updated to kwargs if it is a callable and follows the
+        SampleWeightGenerator interface.
+        """
+        sample_weight = kwargs.get("sample_weight", None)
+        if callable(sample_weight) and check_sample_weight_generator(sample_weight):
+            kwargs["sample_weight"] = sample_weight(y_true, y_pred, **kwargs)
+
+        return kwargs
+
     def evaluate(self, y_true, y_pred, **kwargs):
         """Evaluate the desired metric on given inputs.
 
@@ -264,8 +287,8 @@ class BaseForecastingErrorMetric(BaseMetric):
             Must be of same format as ``y_true``, same columns if indexed,
             but not necessarily same indices.
 
-        sample_weight : optional, 1D array-like, default=None
-            Sample weights for each time point.
+        sample_weight : optional, 1D array-like, or callable, default=None
+            Sample weights or callable for each time point.
 
             * If ``None``, the time indices are considered equally weighted.
             * If an array, must be 1D.
@@ -275,6 +298,10 @@ class BaseForecastingErrorMetric(BaseMetric):
               individual time
               series must be the same, and equal to the length of ``sample_weight``,
               for all instances of time series passed.
+            * If a callable, it must follow ``SampleWeightGenerator`` interface,
+              or have one of the following signatures:
+              ``y_true: pd.DataFrame -> 1D array-like``,
+              or ``y_true: pd.DataFrame x y_pred: pd.DataFrame -> 1D array-like``.
 
         Returns
         -------
@@ -296,11 +323,16 @@ class BaseForecastingErrorMetric(BaseMetric):
         """  # noqa: E501
         multioutput = self.multioutput
         multilevel = self.multilevel
+
         # Input checks and conversions
         y_true_inner, y_pred_inner, multioutput, multilevel, kwargs = self._check_ys(
             y_true, y_pred, multioutput, multilevel, **kwargs
         )
 
+        kwargs = self._apply_sample_weight_to_kwargs(
+            y_true=y_true_inner, y_pred=y_pred_inner, **kwargs
+        )
+
         requires_vectorization = isinstance(y_true_inner, VectorizedDF)
         if not requires_vectorization:
             # pass to inner function
@@ -467,8 +499,8 @@ class BaseForecastingErrorMetric(BaseMetric):
             Must be of same format as ``y_true``, same columns if indexed,
             but not necessarily same indices.
 
-        sample_weight : optional, 1D array-like, default=None
-            Sample weights for each time point.
+        sample_weight : optional, 1D array-like, or callable, default=None
+            Sample weights or callable for each time point.
 
             * If ``None``, the time indices are considered equally weighted.
             * If an array, must be 1D.
@@ -478,6 +510,10 @@ class BaseForecastingErrorMetric(BaseMetric):
               individual time
               series must be the same, and equal to the length of ``sample_weight``,
               for all instances of time series passed.
+            * If a callable, it must follow ``SampleWeightGenerator`` interface,
+              or have one of the following signatures:
+              ``y_true: pd.DataFrame -> 1D array-like``,
+              or ``y_true: pd.DataFrame x y_pred: pd.DataFrame -> 1D array-like``.
 
         Returns
         -------
@@ -494,10 +530,16 @@ class BaseForecastingErrorMetric(BaseMetric):
         """  # noqa: E501
         multioutput = self.multioutput
         multilevel = self.multilevel
+
         # Input checks and conversions
         y_true_inner, y_pred_inner, multioutput, multilevel, kwargs = self._check_ys(
             y_true, y_pred, multioutput, multilevel, **kwargs
         )
+
+        kwargs = self._apply_sample_weight_to_kwargs(
+            y_true=y_true_inner, y_pred=y_pred_inner, **kwargs
+        )
+
         requires_vectorization = isinstance(y_true_inner, VectorizedDF)
         if not requires_vectorization:
             # pass to inner function
@@ -695,7 +737,7 @@ class BaseForecastingErrorMetric(BaseMetric):
 
         return y_true, y_pred, multioutput, multilevel, kwargs
 
-    def _get_sample_weight(self, **kwargs):
+    def _set_sample_weight_on_kwargs(self, **kwargs):
         """Get sample weights from kwargs.
 
         Assumes that either ``sample_weight`` is passed, or not.
@@ -736,7 +778,7 @@ class BaseForecastingErrorMetric(BaseMetric):
         df : pd.DataFrame
             Weighted DataFrame.
         """
-        sample_weight = self._get_sample_weight(**kwargs)
+        sample_weight = self._set_sample_weight_on_kwargs(**kwargs)
         if sample_weight is not None:
             df = df.mul(sample_weight, axis=0)
         return df
@@ -846,7 +888,6 @@ class _DynamicForecastingErrorMetric(BaseForecastingErrorMetricFunc):
         self.func = func
         self.name = name
         self.lower_is_better = lower_is_better
-
         super().__init__(multioutput=multioutput, multilevel=multilevel)
 
         self.set_tags(**{"lower_is_better": lower_is_better})
@@ -1581,13 +1622,6 @@ class MedianAbsoluteError(BaseForecastingErrorMetricFunc):
 
     func = median_absolute_error
 
-    def __init__(
-        self,
-        multioutput="uniform_average",
-        multilevel="uniform_average",
-    ):
-        super().__init__(multioutput=multioutput, multilevel=multilevel)
-
     def _evaluate_by_index(self, y_true, y_pred, **kwargs):
         """Return the metric evaluated at each time point.
 
diff --git a/sktime/performance_metrics/forecasting/probabilistic/_classes.py b/sktime/performance_metrics/forecasting/probabilistic/_classes.py
index 961a3b274..7aff5d314 100644
--- a/sktime/performance_metrics/forecasting/probabilistic/_classes.py
+++ b/sktime/performance_metrics/forecasting/probabilistic/_classes.py
@@ -55,7 +55,7 @@ class _BaseProbaForecastingErrorMetric(BaseForecastingErrorMetric):
                 (fh, n_outputs) where fh is the forecasting horizon
             Ground truth (correct) target values.
 
-        y_pred : return object of probabilistic predictition method scitype:y_pred
+        y_pred : return object of probabilistic prediction method scitype:y_pred
             must be at fh and for variables equal to those in y_true.
 
         Returns
@@ -82,7 +82,7 @@ class _BaseProbaForecastingErrorMetric(BaseForecastingErrorMetric):
                 (fh, n_outputs) where fh is the forecasting horizon
             Ground truth (correct) target values.
 
-        y_pred : return object of probabilistic predictition method scitype:y_pred
+        y_pred : return object of probabilistic prediction method scitype:y_pred
             must be at fh and for variables equal to those in y_true
 
         multioutput : string "uniform_average" or "raw_values" determines how\
@@ -176,7 +176,7 @@ class _BaseProbaForecastingErrorMetric(BaseForecastingErrorMetric):
                 (fh, n_outputs) where fh is the forecasting horizon
             Ground truth (correct) target values.
 
-        y_pred : return object of probabilistic predictition method scitype:y_pred
+        y_pred : return object of probabilistic prediction method scitype:y_pred
             must be at fh and for variables equal to those in y_true
 
         multioutput : string "uniform_average" or "raw_values" determines how\
@@ -855,7 +855,7 @@ class _BaseDistrForecastingMetric(_BaseProbaForecastingErrorMetric):
                 (fh, n_outputs) where fh is the forecasting horizon
             Ground truth (correct) target values.
 
-        y_pred : return object of probabilistic predictition method scitype:y_pred
+        y_pred : return object of probabilistic prediction method scitype:y_pred
             must be at fh and for variables equal to those in y_true
 
         multioutput : {'raw_values', 'uniform_average'} or array-like of shape \
diff --git a/sktime/performance_metrics/forecasting/sample_weight/__init__.py b/sktime/performance_metrics/forecasting/sample_weight/__init__.py
new file mode 100644
index 000000000..e7f69d293
--- /dev/null
+++ b/sktime/performance_metrics/forecasting/sample_weight/__init__.py
@@ -0,0 +1,16 @@
+"""Sample weight generators for sktime performance metrics."""
+
+__author__ = ["markussagen"]
+__all__ = [
+    "BaseSampleWeightGenerator",
+    "check_sample_weight_generator",
+    "SampleWeightGenerator",
+]
+
+from sktime.performance_metrics.forecasting.sample_weight._base import (
+    BaseSampleWeightGenerator,
+)
+from sktime.performance_metrics.forecasting.sample_weight._types import (
+    SampleWeightGenerator,
+    check_sample_weight_generator,
+)
diff --git a/sktime/performance_metrics/forecasting/sample_weight/_base.py b/sktime/performance_metrics/forecasting/sample_weight/_base.py
new file mode 100644
index 000000000..a2530baf0
--- /dev/null
+++ b/sktime/performance_metrics/forecasting/sample_weight/_base.py
@@ -0,0 +1,77 @@
+"""Implements base class for customer sample weights of performance metric in sktime."""
+
+__author__ = ["markussagen"]
+__all__ = ["BaseSampleWeightGenerator", "check_sample_weight_generator"]
+
+from sktime.base import BaseObject
+from sktime.performance_metrics.forecasting.sample_weight._types import (
+    check_sample_weight_generator,
+)
+
+
+class BaseSampleWeightGenerator(BaseObject):
+    """Base class for defining sample weight generators in sktime.
+
+    Extends sktime BaseObject.
+    """
+
+    _tags = {
+        "object_type": "sample_weight",
+        "authors": "markussagen",
+        "maintainers": "markussagen",
+    }
+
+    def __init__(self):
+        super().__init__()
+
+    def __call__(self, y_true, y_pred=None, **kwargs):
+        """Generate sample weights for a given set of true and predicted values.
+
+        Parameters
+        ----------
+        y_true : time series in sktime compatible data container format
+            Ground truth (correct) target values.
+
+        y_pred : time series in sktime compatible data container format, optional
+            Predicted values to evaluate against ground truth.
+            Must be of same format as y_true, same indices and columns if indexed.
+            If None, only y_true is used for weight generation.
+
+        **kwargs : dict
+            Additional keyword arguments specific to the weight generation method.
+
+        Returns
+        -------
+        sample_weight : 1D np.ndarray or None
+            Sample weights for each time point.
+
+            * If an array, must be 1D.
+              If y_true and y_pred are a single time series,
+              sample_weight must be of the same length as y_true.
+              If the time series are panel or hierarchical, the length of all
+              individual time series must be the same, and equal to the length
+              of sample_weight, for all instances of time series passed.
+            * If None, the time indices are considered equally weighted.
+        """
+        raise NotImplementedError("abstract method")
+
+    @classmethod
+    def get_test_params(cls, parameter_set="default"):
+        """Return testing parameter settings for the estimator.
+
+        Parameters
+        ----------
+        parameter_set : str, default="default"
+            Name of the set of test parameters to return, for use in tests. If no
+            special parameters are defined for a value, will return ``"default"`` set.
+
+        Returns
+        -------
+        params : dict or list of dict, default = {}
+            Parameters to create testing instances of the class
+            Each dict are parameters to construct an "interesting" test instance, i.e.,
+            ``MyClass(**params)`` or ``MyClass(**params[i])`` creates a valid test
+            instance.
+            ``create_test_instance`` uses the first (or only) dictionary in ``params``
+        """
+        return [{}]
diff --git a/sktime/performance_metrics/forecasting/sample_weight/_types.py b/sktime/performance_metrics/forecasting/sample_weight/_types.py
new file mode 100644
index 000000000..9e5aeb93a
--- /dev/null
+++ b/sktime/performance_metrics/forecasting/sample_weight/_types.py
@@ -0,0 +1,113 @@
+"""Implements base class for customer sample weights of performance metric in sktime."""
+
+__author__ = ["markussagen"]
+__all__ = ["SampleWeightGenerator", "check_sample_weight_generator"]
+
+from inspect import signature
+from typing import Protocol, runtime_checkable
+
+
+@runtime_checkable
+class SampleWeightGenerator(Protocol):
+    """Protocol for sample weight generators.
+
+    This protocol defines the interface for sample weight generator functions used in
+    performance metrics calculations. Sample weight generators are used to assign
+    weights to individual time points in time series data.
+
+    The generator must have at least one parameter (y_true) and accept **kwargs.
+    It must return a 1D numpy array of sample weights.
+
+    Parameters
+    ----------
+    y_true : array-like of shape (n_samples,) or (n_samples, n_outputs)
+        Ground truth (correct) target values.
+    y_pred : array-like of shape (n_samples,) or (n_samples, n_outputs), optional
+        Estimated target values.
+    **kwargs : dict
+        Additional keyword arguments that may be used in weight calculation.
+
+    Returns
+    -------
+    sample_weights : np.ndarray of shape (n_samples,)
+        1D array of sample weights.
+
+    Raises
+    ------
+    ValueError
+        If the sample weight generator does not have at least one parameter (y_true).
+        If the first parameter is not y_true.
+        If the sample weight generator does not accept **kwargs.
+
+    Notes
+    -----
+    Implementations of this protocol should ensure that the returned weights are
+    non-negative and sum to a value greater than zero.
+
+    Examples
+    --------
+    >>> import numpy as np
+    >>> class ExampleWeightGenerator:
+    ...     def __call__(self, y_true, y_pred=None, **kwargs):
+    ...         return np.ones_like(y_true)
+    >>> weight_gen = ExampleWeightGenerator()
+    >>> y_true = np.array([1, 2, 3, 4, 5])
+    >>> weights = weight_gen(y_true)
+    >>> print(weights)
+    [1. 1. 1. 1. 1.]
+    """
+
+    def __call__(self, y_true, y_pred=None, **kwargs):
+        """Generate sample weights for time series data.
+
+        This method should be implemented by concrete sample weight generators.
+
+        Parameters
+        ----------
+        y_true : array-like of shape (n_samples,) or (n_samples, n_outputs)
+            Ground truth (correct) target values.
+        y_pred : array-like of shape (n_samples,) or (n_samples, n_outputs), optional
+            Estimated target values. Default is None.
+        **kwargs : dict
+            Additional keyword arguments that may be used in weight calculation.
+
+        Returns
+        -------
+        sample_weights : np.ndarray of shape (n_samples,)
+            1D array of sample weights.
+
+        Notes
+        -----
+        - The method should always return a 1D numpy array of weights.
+        - The length of the returned array should match the number of samples in y_true.
+        - If y_pred is not used in the weight calculation, it should be ignored.
+        """
+        ...
+
+
+def check_sample_weight_generator(obj):
+    """Check if obj is a valid SampleWeightGenerator."""
+    if obj is None or not callable(obj):
+        return False
+
+    sig = signature(obj)
+    params = sig.parameters
+
+    # Check if the function has at least one parameter (y_true)
+    if len(params) < 1:
+        msg = "Sample weight generator must have at least one parameter (y_true)"
+        raise ValueError(msg)
+
+    param_names = list(params.keys())
+
+    # Check if the first parameter is y_true
+    if param_names[0] != "y_true":
+        raise ValueError("First parameter of sample weight generator must be 'y_true'")
+
+    # Check if the function accepts **kwargs
+    if not any(param.kind == param.VAR_KEYWORD for param in params.values()):
+        raise ValueError("Sample weight generator must accept **kwargs")
+
+    # NOTE: this does not check the return type!
+
+    return True
diff --git a/sktime/proba/_empirical.py b/sktime/proba/_empirical.py
index 8c25bee0c..c54b64944 100644
--- a/sktime/proba/_empirical.py
+++ b/sktime/proba/_empirical.py
@@ -25,8 +25,8 @@ class Empirical(BaseDistribution):
     index : pd.Index, optional, default = RangeIndex
     columns : pd.Index, optional, default = RangeIndex
 
-    Example
-    -------
+    Examples
+    --------
     >>> import pandas as pd
     >>> from sktime.proba.empirical import Empirical
 
diff --git a/sktime/proba/_laplace.py b/sktime/proba/_laplace.py
index 3da08916f..e756d823d 100644
--- a/sktime/proba/_laplace.py
+++ b/sktime/proba/_laplace.py
@@ -21,8 +21,8 @@ class Laplace(BaseDistribution):
     index : pd.Index, optional, default = RangeIndex
     columns : pd.Index, optional, default = RangeIndex
 
-    Example
-    -------
+    Examples
+    --------
     >>> from sktime.proba.laplace import Laplace
 
     >>> n = Laplace(mu=[[0, 1], [2, 3], [4, 5]], scale=1)
diff --git a/sktime/proba/_mixture.py b/sktime/proba/_mixture.py
index 1a8c75ea8..1ac7a6707 100644
--- a/sktime/proba/_mixture.py
+++ b/sktime/proba/_mixture.py
@@ -23,8 +23,8 @@ class Mixture(_HeterogenousMetaEstimator, BaseDistribution):
     index : pd.Index, optional, default = inferred from component distributions
     columns : pd.Index, optional, default = inferred from component distributions
 
-    Example
-    -------
+    Examples
+    --------
     >>> from sktime.proba.mixture import Mixture
     >>> from sktime.proba.normal import Normal
     >>>
diff --git a/sktime/proba/_normal.py b/sktime/proba/_normal.py
index 8edbc5ac0..46708f0b9 100644
--- a/sktime/proba/_normal.py
+++ b/sktime/proba/_normal.py
@@ -22,8 +22,8 @@ class Normal(BaseDistribution):
     index : pd.Index, optional, default = RangeIndex
     columns : pd.Index, optional, default = RangeIndex
 
-    Example
-    -------
+    Examples
+    --------
     >>> from sktime.proba.normal import Normal
 
     >>> n = Normal(mu=[[0, 1], [2, 3], [4, 5]], sigma=1)
diff --git a/sktime/proba/_t.py b/sktime/proba/_t.py
index 2e985a26d..c0f114d80 100644
--- a/sktime/proba/_t.py
+++ b/sktime/proba/_t.py
@@ -24,8 +24,8 @@ class TDistribution(BaseDistribution):
     index : pd.Index, optional, default = RangeIndex
     columns : pd.Index, optional, default = RangeIndex
 
-    Example
-    -------
+    Examples
+    --------
     >>> from sktime.proba.t import TDistribution
 
     >>> n = TDistribution(mu=[[0, 1], [2, 3], [4, 5]], sigma=1, df=10)
diff --git a/sktime/proba/tfp.py b/sktime/proba/tfp.py
index 6a9488946..9996afb10 100644
--- a/sktime/proba/tfp.py
+++ b/sktime/proba/tfp.py
@@ -22,8 +22,8 @@ class TFNormal(_BaseTFDistribution):
     index : pd.Index, optional, default = RangeIndex
     columns : pd.Index, optional, default = RangeIndex
 
-    Example
-    -------
+    Examples
+    --------
     >>> from sktime.proba.tfp import TFNormal  # doctest: +SKIP
 
     >>> n = TFNormal(mu=[[0, 1], [2, 3], [4, 5]], sigma=1)  # doctest: +SKIP
diff --git a/sktime/regression/deep_learning/cntc.py b/sktime/regression/deep_learning/cntc.py
index 2febb06a7..830c9dfa6 100644
--- a/sktime/regression/deep_learning/cntc.py
+++ b/sktime/regression/deep_learning/cntc.py
@@ -106,14 +106,12 @@ class CNTCRegressor(BaseDeepRegressor):
         """Construct a compiled, un-trained, keras model that is ready for training.
 
         In sktime, time series are stored in numpy arrays of shape (d,m), where d
-        is the number of dimensions, m is the series length. Keras/tensorflow assume
-        data is in shape (m,d). This method also assumes (m,d). Transpose should
-        happen in fit.
+        is the number of dimensions, m is the series length.
 
         Parameters
         ----------
         input_shape : tuple
-            The shape of the data fed into the input layer, should be (m,d)
+            The shape of the data fed into the input layer, should be (d,m)
 
         Returns
         -------
@@ -215,8 +213,6 @@ class CNTCRegressor(BaseDeepRegressor):
         """
         if self.callbacks is None:
             self._callbacks = []
-        # Transpose to conform to Keras input style.
-        X = X.transpose(0, 2, 1)
 
         check_random_state(self.random_state)
         self.input_shape = X.shape[1:]
@@ -246,8 +242,6 @@ class CNTCRegressor(BaseDeepRegressor):
         -------
         output : array of shape = [n_instances, n_classes] of probabilities
         """
-        # Transpose to work correctly with keras
-        X = X.transpose((0, 2, 1))
         X2 = self.prepare_input(X)
         preds = self.model_.predict([X2, X, X], self.batch_size, **kwargs)
         return preds
diff --git a/sktime/transformations/panel/dwt.py b/sktime/transformations/panel/dwt.py
index c84424354..95ee37745 100644
--- a/sktime/transformations/panel/dwt.py
+++ b/sktime/transformations/panel/dwt.py
@@ -21,8 +21,8 @@ class DWTTransformer(BaseTransformer):
     num_levels : int, number of levels to perform the Haar wavelet
         transformation.
 
-    Example
-    -------
+    Examples
+    --------
     >>> from sktime.transformations.panel.dwt import DWTTransformer
     >>> from sktime.datasets import load_airline
     >>> from sktime.datatypes import convert
diff --git a/sktime/transformations/panel/hog1d.py b/sktime/transformations/panel/hog1d.py
index 785b87da7..455c32272 100644
--- a/sktime/transformations/panel/hog1d.py
+++ b/sktime/transformations/panel/hog1d.py
@@ -42,8 +42,8 @@ class HOG1DTransformer(BaseTransformer):
     scaling_factor : float, default=0.1
         a constant that is multiplied to modify the distribution.
 
-    Example
-    ----------
+    Examples
+    --------
     >>> from sktime.transformations.panel.hog1d import HOG1DTransformer
     >>> from sktime.datasets import load_arrow_head
     >>>
diff --git a/sktime/transformations/panel/padder.py b/sktime/transformations/panel/padder.py
index 2bfb779c5..80b8251fc 100644
--- a/sktime/transformations/panel/padder.py
+++ b/sktime/transformations/panel/padder.py
@@ -23,8 +23,8 @@ class PaddingTransformer(BaseTransformer):
     pad_length : int, optional (default=None) length to pad the series too.
         if None, will find the longest sequence and use instead.
 
-    Example
-    -------
+    Examples
+    --------
     >>> import pandas as pd
     >>> from sktime.transformations.panel.padder import PaddingTransformer
     >>>
diff --git a/sktime/transformations/panel/rocket/_multirocket_multivariate.py b/sktime/transformations/panel/rocket/_multirocket_multivariate.py
index 8532d6e74..14c0ffc42 100644
--- a/sktime/transformations/panel/rocket/_multirocket_multivariate.py
+++ b/sktime/transformations/panel/rocket/_multirocket_multivariate.py
@@ -274,3 +274,19 @@ class MultiRocketMultivariate(BaseTransformer):
             num_features_per_dilation,
             biases,
         )
+
+    @classmethod
+    def get_test_params(cls):
+        """Return testing parameter sets for the estimator."""
+        return [
+            {
+                "num_kernels": 10,
+                "max_dilations_per_kernel": 32,
+                "random_state": 42,
+            },  # First parameter set
+            {
+                "num_kernels": 20,
+                "max_dilations_per_kernel": 16,
+                "random_state": 0,
+            },  # Second parameter set
+        ]
diff --git a/sktime/transformations/series/binning.py b/sktime/transformations/series/binning.py
index 301e45e29..37de70f16 100644
--- a/sktime/transformations/series/binning.py
+++ b/sktime/transformations/series/binning.py
@@ -44,8 +44,8 @@ class TimeBinAggregate(BaseTransformer):
         "bin_mid" = transformed pd.DataFrame will be indexed by bin midpoints
         "bin" = transformed pd.DataFrame will have ``bins`` as ``IntervalIndex``
 
-    Example
-    -------
+    Examples
+    --------
     from sktime.datatypes import get_examples
     from sktime.transformations.series.binning import TimeBinAggregate
 
diff --git a/sktime/transformations/series/dilation_mapping.py b/sktime/transformations/series/dilation_mapping.py
index 63afda3ff..4ba62c894 100644
--- a/sktime/transformations/series/dilation_mapping.py
+++ b/sktime/transformations/series/dilation_mapping.py
@@ -47,8 +47,8 @@ class DilationMappingTransformer(BaseTransformer):
        Accurate and Memory Constrained Time Series Classification", 2023,
        arXiv preprint arXiv:2301.10194.
 
-    Example
-    ----------
+    Examples
+    --------
     >>> from sktime.transformations.series.dilation_mapping import \
     ...     DilationMappingTransformer
     >>> from sktime.datasets import load_airline
diff --git a/sktime/transformations/series/dropna.py b/sktime/transformations/series/dropna.py
index b1655195e..e1be5c5a3 100644
--- a/sktime/transformations/series/dropna.py
+++ b/sktime/transformations/series/dropna.py
@@ -40,6 +40,19 @@ class DropNA(BaseTransformer):
         If True, drops the same rows/columns in transform as in fit. If false,
         drops rows/columns according to the NAs seen in transform (equivalent
         to PandasTransformAdaptor(method="dropna")).
+
+    Examples
+    --------
+    >>> from sktime.transformations.series.dropna import DropNA
+    >>> import pandas as pd
+    >>> import numpy as np
+    >>> X = pd.DataFrame({'a': [1, 2, np.nan, 4], 'b': [5, np.nan, 7, 8]})
+    >>> transformer = DropNA(axis=0, how='any')
+    >>> X_transformed = transformer.fit_transform(X)
+    >>> print(X_transformed)
+       a    b
+    0  1.0  5.0
+    3  4.0  8.0
     """
 
     _tags = {
diff --git a/sktime/transformations/series/filter.py b/sktime/transformations/series/filter.py
index 8752b634e..c98a07f37 100644
--- a/sktime/transformations/series/filter.py
+++ b/sktime/transformations/series/filter.py
@@ -32,8 +32,8 @@ class Filter(BaseTransformer):
         See ``mne.filter.filter_data``
         documentation for a detailed description of all options.
 
-    Example
-    -------
+    Examples
+    --------
     >>> from sktime.transformations.series.filter import Filter
     >>> from sktime.datasets import load_arrow_head
     >>> X, y = load_arrow_head(return_X_y=True, return_type="pd-multiindex")
diff --git a/sktime/transformations/series/kinematic.py b/sktime/transformations/series/kinematic.py
index e38743790..e26a4271a 100644
--- a/sktime/transformations/series/kinematic.py
+++ b/sktime/transformations/series/kinematic.py
@@ -49,8 +49,8 @@ class KinematicFeatures(BaseTransformer):
         * "a_abs" - absolute acceleration
         * "curv" - curvature
 
-    Example
-    -------
+    Examples
+    --------
     >>> import numpy as np
     >>> import pandas as pd
     >>> from sktime.transformations.series.kinematic import KinematicFeatures


Test patch:
diff --git a/sktime/performance_metrics/forecasting/sample_weight/tests/__init__.py b/sktime/performance_metrics/forecasting/sample_weight/tests/__init__.py
new file mode 100644
index 000000000..1af936133
--- /dev/null
+++ b/sktime/performance_metrics/forecasting/sample_weight/tests/__init__.py
@@ -0,0 +1,1 @@
+"""Tests for sktime sample weight generators."""
diff --git a/sktime/performance_metrics/forecasting/sample_weight/tests/test_sample_weight_generator.py b/sktime/performance_metrics/forecasting/sample_weight/tests/test_sample_weight_generator.py
new file mode 100644
index 000000000..4cd7e1ca0
--- /dev/null
+++ b/sktime/performance_metrics/forecasting/sample_weight/tests/test_sample_weight_generator.py
@@ -0,0 +1,260 @@
+"""Tests for sample weight generators."""
+
+import numpy as np
+import pandas as pd
+import pytest
+
+from sktime.performance_metrics.forecasting.sample_weight._base import (
+    BaseSampleWeightGenerator,
+)
+from sktime.performance_metrics.forecasting.sample_weight._types import (
+    check_sample_weight_generator,
+)
+from sktime.tests.test_switch import run_test_module_changed
+
+
+class DummyWeightGenerator(BaseSampleWeightGenerator):
+    def __call__(self, y_true, y_pred=None, **kwargs) -> None:
+        return None
+
+
+class UniformWeightGenerator(BaseSampleWeightGenerator):
+    def __call__(self, y_true, y_pred=None, **kwargs) -> np.ndarray:
+        return np.ones_like(y_true)
+
+
+class RecencyWeightGenerator(BaseSampleWeightGenerator):
+    def __init__(self, decay_rate=0.1):
+        self.decay_rate = decay_rate
+
+    def __call__(self, y_true, y_pred=None, **kwargs):
+        weights = np.exp(-self.decay_rate * np.arange(len(y_true))[::-1])
+        return weights / np.sum(weights)
+
+
+class DateRangeWeightGenerator(BaseSampleWeightGenerator):
+    def __init__(self, date_ranges, default_weight=1.0):
+        self.date_ranges = date_ranges
+        self.default_weight = default_weight
+
+    def _get_dt(self, date_series):
+        """Get the datetime object from the series."""
+        dt = date_series
+        if isinstance(date_series, pd.Timestamp):
+            dt = date_series
+        elif isinstance(date_series, pd.DatetimeIndex):
+            dt = date_series
+        elif isinstance(date_series, pd.Series):
+            dt = date_series.dt
+        elif isinstance(date_series, pd.DataFrame):
+            dt = date_series.dt
+        elif isinstance(date_series, np.datetime64):
+            dt = pd.to_datetime(date_series)
+        else:
+            msg = f"Invalid type for conversion to datetime: {type(date_series)}"
+            raise ValueError(msg)
+        return dt
+
+    def __call__(self, y_true, y_pred=None, **kwargs):
+        if "dates" not in kwargs:
+            msg = "'dates' must be provided in kwargs for DateRangeWeightGenerator"
+            raise ValueError(msg)
+
+        if not (
+            isinstance(self.date_ranges, list)
+            and (
+                all(isinstance(range_config, dict) for range_config in self.date_ranges)
+            )
+        ):
+            raise ValueError("'date_ranges' must be a list of dictionaries")
+
+        if not all(
+            "start" in range_config
+            and "end" in range_config
+            and "weight" in range_config
+            for range_config in self.date_ranges
+        ):
+            msg = "'date_ranges' must be a list of dicts with the keys: "
+            msg += "'start', 'end', and 'weight'"
+            raise ValueError(msg)
+
+        dates = pd.to_datetime(kwargs["dates"])
+        weights = np.full(len(y_true), self.default_weight)
+
+        for range_config in self.date_ranges:
+            start_date = pd.to_datetime(range_config["start"], format="%m-%d")
+            end_date = pd.to_datetime(range_config["end"], format="%m-%d")
+
+            dt = self._get_dt(dates)
+            mask = (
+                (dt.month >= start_date.month)
+                & (dt.month <= end_date.month)
+                & (dt.day >= start_date.day)
+                & (dt.day <= end_date.day)
+            )
+            weights[mask] = range_config["weight"]
+
+        return weights / np.sum(weights)
+
+
+@pytest.mark.skipif(
+    not run_test_module_changed(["sktime.performance_metrics"]),
+    reason="Run if performance_metrics module has changed.",
+)
+def test_base_sample_weight_generator():
+    """Test BaseSampleWeightGenerator."""
+
+    class TestGenerator(BaseSampleWeightGenerator):
+        def __call__(self, y_true, y_pred=None, **kwargs):
+            return np.ones_like(y_true)
+
+    generator = TestGenerator()
+    y_true = np.array([1, 2, 3])
+    weights = generator(y_true)
+
+    assert isinstance(weights, np.ndarray)
+    assert np.array_equal(weights, np.ones_like(y_true))
+
+
+@pytest.mark.skipif(
+    not run_test_module_changed(["sktime.performance_metrics"]),
+    reason="Run if performance_metrics module has changed.",
+)
+def test_check_sample_weight_generator():
+    """Test check_sample_weight_generator function."""
+
+    def valid_generator(y_true, y_pred=None, **kwargs):
+        return np.ones_like(y_true)
+
+    def valid_generator_no_params(y_true, y_pred=None, **kwargs):
+        return None
+
+    def valid_generator_no_y_pred(y_true, **kwargs):
+        return np.ones_like(y_true)
+
+    assert check_sample_weight_generator(valid_generator)
+    assert check_sample_weight_generator(valid_generator_no_params)
+    assert check_sample_weight_generator(valid_generator_no_y_pred)
+
+
+@pytest.mark.skipif(
+    not run_test_module_changed(["sktime.performance_metrics"]),
+    reason="Run if performance_metrics module has changed.",
+)
+def test_check_sample_weight_generator_are_invalid():
+    """Test check_sample_weight_generator function."""
+
+    def valid_generator_no_kwargs(y_true, y_pred=None):
+        return np.ones_like(y_true)
+
+    def invalid_generator():
+        return "invalid"
+
+    def invalid_generator_no_param_y_true(invalid):
+        return np.ones((1, 1))
+
+    # Validate
+    with pytest.raises(ValueError):
+        check_sample_weight_generator(valid_generator_no_kwargs)
+
+    with pytest.raises(ValueError):
+        check_sample_weight_generator(invalid_generator)
+
+    with pytest.raises(ValueError):
+        check_sample_weight_generator(invalid_generator_no_param_y_true)
+
+
+@pytest.mark.skipif(
+    not run_test_module_changed(["sktime.performance_metrics"]),
+    reason="Run if performance_metrics module has changed.",
+)
+def test_dummy_weight_generator():
+    generator = DummyWeightGenerator()
+    y_true = np.array([1, 2, 3])
+    weights = generator(y_true)
+
+    assert weights is None
+
+
+@pytest.mark.skipif(
+    not run_test_module_changed(["sktime.performance_metrics"]),
+    reason="Run if performance_metrics module has changed.",
+)
+def test_uniform_weight_generator():
+    generator = UniformWeightGenerator()
+    y_true = np.array([1, 2, 3])
+    weights = generator(y_true)
+
+    assert isinstance(weights, np.ndarray)
+    assert np.array_equal(weights, np.ones_like(y_true))
+
+
+@pytest.mark.skipif(
+    not run_test_module_changed(["sktime.performance_metrics"]),
+    reason="Run if performance_metrics module has changed.",
+)
+def test_recency_weight_generator():
+    generator = RecencyWeightGenerator(decay_rate=0.1)
+    y_true = np.array([1, 2, 3])
+    weights = generator(y_true)
+
+    assert isinstance(weights, np.ndarray)
+    assert len(weights) == len(y_true)
+    assert np.all(weights[:-1] < weights[1:])  # Weights should be increasing
+
+
+@pytest.mark.skipif(
+    not run_test_module_changed(["sktime.performance_metrics"]),
+    reason="Run if performance_metrics module has changed.",
+)
+def test_date_range_weight_generator():
+    date_ranges = [
+        {"start": "01-01", "end": "03-31", "weight": 2},
+        {"start": "06-01", "end": "08-31", "weight": 3},
+    ]
+    generator = DateRangeWeightGenerator(date_ranges=date_ranges)
+    y_true = np.array([1, 2, 3, 4, 5])
+    dates = pd.date_range(start="2023-01-01", periods=5, freq="MS")
+    weights = generator(y_true, dates=dates)
+
+    assert isinstance(weights, np.ndarray)
+    assert len(weights) == len(y_true)
+    assert weights[0] > weights[3]  # January should have higher weight than April
+
+
+@pytest.mark.skipif(
+    not run_test_module_changed(["sktime.performance_metrics"]),
+    reason="Run if performance_metrics module has changed.",
+)
+def test_sample_weight_generator_with_y_pred():
+    class TestGenerator(BaseSampleWeightGenerator):
+        def __call__(self, y_true, y_pred=None, **kwargs):
+            if y_pred is None:
+                return np.ones_like(y_true)
+            return np.abs(y_true - y_pred)
+
+    generator = TestGenerator()
+    y_true = np.array([1, 2, 3])
+    y_pred = np.array([1.1, 2.2, 2.8])
+    weights = generator(y_true, y_pred)
+
+    assert isinstance(weights, np.ndarray)
+    assert np.allclose(weights, np.abs(y_true - y_pred))
+
+
+@pytest.mark.skipif(
+    not run_test_module_changed(["sktime.performance_metrics"]),
+    reason="Run if performance_metrics module has changed.",
+)
+def test_sample_weight_generator_with_kwargs():
+    class TestGenerator(BaseSampleWeightGenerator):
+        def __call__(self, y_true, y_pred=None, **kwargs):
+            factor = kwargs.get("factor", 1)
+            return np.ones_like(y_true) * factor
+
+    generator = TestGenerator()
+    y_true = np.array([1, 2, 3])
+    weights = generator(y_true, factor=2)
+
+    assert isinstance(weights, np.ndarray)
+    assert np.array_equal(weights, np.ones_like(y_true) * 2)
diff --git a/sktime/performance_metrics/tests/test_performance_metrics_forecasting.py b/sktime/performance_metrics/tests/test_performance_metrics_forecasting.py
index 5789c8121..63ff1d140 100644
--- a/sktime/performance_metrics/tests/test_performance_metrics_forecasting.py
+++ b/sktime/performance_metrics/tests/test_performance_metrics_forecasting.py
@@ -47,6 +47,9 @@ from sktime.performance_metrics.forecasting import (
     median_squared_scaled_error,
     relative_loss,
 )
+from sktime.performance_metrics.forecasting.sample_weight._base import (
+    BaseSampleWeightGenerator,
+)
 from sktime.performance_metrics.tests._config import RANDOM_SEED
 from sktime.tests.test_switch import run_test_module_changed
 from sktime.utils._testing.series import _make_series
@@ -588,3 +591,99 @@ def test_y_true_y_pred_inconsistent_n_variables_raises_error(metric_func_name):
         ValueError, match="y_true and y_pred have different number of output"
     ):
         metric_func(y_true, y_pred, y_train=y_train, y_pred_benchmark=y_pred_benchmark)
+
+
+@pytest.mark.skipif(
+    not run_test_module_changed(["sktime.performance_metrics"]),
+    reason="Run if performance_metrics module has changed.",
+)
+@pytest.mark.parametrize("random_state", RANDOM_STATES)
+@pytest.mark.parametrize("metric_func_name", LOSS_RESULTS.keys())
+def test_sample_weight_generator_has_no_effect_on_metric_function(
+    metric_func_name, random_state
+):
+    """Tests that loss function with univariate input should return scalar number."""
+    y = _make_series(n_timepoints=75, random_state=random_state)
+    y_train, y_true = y.iloc[:50], y.iloc[50:]
+    y_pred = y.shift(1).iloc[50:]
+    y_pred_benchmark = y.rolling(2).mean().iloc[50:]
+
+    class TestWeightGenerator(BaseSampleWeightGenerator):
+        def __call__(self, y_true, y_pred=None, **kwargs):
+            return np.arange(1, len(y_true) + 1)
+
+    weight_generator = TestWeightGenerator()
+
+    metric_func = LOSS_RESULTS[metric_func_name]["func"]
+
+    function_loss = metric_func(
+        y_true, y_pred, y_train=y_train, y_pred_benchmark=y_pred_benchmark
+    )
+
+    function_loss_with_weights = metric_func(
+        y_true,
+        y_pred,
+        y_train=y_train,
+        y_pred_benchmark=y_pred_benchmark,
+        sample_weight=weight_generator,
+    )
+
+    is_num = is_numeric_dtype(function_loss)
+    is_scalar = np.isscalar(function_loss)
+    assert is_num and is_scalar, " ".join(
+        ["Loss function with univariate input should return scalar number"]
+    )
+
+    is_num = is_numeric_dtype(function_loss_with_weights)
+    is_scalar = np.isscalar(function_loss_with_weights)
+    assert is_num and is_scalar, " ".join(
+        ["Loss function with sample weight generator should return scalar number"]
+    )
+
+    assert np.isclose(function_loss, function_loss_with_weights), " ".join(
+        ["Loss function with sample weight generator should return different value"]
+    )
+
+
+@pytest.mark.skipif(
+    not run_test_module_changed(["sktime.performance_metrics"]),
+    reason="Run if performance_metrics module has changed.",
+)
+@pytest.mark.parametrize("random_state", RANDOM_STATES)
+@pytest.mark.parametrize(
+    "metric_class",
+    [
+        MeanSquaredError,
+        MeanAbsoluteError,
+        MeanAbsolutePercentageError,
+    ],
+)
+def test_sample_weight_generator_is_passed_to_func(metric_class, random_state):
+    """Tests that loss function with univariate input should return scalar number."""
+    y = _make_series(n_timepoints=75, random_state=random_state)
+    y_train, y_true = y.iloc[:50], y.iloc[50:]
+    y_pred = y.shift(1).iloc[50:]
+    y_pred_benchmark = y.rolling(2).mean().iloc[50:]
+
+    class TestWeightGenerator(BaseSampleWeightGenerator):
+        def __call__(self, y_true, y_pred=None, **kwargs):
+            return np.arange(1, len(y_true) + 1)
+
+    weight_generator = TestWeightGenerator()
+
+    metric = metric_class()
+
+    function_loss = metric(
+        y_true, y_pred, y_train=y_train, y_pred_benchmark=y_pred_benchmark
+    )
+    function_loss_with_weights = metric(
+        y_true,
+        y_pred,
+        y_train=y_train,
+        y_pred_benchmark=y_pred_benchmark,
+        sample_weight=weight_generator,
+    )
+
+    assert not np.isclose(function_loss, function_loss_with_weights), " ".join(
+        ["Loss function with sample weight generator should return different value"]
+    )
diff --git a/sktime/tests/_config.py b/sktime/tests/_config.py
index 721a57713..a43c609cc 100644
--- a/sktime/tests/_config.py
+++ b/sktime/tests/_config.py
@@ -42,8 +42,6 @@ EXCLUDE_ESTIMATORS = [
     "FCNRegressor",
     "LSTMFCNRegressor",
     "MACNNRegressor",
-    "CNTCClassifier",
-    "CNTCRegressor",
     # splitters excluded with undiagnosed failures, see #6194
     # these are temporarily skipped to allow merging of the base test framework
     "SameLocSplitter",
@@ -321,7 +319,6 @@ EXCLUDED_TESTS_BY_TEST = {
         "MiniRocketMultivariate",
         "MiniRocketMultivariateVariable",
         "MultiRocket",
-        "MultiRocketMultivariate",
         "MultioutputTabularRegressionForecaster",
         "MultioutputTimeSeriesRegressionForecaster",
         "OnlineEnsembleForecaster",