Repository: sktime/sktime
Pull request #7772 (2025-02-06 20:00:36)

[ENH] Implementing ES-RNN
<!--
Welcome to sktime, and thanks for contributing!
Please have a look at our contribution guide:
https://www.sktime.net/en/latest/get_involved/contributing.html
-->

#### Reference Issues/PRs
Mentioned at #6660 
<!--
Example: Fixes #1234. See also #3456.

Please use keywords (e.g., Fixes) to create link to the issues or pull requests
you resolved, so that they will automatically be closed when your pull request
is merged. See https://github.com/blog/1506-closing-issues-via-pull-requests.
If no issue exists, you can open one here: https://github.com/sktime/sktime/issues
-->


#### What does this implement/fix? Explain your changes.
Implements the ES-RNN, with reference from https://www.sciencedirect.com/science/article/pii/S0169207019301153
for non, single and double seasonality


Ankit-1204: @fkiraly , Since this is my first time implementing a network for sktime, I could use some review related to integration,
- First off, I was comfortable with torch so I used that, I could change it if its not correct
- There may be some syntax error related to tensor, as I initially just used numpy(I'll fix that)

- [ESRNN](https://github.com/damitkwr/ESRNN-GPU/blob/master/es_rnn/model.py), I found this implementation, and I think my implementation for seasonality is constant and probably wrong, should I try to align it more with the given implementation?. This is the part I am confused about. I think the paper also mentioned the seasonality to evolve with time

geetu040: > * [ESRNN](https://github.com/damitkwr/ESRNN-GPU/blob/master/es_rnn/model.py), I found this implementation from kaggle, and I think my implementation for seasonality is constant and probably wrong

can you link both the implementations you are referring?

> * First off, I was comfortable with torch so I used that, I could change it if its not correct

what does the other implementation use?
Ankit-1204: > can you link both the implementations you are referring?

oh I think I phrased it incorrectly, i meant i found [ESRNN](https://github.com/damitkwr/ESRNN-GPU/blob/master/es_rnn/model.py) mentioned on kaggle and compared that to what i wrote for this PR.

geetu040: > oh I think I phrased it incorrectly, i meant i found [ESRNN](https://github.com/damitkwr/ESRNN-GPU/blob/master/es_rnn/model.py) mentioned on kaggle and compared that to what i wrote for this PR.

I see, complete the forecasting class and we'll see how it works
Ankit-1204: @fkiraly @geetu040 can you review it now?
fkiraly: I started the full suite of tests
geetu040: And sorry for a belated reply, just ping when you are done or have any question
Ankit-1204: @geetu040 can you review it again now? Thanks.
geetu040: @Ankit-1204 can you merge main branch and resolve the conflicts?

Patch:
diff --git a/docs/source/api_reference/forecasting.rst b/docs/source/api_reference/forecasting.rst
index 580e9f97a..5ad0087cb 100644
--- a/docs/source/api_reference/forecasting.rst
+++ b/docs/source/api_reference/forecasting.rst
@@ -526,6 +526,14 @@ Deep learning based forecasters
 
     RBFForecaster
 
+.. currentmodule:: sktime.forecasting.es_rnn
+
+.. autosummary::
+    :toctree: auto_generated/
+    :template: class.rst
+
+    ESRNNForecaster
+
 Pre-trained and foundation models
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
diff --git a/docs/source/developer_guide/add_estimators.rst b/docs/source/developer_guide/add_estimators.rst
index b6b1dafb8..cccbbba2f 100644
--- a/docs/source/developer_guide/add_estimators.rst
+++ b/docs/source/developer_guide/add_estimators.rst
@@ -176,7 +176,7 @@ A useful workflow for using ``check_estimator`` to debug an estimator is as foll
 
 1. Run ``check_estimator(MyEstimator)`` to find failing tests
 2. Subset to failing tests or fixtures using ``fixtures_to_run`` or ``tests_to_run``
-3. If the failure is not obvious, set ``raise_exceptions=True`` to raise the exception and inspecet the traceback.
+3. If the failure is not obvious, set ``raise_exceptions=True`` to raise the exception and inspect the traceback.
 4. If the failure is still not clear, use advanced debuggers on the line of code with ``check_estimator``.
 
 Running the test suite in a repository clone
diff --git a/sktime/base/_base.py b/sktime/base/_base.py
index 3ebe1eb75..ff6c131f1 100644
--- a/sktime/base/_base.py
+++ b/sktime/base/_base.py
@@ -151,59 +151,6 @@ class BaseObject(_HTMLDocumentationLinkMixin, _BaseObject):
         """,
     }
 
-    def __init__(self):
-        super().__init__()
-
-        # handle numpy 2 incompatible soft dependencies
-        # for rationale, see _handle_numpy2_softdeps
-        self._handle_numpy2_softdeps()
-
-    # TODO 0.36.0: check list of numpy 2 incompatible soft deps
-    # remove any from NOT_NP2_COMPATIBLE that become compatible
-    def _handle_numpy2_softdeps(self):
-        """Handle tags for soft deps that are not numpy 2 compatible.
-
-        A number of soft dependencies are not numpy 2 compatible yet,
-        but do not set the bound in their setup.py. This method is a patch over
-        those packages' missing bound setting to provide informative
-        errors to users.
-
-        This method does the following:
-
-        * checks if any soft dependencies in the python_dependencies tag
-          are in NOT_NP2_COMPATIBLE, this is a hard-coded
-          list of soft dependencies that are not numpy 2 compatible
-        * if any are found, adds a numpy<2.0 soft dependency to the list,
-          and sets it as a dynamic override of the python_dependencies tag
-        """
-        from packaging.requirements import Requirement
-
-        # pypi package names of soft dependencies that are not numpy 2 compatibleS
-        NOT_NP2_COMPATIBLE = ["pmdarima"]
-
-        softdeps = self.get_class_tag("python_dependencies", [])
-        if softdeps is None:
-            return None
-        if not isinstance(softdeps, list):
-            softdeps = [softdeps]
-        # make copy of list to avoid side effects
-        softdeps = softdeps.copy()
-
-        def _pkg_name(req):
-            """Get package name from requirement string."""
-            return Requirement(req).name
-
-        noncomp = False
-        for softdep in softdeps:
-            # variable: does any softdep string start with one of the non-compatibles
-            noncomp_sd = any([_pkg_name(softdep) == pkg for pkg in NOT_NP2_COMPATIBLE])
-            noncomp = noncomp or noncomp_sd
-
-        if noncomp:
-            softdeps = softdeps + ["numpy<2.0"]
-            self.set_tags(python_dependencies=softdeps)
-        return None
-
     def __eq__(self, other):
         """Equality dunder. Checks equal class and parameters.
 
diff --git a/sktime/forecasting/base/adapters/_pmdarima.py b/sktime/forecasting/base/adapters/_pmdarima.py
index 0733b1fbf..fd336da73 100644
--- a/sktime/forecasting/base/adapters/_pmdarima.py
+++ b/sktime/forecasting/base/adapters/_pmdarima.py
@@ -23,7 +23,8 @@ class _PmdArimaAdapter(BaseForecaster):
         "capability:pred_int:insample": True,
         "requires-fh-in-fit": False,
         "handles-missing-data": True,
-        "python_dependencies": "pmdarima",
+        # TODO 0.37.0: check if numpy 2 incompatiblity can be removed
+        "python_dependencies": ["pmdarima", "numpy<2"],
     }
 
     def __init__(self):
diff --git a/sktime/forecasting/base/adapters/_tbats.py b/sktime/forecasting/base/adapters/_tbats.py
index 34d90e1a7..dc24cf0fa 100644
--- a/sktime/forecasting/base/adapters/_tbats.py
+++ b/sktime/forecasting/base/adapters/_tbats.py
@@ -24,7 +24,7 @@ class _TbatsAdapter(BaseForecaster):
         "capability:pred_int:insample": True,
         "requires-fh-in-fit": False,
         "handles-missing-data": False,
-        "python_dependencies": "tbats",
+        "python_dependencies": ["tbats", "numpy<2"],
     }
 
     def __init__(
diff --git a/sktime/forecasting/es_rnn.py b/sktime/forecasting/es_rnn.py
new file mode 100644
index 000000000..6670ac1b2
--- /dev/null
+++ b/sktime/forecasting/es_rnn.py
@@ -0,0 +1,307 @@
+"""Interface for ES RNN for Time Series Forecasting."""
+
+import numpy as np
+
+from sktime.forecasting.base.adapters._pytorch import BaseDeepNetworkPyTorch
+from sktime.networks.es_rnn import ESRNN
+from sktime.utils.dependencies import _check_soft_dependencies
+
+if _check_soft_dependencies("torch", severity="none"):
+    import torch
+    from torch.utils.data import Dataset
+else:
+
+    class Dataset:
+        """Dummy class if torch is unavailable."""
+
+
+class ESRNNTrainDataset(Dataset):
+    """Implements Pytorch Dataset class for Training ESS-RNN."""
+
+    def __init__(self, y, pred_len, window, stride) -> None:
+        self.y = y
+        self.window = window
+        self.pred_len = pred_len
+        self.stride = stride
+        self._get_data()
+
+    def _get_data(self):
+        length = len(self.y)
+        x_arr = []
+        y_arr = []
+        for i in range(0, length - self.window - self.pred_len + 1, self.stride):
+            inp = self.y[i : i + self.window]
+            out = self.y[i + self.window : i + self.window + self.pred_len]
+
+            x_arr.append(inp)
+            y_arr.append(out)
+
+        if not x_arr:
+            raise ValueError("Input size to small")
+
+        self.x_train, self.y_train = (
+            torch.FloatTensor(np.array(x_arr)),
+            torch.FloatTensor(np.array(y_arr)),
+        )
+
+    def __len__(self):
+        """Get length of the dataset."""
+        return len(self.x_train)
+
+    def __getitem__(self, idx):
+        """Get data pairs at this index."""
+        return self.x_train[idx], self.y_train[idx]
+
+
+class ESRNNPredDataset(Dataset):
+    """Implements Pytorch Dataset class for Prediction ESS-RNN."""
+
+    def __init__(self, y, window) -> None:
+        self.y = y
+        self.window = window
+        self._get_data()
+
+    def _get_data(self):
+        x_pred = self.y[-self.window :]
+        x_pred = torch.FloatTensor(np.array(x_pred))
+        self.x_pred = x_pred.unsqueeze(0)
+
+    def __len__(self):
+        """Get length of the dataset."""
+        return len(self.x_pred)
+
+    def __getitem__(self, idx):
+        """Return data point."""
+        return self.x_pred[idx], torch.zeros(1)
+
+
+class ESRNNForecaster(BaseDeepNetworkPyTorch):
+    """
+    Exponential Smoothing Recurrant Neural Network.
+
+    This model combines Exponential Smoothing (ES) and (LSTM) networks
+    for time series forecasting. ES is used to balance the level and
+    seasonality of the series. This method has been proposed in [1]_.
+
+    Parameters
+    ----------
+    hidden_size : int
+        Number of features in the hidden state
+    num_layer : int
+        Number of layers
+    seasonality_type : string
+        Type of seasonality_type, could be zero ,single or double
+    season1_length : int
+        Period of season 1
+    season2_length : int
+        Period of season 2
+    stride : int
+        stride for sliding window
+    batch_size : int
+        size of batch during training
+    num_epochs : int
+        number of epochs during training
+    criterion : torch.nn Loss Function, default=torch.nn.MSELoss
+        loss function to be used for training
+    criterion_kwargs : dict, default=None
+        keyword arguments to pass to criterion
+    optimizer : torch.optim.Optimizer, default=torch.optim.Adam
+        optimizer to be used for training
+    optimizer_kwargs : dict, default=None
+        keyword arguments to pass to optimizer
+    window : int
+        Size of Input window, default=5
+    lr : int
+        Learning rate for training
+
+    References
+    ----------
+    .. [1] Smyl, S. 2020.
+    A hybrid method of exponential smoothing and recurrent \
+    neural networks for time series forecasting.
+    https://www.sciencedirect.com/science/article/pii/S0169207019301153
+
+    Examples
+    --------
+    >>> from sktime.forecasting.es_rnn import ESRNNForecaster # doctest: +SKIP
+    >>> from sktime.datasets import load_airline
+    >>> from sktime.transformations.series.boxcox import LogTransformer
+    >>> y = load_airline()
+    >>> scaler=LogTransformer()
+    >>> forecaster=ESRNNForecaster(15,6,12,6,'double',20,1,32,100,'MSE')# doctest: +SKIP
+    >>> y_new=scaler.fit_transform(y)
+    >>> forecaster.fit(y_new, fh=[1,2,3]) # doctest: +SKIP
+    >>> y_pred = forecaster.predict() # doctest: +SKIP
+    >>> y_pred=scaler.inverse_transform(y_pred)
+    """
+
+    _tags = {
+        # packaging info
+        # --------------
+        "authors": ["Ankit-1204"],
+    }
+
+    def __init__(
+        self,
+        hidden_size=10,
+        num_layer=5,
+        season1_length=12,
+        season2_length=6,
+        seasonality_type="single",
+        window=10,
+        stride=1,
+        batch_size=32,
+        num_epochs=1000,
+        criterion=None,
+        optimizer="Adam",
+        lr=1e-1,
+        optimizer_kwargs=None,
+        criterion_kwargs=None,
+        custom_dataset_train=None,
+        custom_dataset_pred=None,
+    ) -> None:
+        super().__init__()
+        self.hidden_size = hidden_size
+        self.num_layer = num_layer
+        self.seasonality_type = seasonality_type
+        self.season1_length = season1_length
+        self.season2_length = season2_length
+        self.window = window
+        self.stride = stride
+        self.batch_size = batch_size
+        self.num_epochs = num_epochs
+        self.optimizer = optimizer
+        self.criterion = criterion
+        self.optimizer_kwargs = optimizer_kwargs
+        self.criterion_kwargs = criterion_kwargs
+        self.custom_dataset_train = custom_dataset_train
+        self.custom_dataset_pred = custom_dataset_pred
+        self.lr = lr
+        if _check_soft_dependencies("torch", severity="none"):
+            import torch
+
+            self.criterions = {
+                "MSE": torch.nn.MSELoss,
+                "L1": torch.nn.L1Loss,
+                "SmoothL1": torch.nn.SmoothL1Loss,
+                "Huber": torch.nn.HuberLoss,
+            }
+
+            self.optimizers = {
+                "Adadelta": torch.optim.Adadelta,
+                "Adagrad": torch.optim.Adagrad,
+                "Adam": torch.optim.Adam,
+                "AdamW": torch.optim.AdamW,
+                "SGD": torch.optim.SGD,
+            }
+
+    def _instantiate_criterion(self):
+        if self.criterion:
+            return super()._instantiate_criterion()
+        else:
+            return ESRNN().pin_ball()
+
+    def _build_network(self, fh):
+        self.pred_len = fh
+        self.input_shape = self._y.shape[-1]
+        return ESRNN(
+            self.input_shape,
+            self.hidden_size,
+            self.pred_len,
+            self.num_layer,
+            self.season1_length,
+            self.season2_length,
+            self.seasonality_type,
+        )._build()
+
+    def build_pytorch_train_dataloader(self, y):
+        """Build PyTorch DataLoader for training."""
+        from torch.utils.data import DataLoader
+
+        if self.custom_dataset_train:
+            if hasattr(self.custom_dataset_train, "build_dataset") and callable(
+                self.custom_dataset_train.build_dataset
+            ):
+                self.custom_dataset_train.build_dataset(y)
+                dataset = self.custom_dataset_train
+            else:
+                raise NotImplementedError(
+                    "Custom Dataset `build_dataset` method is not available. Please "
+                    f"refer to the {self.__class__.__name__}.build_dataset "
+                    "documentation."
+                )
+        else:
+            dataset = ESRNNTrainDataset(
+                y=y,
+                window=self.window,
+                pred_len=self.pred_len,
+                stride=self.stride,
+            )
+
+        return DataLoader(dataset, self.batch_size, shuffle=True)
+
+    def build_pytorch_pred_dataloader(self, y, fh):
+        """Build PyTorch DataLoader for prediction."""
+        from torch.utils.data import DataLoader
+
+        if self.custom_dataset_pred:
+            if hasattr(self.custom_dataset_pred, "build_dataset") and callable(
+                self.custom_dataset_pred.build_dataset
+            ):
+                self.custom_dataset_train.build_dataset(y)
+                dataset = self.custom_dataset_train
+            else:
+                raise NotImplementedError(
+                    "Custom Dataset `build_dataset` method is not available. Please"
+                    f"refer to the {self.__class__.__name__}.build_dataset"
+                    "documentation."
+                )
+        else:
+            dataset = ESRNNPredDataset(
+                y=y,
+                window=self.window,
+            )
+        return DataLoader(
+            dataset,
+            self.batch_size,
+        )
+
+    @classmethod
+    def get_test_params(cls, parameter_set="default"):
+        """Return testing parameter settings for the estimator.
+
+        Parameters
+        ----------
+        parameter_set : str, default="default"
+            Name of the set of test parameters to return, for use in tests. If no
+            special parameters are defined for a value, will return ``"default"`` set.
+            Reserved values for classifiers:
+                "results_comparison" - used for identity testing in some classifiers
+                should contain parameter settings comparable to "TSC bakeoff"
+
+        Returns
+        -------
+        params : dict or list of dict, default = {}
+            Parameters to create testing instances of the class
+            Each dict are parameters to construct an "interesting" test instance, i.e.,
+            ``MyClass(**params)`` or ``MyClass(**params[i])`` creates a valid test
+            instance.
+            ``create_test_instance`` uses the first (or only) dictionary in ``params``
+        """
+        params1 = {}
+        params2 = {
+            "hidden_size": 10,
+            "num_layer": 5,
+            "season1_length": 2,
+            "season2_length": 2,
+            "seasonality_type": "single",
+            "window": 3,
+            "stride": 1,
+            "batch_size": 32,
+            "num_epochs": 10,
+            "custom_dataset_train": None,
+            "custom_dataset_pred": None,
+            "optimizer": "Adam",
+            "lr": 1e-3,
+        }
+        return [params1, params2]
diff --git a/sktime/networks/es_rnn.py b/sktime/networks/es_rnn.py
new file mode 100644
index 000000000..1be5065cf
--- /dev/null
+++ b/sktime/networks/es_rnn.py
@@ -0,0 +1,351 @@
+"""Exponential Smoothing Recurrant Neural Network (ES-RNN)."""
+
+__author__ = ["Ankit-1204"]
+
+from warnings import warn
+
+from sktime.utils.dependencies import _check_soft_dependencies
+
+
+class ESRNN:
+    """
+    Exponential Smoothing Recurrant Neural Network.
+
+    This model combines Exponential Smoothing (ES) and (LSTM) networks
+    for time series forecasting. ES is used to balance the level and
+    seasonality of the series.
+
+    Parameters
+    ----------
+    input_shape : int
+        Number of features in the input
+    hidden_size : int
+        Number of features in the hidden state
+    pred_len : int
+        Forecasting horizon
+    num_layer : int
+        Number of layers
+    season1_length : int
+        Period of season
+    seasonality : string
+        Type of seasonality
+
+    References
+    ----------
+    .. [1] Smyl, S. 2020.
+    A hybrid method of exponential smoothing and recurrent \
+    neural networks for time series forecasting.
+    https://www.sciencedirect.com/science/article/pii/S0169207019301153
+    """
+
+    _tags = {
+        # packaging info
+        # --------------
+        "authors": ["Ankit-1204"],
+    }
+
+    def __init__(
+        self,
+        input_shape=1,
+        hidden_size=1,
+        pred_len=1,
+        num_layer=1,
+        season1_length=12,
+        season2_length=2,
+        seasonality="single",
+    ) -> None:
+        super().__init__()
+
+        self.input_shape = input_shape
+        self.hidden_size = hidden_size
+        self.num_layer = num_layer
+        self.pred_len = pred_len
+        self.season1_length = season1_length
+        self.season2_length = season2_length
+        self.seasonality = seasonality
+
+        if _check_soft_dependencies("torch", severity="none"):
+            import torch
+            import torch.nn as nn
+
+            NNModule = nn.Module
+        else:
+
+            class NNModule:
+                """Dummy class if torch is unavailable."""
+
+        class PinballLoss(NNModule):
+            """
+            Default Pinball/Quantile Loss.
+
+            Parameters
+            ----------
+            tau : Quantile Value
+            target : Ground truth
+            predec: Predicted value
+            loss = max( (predec-target)(1-tau), (target-predec)*tau)
+            """
+
+            def __init__(self, tau=0.49):
+                super().__init__()
+                self.tau = tau
+
+            def forward(self, predec, target):
+                """Calculate Pinball Loss."""
+                predec = predec.float()
+                target = target.float()
+                diff = predec - target
+                loss = torch.maximum(-diff * (1 - self.tau), diff * self.tau)
+                return loss.mean()
+
+        class _ESRNN(NNModule):
+            def __init__(
+                self,
+                input_shape,
+                hidden_size,
+                pred_len,
+                num_layer,
+                season1_length,
+                season2_length,
+                seasonality,
+            ) -> None:
+                self.input_shape = input_shape
+                self.hidden_size = hidden_size
+                self.num_layer = num_layer
+                self.pred_len = pred_len
+                self.seasonality = seasonality
+                self.season1_length = season1_length
+                self.season2_length = season2_length
+                super().__init__()
+                self.level_coeff = torch.nn.Parameter(torch.rand(1), requires_grad=True)
+                self.seasonal_coeff_1 = torch.nn.Parameter(
+                    torch.rand(1), requires_grad=True
+                )
+                self.seasonal_coeff_2 = torch.nn.Parameter(
+                    torch.rand(1), requires_grad=True
+                )
+                self.input_layer = nn.Linear(input_shape, input_shape)
+                self.lstm = nn.LSTM(
+                    self.input_shape, self.hidden_size, self.num_layer, batch_first=True
+                )
+                self.output_layer = nn.Linear(self.hidden_size, self.input_shape)
+
+            def _nonseasonal(self, x):
+                """
+                Calculate level component for time series without seasonal patterns.
+
+                Args:
+                    x: Input tensor of shape (batch, seq_length, num_features)
+
+                Returns
+                -------
+                    tuple: (level, remainder)
+                """
+                batch, seq_length, num_features = x.shape
+                level = x[:, 0, :]
+                level_coeff = torch.sigmoid(self.level_coeff)
+                for t in range(1, seq_length):
+                    level = level_coeff * x[:, t, :] + (1 - level_coeff) * level
+
+                return level, x / level.unsqueeze(1)
+
+            def _single_seasonal(self, x):
+                """
+                Calculate and return the level and one seasonality components.
+
+                Used for single seasonality condition where the time series exhibits
+                one distinct seasonal patterns (e.g., Yearly pattern).
+
+                Args:
+                    x: Input tensor of shape (batch, seq_length, num_features)
+
+                Returns
+                -------
+                    tuple: (level, seasonality1, remainder)
+                """
+                batch, seq_length, num_features = x.shape
+                season1_length = self.season1_length
+                if self.season1_length > seq_length:
+                    warn(f"Input window should atleast cover one season,{seq_length}")
+                    season1_length = seq_length
+                level = x[:, :season1_length, :].mean(dim=1, keepdim=True)
+                initial_seasonality_1 = x[:, :season1_length, :] / level
+                seasonality_1 = []
+                for i in range(season1_length):
+                    seasonality_1.append(torch.exp(initial_seasonality_1[:, i, :]))
+                level_coeff = torch.sigmoid(self.level_coeff)
+                seasonal_coeff_1 = torch.sigmoid(self.seasonal_coeff_1)
+                for i in range(seq_length):
+                    new_level = level_coeff * (x[:, i, :] / seasonality_1[i]) + (
+                        1 - level_coeff
+                    ) * level.squeeze(1)
+
+                    seasonality_1.append(
+                        seasonal_coeff_1 * (x[:, i, :] / new_level)
+                        + (1 - seasonal_coeff_1) * seasonality_1[i]
+                    )
+                    level = new_level.unsqueeze(1)
+                seasonality_1 = torch.stack(seasonality_1, dim=1)
+                return (
+                    level,
+                    seasonality_1,
+                    x / (level * seasonality_1[:, -seq_length:, :]),
+                )
+
+            def _double_seasonal(self, x):
+                """
+                Calculate and return the level and two seasonality components.
+
+                Used for double seasonality condition where the time series exhibits
+                two distinct seasonal patterns (e.g., daily and weekly patterns).
+
+                Args:
+                    x: Input tensor of shape (batch, seq_length, num_features)
+
+                Returns
+                -------
+                    tuple: (level, seasonality1, seasonality2, remainder)
+                """
+                batch, seq_length, num_features = x.shape
+                season1_length = self.season1_length
+                if self.season1_length > seq_length:
+                    season1_length = seq_length
+                    warn(f"Input window should atleast cover one season,{seq_length}")
+
+                season2_length = self.season2_length
+                if self.season2_length > seq_length:
+                    season2_length = seq_length
+                    warn(f"Input window should atleast cover one season,{seq_length}")
+
+                level = x[:, : max(season1_length, season2_length), :].mean(
+                    dim=1, keepdim=True
+                )
+                initial_seasonality_1 = x[:, :season1_length, :] / level
+                initial_seasonality_2 = x[:, :season2_length, :] / level
+                seasonality_1 = []
+                seasonality_2 = []
+                for i in range(season1_length):
+                    seasonality_1.append(torch.exp(initial_seasonality_1[:, i, :]))
+                for i in range(season2_length):
+                    seasonality_2.append(torch.exp(initial_seasonality_2[:, i, :]))
+
+                level_coeff = torch.sigmoid(self.level_coeff)
+                seasonal_coeff_1 = torch.sigmoid(self.seasonal_coeff_1)
+                seasonal_coeff_2 = torch.sigmoid(self.seasonal_coeff_2)
+                for i in range(seq_length):
+                    new_level = level_coeff * (
+                        x[:, i, :] / seasonality_1[i] * seasonality_2[i]
+                    ) + (1 - level_coeff) * level.squeeze(1)
+
+                    seasonality_1.append(
+                        seasonal_coeff_1 * (x[:, i, :] / new_level * seasonality_2[i])
+                        + (1 - seasonal_coeff_1) * seasonality_1[i]
+                    )
+                    seasonality_2.append(
+                        seasonal_coeff_2 * (x[:, i, :] / new_level * seasonality_1[i])
+                        + (1 - seasonal_coeff_2) * seasonality_2[i]
+                    )
+                    level = new_level.unsqueeze(1)
+                seasonality_1 = torch.stack(seasonality_1, dim=1)
+                seasonality_2 = torch.stack(seasonality_2, dim=1)
+                return (
+                    level,
+                    seasonality_1,
+                    seasonality_2,
+                    x
+                    / (
+                        level
+                        * seasonality_1[:, -seq_length:, :]
+                        * seasonality_2[:, -seq_length:, :]
+                    ),
+                )
+
+            def forward(self, x):
+                """
+                Forward pass through ES-RNN.
+
+                Parameters
+                ----------
+                x : torch.Tensor
+                    Input tensor of shape (batch_size, input_length).
+                """
+                if self.seasonality == "zero":
+                    level, new_x = self._nonseasonal(x)
+                    x_input = self.input_layer(new_x.float())
+                    output, (h, c) = self.lstm(x_input)
+                    last_output = self.output_layer(output[:, -1, :]).unsqueeze(1)
+                    out_list = [last_output]
+                    for t in range(self.pred_len - 1):
+                        next_out = self.input_layer(last_output)
+                        next_out = next_out
+                        lstm_out, (h, c) = self.lstm(next_out, (h, c))
+                        next_output = self.output_layer(lstm_out)
+                        out_list.append(next_output)
+                        last_output = next_output
+                    output_seq = torch.stack(out_list, dim=0)
+                    output_seq = output_seq.squeeze(2).transpose(0, 1)
+                    output_leveled = (output_seq) * level.unsqueeze(1)
+                    return output_leveled
+
+                elif self.seasonality == "single":
+                    level, seasonality_1, new_x = self._single_seasonal(x)
+                    x_input = self.input_layer(new_x.float())
+                    output, (h, c) = self.lstm(x_input)
+                    last_output = self.output_layer(output[:, -1, :]).unsqueeze(1)
+                    out_list = [last_output]
+                    for t in range(self.pred_len - 1):
+                        next_out = self.input_layer(last_output)
+                        next_out = next_out
+                        lstm_out, (h, c) = self.lstm(next_out, (h, c))
+                        next_output = self.output_layer(lstm_out)
+                        out_list.append(next_output)
+                        last_output = next_output
+                    output_seq = torch.stack(out_list, dim=0)
+                    output_seq = output_seq.squeeze(2).transpose(0, 1)
+                    output_leveled = (
+                        (output_seq) * level * seasonality_1[:, -self.pred_len :, :]
+                    )
+                    return output_leveled
+                else:
+                    level, seasonality_1, seasonality_2, new_x = self._double_seasonal(
+                        x
+                    )
+                    x_input = self.input_layer(new_x.float())
+                    output, (h, c) = self.lstm(x_input)
+                    last_output = self.output_layer(output[:, -1, :]).unsqueeze(1)
+                    out_list = [last_output]
+                    for t in range(self.pred_len - 1):
+                        next_out = self.input_layer(last_output)
+                        next_out = next_out
+                        lstm_out, (h, c) = self.lstm(next_out, (h, c))
+                        next_output = self.output_layer(lstm_out)
+                        out_list.append(next_output)
+                        last_output = next_output
+                    output_seq = torch.stack(out_list, dim=0)
+                    output_seq = output_seq.squeeze(2).transpose(0, 1)
+                    output_leveled = (
+                        (output_seq)
+                        * level
+                        * seasonality_1[:, -self.pred_len :, :]
+                        * seasonality_2[:, -self.pred_len :, :]
+                    )
+                    return output_leveled
+
+        self._network_class = _ESRNN
+        self.loss = PinballLoss
+
+    def pin_ball(self):
+        """Return the default Pinball Loss."""
+        return self.loss()
+
+    def _build(self):
+        """Build the ES-RNN."""
+        return self._network_class(
+            self.input_shape,
+            self.hidden_size,
+            self.pred_len,
+            self.num_layer,
+            self.season1_length,
+            self.season2_length,
+            self.seasonality,
+        )


Test patch:
diff --git a/sktime/registry/tests/test_craft.py b/sktime/registry/tests/test_craft.py
index b9b6e4dc7..79d005524 100644
--- a/sktime/registry/tests/test_craft.py
+++ b/sktime/registry/tests/test_craft.py
@@ -102,7 +102,7 @@ def test_deps(spec):
     assert deps(pipe_spec_with_deps) == ["statsmodels"]
 
     # example with two dependencies, should be identified, order does not matter
-    assert set(deps(dunder_spec_with_deps)) == {"statsmodels", "pmdarima"}
+    assert set(deps(dunder_spec_with_deps)) == {"statsmodels", "pmdarima", "numpy<2"}
 
 
 def test_imports():
diff --git a/sktime/tests/_config.py b/sktime/tests/_config.py
index 1db91f5f2..ce3294e78 100644
--- a/sktime/tests/_config.py
+++ b/sktime/tests/_config.py
@@ -152,9 +152,6 @@ EXCLUDED_TESTS = {
         "test_persistence_via_pickle",
         "test_save_estimators_to_file",
     ],
-    "MCDCNNClassifier": [
-        "test_fit_idempotent",
-    ],
     "MCDCNNRegressor": [
         "test_fit_idempotent",
     ],
@@ -247,6 +244,16 @@ EXCLUDED_TESTS = {
     ],
     # see PR 7921
     "RocketClassifier": ["test_classifier_on_basic_motions"],
+    # see bug report #6465 and #7958
+    "MACNNClassifier": [
+        "test_multioutput",
+        "test_classifier_on_unit_test_data",
+    ],
+    "MCDCNNClassifier": [
+        "test_multioutput",
+        "test_classifier_on_unit_test_data",
+        "test_fit_idempotent",  # not part of bug reports but due to randomness
+    ],
 }
 
 # exclude tests but keyed by test name