Repository: aeon-toolkit/aeon
Pull request #1608 (2024-06-05 14:28:52)

[ENH] Implement AEAttentionBiGRUNetwork
Implements Attention Bidirectional Gated Recurrent Unit based Auto Encoder .

aeon-actions-bot[bot]: 
## Thank you for contributing to `aeon`

I have added the following labels to this PR based on the title: **[ $\\color{#FEF1BE}{\\textsf{enhancement}}$ ]**.
I have added the following labels to this PR based on the changes made: **[ $\\color{#379E11}{\\textsf{networks}}$ ]**. Feel free to change these if they do not properly represent the PR.

The [Checks](https://github.com/aeon-toolkit/aeon/pull/1608/checks) tab will show the status of our automated tests. You can click on individual test runs in the tab or "Details" in the panel below to see more information if there is a failure.

If our `pre-commit` code quality check fails, any trivial fixes will automatically be pushed to your PR unless it is a draft.

Don't hesitate to ask questions on the `aeon` [Slack](
https://join.slack.com/t/aeon-toolkit/shared_invite/zt-22vwvut29-HDpCu~7VBUozyfL_8j3dLA) channel if you have any.
    
aadya940: @hadifawaz1999 How can we use the Keras Subclassing API in this case, since the tests that don't have tensorflow will fail?
MatthewMiddlehurst: See the Hydra transformation for ideas, there is ways to get around it. Not sure you can do local classes in Python? Probably dont want to either way 🙂 
aadya940: @MatthewMiddlehurst Thanks :-)
aadya940: @hadifawaz1999 What will change if we add support for `temporal_latent_space` in this? Do we apply the `Attention` and `self._gate` if `temporal_latent_space` is True since `self._gate` will Flatten the input tensor and convert it from 3 Dimensions to 2 Dimension and the output will no longer be temporal? 
hadifawaz1999: > See the Hydra transformation for ideas, there is ways to get around it. Not sure you can do local classes in Python? Probably dont want to either way 🙂

yup should work like we have in hydra
hadifawaz1999: > @hadifawaz1999 What will change if we add support for `temporal_latent_space` in this? Do we apply the `Attention` and `self._gate` if `temporal_latent_space` is True since `self._gate` will Flatten the input tensor and convert it from 3 Dimensions to 2 Dimension and the output will no longer be temporal?

let me check the architecture and get back to you
hadifawaz1999: > > @hadifawaz1999 What will change if we add support for `temporal_latent_space` in this? Do we apply the `Attention` and `self._gate` if `temporal_latent_space` is True since `self._gate` will Flatten the input tensor and convert it from 3 Dimensions to 2 Dimension and the output will no longer be temporal?
> 
> let me check the architecture and get back to you

hadifawaz1999: 
if i understand correctly, the number of params for RNN layers depend on the batch size
right ? as it depends on x.shape[0]
aadya940: Yep, that's what I understood. 
aadya940: @hadifawaz1999 Can we really parametrize the number of layers here? Since,  after the input passes through a Dense layers , its dimensions decrease from 3D to 2D and hence, we cant pass it to the next GRU layer. Also, the implementation here uses just 1 layer in the encoder, as well as the decoder:
https://github.com/blafabregue/TimeSeriesDeepClustering/blob/main/networks/attention_rnn.py
aadya940: Could we just expand the dims of the tensor after each dense layer as follows?
```
if not (i == self.n_layers_encoder - 1):
                x = tf.expand_dims(x, -1)
```
hadifawaz1999: should merge main here too @aadya940 
aadya940: #1962

Patch:
diff --git a/.all-contributorsrc b/.all-contributorsrc
index 3c7419da..c6e9a8c1 100644
--- a/.all-contributorsrc
+++ b/.all-contributorsrc
@@ -2454,7 +2454,8 @@
       "profile": "https://github.com/Cyril-Meyer",
       "contributions": [
         "test",
-        "doc"
+        "doc",
+        "code"
       ]
     },
     {
diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md
index ca17b59a..096c600d 100644
--- a/CONTRIBUTORS.md
+++ b/CONTRIBUTORS.md
@@ -1,7 +1,7 @@
 # Contributors
 
 <!-- ALL-CONTRIBUTORS-BADGE:START - Do not remove or modify this section -->
-[![All Contributors](https://img.shields.io/badge/all_contributors-243-orange.svg)](#contributors)
+[![All Contributors](https://img.shields.io/badge/all_contributors-244-orange.svg)](#contributors)
 <!-- ALL-CONTRIBUTORS-BADGE:END -->
 
 This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome!
@@ -76,7 +76,7 @@ Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/d
     <tr>
       <td align="center" valign="top" width="12.5%"><a href="https://github.com/ClaudiaSanches"><img src="https://avatars3.githubusercontent.com/u/28742178?v=4?s=80" width="80px;" alt="ClaudiaSanches"/><br /><sub><b>ClaudiaSanches</b></sub></a><br /><a href="https://github.com/aeon-toolkit/aeon/commits?author=ClaudiaSanches" title="Code">💻</a> <a href="https://github.com/aeon-toolkit/aeon/commits?author=ClaudiaSanches" title="Tests">⚠️</a></td>
       <td align="center" valign="top" width="12.5%"><a href="https://sites.google.com/view/corvinpaul/"><img src="https://avatars.githubusercontent.com/u/5957191?v=4?s=80" width="80px;" alt="Corvin Paul"/><br /><sub><b>Corvin Paul</b></sub></a><br /><a href="https://github.com/aeon-toolkit/aeon/commits?author=corvusrabus" title="Documentation">📖</a></td>
-      <td align="center" valign="top" width="12.5%"><a href="https://github.com/Cyril-Meyer"><img src="https://avatars.githubusercontent.com/u/69190238?v=4?s=80" width="80px;" alt="Cyril Meyer"/><br /><sub><b>Cyril Meyer</b></sub></a><br /><a href="https://github.com/aeon-toolkit/aeon/commits?author=Cyril-Meyer" title="Tests">⚠️</a> <a href="https://github.com/aeon-toolkit/aeon/commits?author=Cyril-Meyer" title="Documentation">📖</a></td>
+      <td align="center" valign="top" width="12.5%"><a href="https://github.com/Cyril-Meyer"><img src="https://avatars.githubusercontent.com/u/69190238?v=4?s=80" width="80px;" alt="Cyril Meyer"/><br /><sub><b>Cyril Meyer</b></sub></a><br /><a href="https://github.com/aeon-toolkit/aeon/commits?author=Cyril-Meyer" title="Tests">⚠️</a> <a href="https://github.com/aeon-toolkit/aeon/commits?author=Cyril-Meyer" title="Documentation">📖</a> <a href="https://github.com/aeon-toolkit/aeon/commits?author=Cyril-Meyer" title="Code">💻</a></td>
       <td align="center" valign="top" width="12.5%"><a href="https://github.com/DBCerigo"><img src="https://avatars.githubusercontent.com/u/8318425?v=4?s=80" width="80px;" alt="Daniel Burkhardt Cerigo"/><br /><sub><b>Daniel Burkhardt Cerigo</b></sub></a><br /><a href="https://github.com/aeon-toolkit/aeon/commits?author=DBCerigo" title="Code">💻</a></td>
       <td align="center" valign="top" width="12.5%"><a href="https://www.linkedin.com/in/dnylam/"><img src="https://avatars.githubusercontent.com/u/30182852?v=4?s=80" width="80px;" alt="Daniel L."/><br /><sub><b>Daniel L.</b></sub></a><br /><a href="https://github.com/aeon-toolkit/aeon/commits?author=Datadote" title="Documentation">📖</a></td>
       <td align="center" valign="top" width="12.5%"><a href="https://www.linkedin.com/in/daniel-martin-martinez"><img src="https://avatars.githubusercontent.com/dainelli98?s=80" width="80px;" alt="Daniel Martín Martínez"/><br /><sub><b>Daniel Martín Martínez</b></sub></a><br /><a href="https://github.com/aeon-toolkit/aeon/commits?author=dainelli98" title="Documentation">📖</a> <a href="https://github.com/aeon-toolkit/aeon/issues?q=author%3Adainelli98" title="Bug reports">🐛</a></td>
@@ -298,13 +298,13 @@ Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/d
       <td align="center" valign="top" width="12.5%"><a href="https://github.com/julu98"><img src="https://avatars.githubusercontent.com/u/133243896?v=4?s=80" width="80px;" alt="julu98"/><br /><sub><b>julu98</b></sub></a><br /><a href="https://github.com/aeon-toolkit/aeon/issues?q=author%3Ajulu98" title="Bug reports">🐛</a></td>
       <td align="center" valign="top" width="12.5%"><a href="https://github.com/kkoziara"><img src="https://avatars1.githubusercontent.com/u/4346849?v=4?s=80" width="80px;" alt="kkoziara"/><br /><sub><b>kkoziara</b></sub></a><br /><a href="https://github.com/aeon-toolkit/aeon/commits?author=kkoziara" title="Code">💻</a> <a href="https://github.com/aeon-toolkit/aeon/issues?q=author%3Akkoziara" title="Bug reports">🐛</a></td>
       <td align="center" valign="top" width="12.5%"><a href="https://github.com/matteogales"><img src="https://avatars0.githubusercontent.com/u/9269326?v=4?s=80" width="80px;" alt="matteogales"/><br /><sub><b>matteogales</b></sub></a><br /><a href="https://github.com/aeon-toolkit/aeon/commits?author=matteogales" title="Code">💻</a> <a href="#design-matteogales" title="Design">🎨</a> <a href="#ideas-matteogales" title="Ideas, Planning, & Feedback">🤔</a></td>
-      <td align="center" valign="top" width="12.5%"><a href="https://github.com/neuron283"><img src="https://avatars.githubusercontent.com/u/177816976?v=4?s=80" width="80px;" alt="neuron283"/><br /><sub><b>neuron283</b></sub></a><br /><a href="#maintenance-neuron283" title="Maintenance">🚧</a></td>
+      <td align="center" valign="top" width="12.5%"><a href="https://github.com/neuron283"><img src="https://avatars.githubusercontent.com/u/177816976?v=4?s=80" width="80px;" alt="neuron283"/><br /><sub><b>neuron283</b></sub></a><br /><a href="https://github.com/aeon-toolkit/aeon/commits?author=neuron283" title="Code">💻</a></td>
       <td align="center" valign="top" width="12.5%"><a href="https://github.com/nileenagp"><img src="https://avatars.githubusercontent.com/u/48993630?v=4?s=80" width="80px;" alt="nileenagp"/><br /><sub><b>nileenagp</b></sub></a><br /><a href="https://github.com/aeon-toolkit/aeon/commits?author=nileenagp" title="Code">💻</a></td>
       <td align="center" valign="top" width="12.5%"><a href="https://github.com/oleskiewicz"><img src="https://avatars1.githubusercontent.com/u/5682158?v=4?s=80" width="80px;" alt="oleskiewicz"/><br /><sub><b>oleskiewicz</b></sub></a><br /><a href="https://github.com/aeon-toolkit/aeon/commits?author=oleskiewicz" title="Code">💻</a> <a href="https://github.com/aeon-toolkit/aeon/commits?author=oleskiewicz" title="Documentation">📖</a> <a href="https://github.com/aeon-toolkit/aeon/commits?author=oleskiewicz" title="Tests">⚠️</a></td>
       <td align="center" valign="top" width="12.5%"><a href="https://github.com/pabworks"><img src="https://avatars.githubusercontent.com/u/32725127?v=4?s=80" width="80px;" alt="pabworks"/><br /><sub><b>pabworks</b></sub></a><br /><a href="https://github.com/aeon-toolkit/aeon/commits?author=pabworks" title="Code">💻</a> <a href="https://github.com/aeon-toolkit/aeon/commits?author=pabworks" title="Tests">⚠️</a></td>
-      <td align="center" valign="top" width="12.5%"><a href="https://github.com/xloem"><img src="https://avatars.githubusercontent.com/u/279585?v=4?s=80" width="80px;" alt="patiently pending world peace"/><br /><sub><b>patiently pending world peace</b></sub></a><br /><a href="https://github.com/aeon-toolkit/aeon/commits?author=xloem" title="Code">💻</a></td>
     </tr>
     <tr>
+      <td align="center" valign="top" width="12.5%"><a href="https://github.com/xloem"><img src="https://avatars.githubusercontent.com/u/279585?v=4?s=80" width="80px;" alt="patiently pending world peace"/><br /><sub><b>patiently pending world peace</b></sub></a><br /><a href="https://github.com/aeon-toolkit/aeon/commits?author=xloem" title="Code">💻</a></td>
       <td align="center" valign="top" width="12.5%"><a href="https://github.com/raishubham1"><img src="https://avatars3.githubusercontent.com/u/29356417?v=4?s=80" width="80px;" alt="raishubham1"/><br /><sub><b>raishubham1</b></sub></a><br /><a href="https://github.com/aeon-toolkit/aeon/commits?author=raishubham1" title="Documentation">📖</a></td>
       <td align="center" valign="top" width="12.5%"><a href="https://github.com/simone-pignotti"><img src="https://avatars1.githubusercontent.com/u/44410066?v=4?s=80" width="80px;" alt="simone-pignotti"/><br /><sub><b>simone-pignotti</b></sub></a><br /><a href="https://github.com/aeon-toolkit/aeon/commits?author=simone-pignotti" title="Code">💻</a> <a href="https://github.com/aeon-toolkit/aeon/issues?q=author%3Asimone-pignotti" title="Bug reports">🐛</a></td>
       <td align="center" valign="top" width="12.5%"><a href="https://github.com/sophijka"><img src="https://avatars2.githubusercontent.com/u/47450591?v=4?s=80" width="80px;" alt="sophijka"/><br /><sub><b>sophijka</b></sub></a><br /><a href="https://github.com/aeon-toolkit/aeon/commits?author=sophijka" title="Documentation">📖</a> <a href="#maintenance-sophijka" title="Maintenance">🚧</a></td>
@@ -312,9 +312,9 @@ Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/d
       <td align="center" valign="top" width="12.5%"><a href="https://github.com/tensorflow-as-tf"><img src="https://avatars.githubusercontent.com/u/51345718?v=4?s=80" width="80px;" alt="tensorflow-as-tf"/><br /><sub><b>tensorflow-as-tf</b></sub></a><br /><a href="https://github.com/aeon-toolkit/aeon/commits?author=tensorflow-as-tf" title="Code">💻</a></td>
       <td align="center" valign="top" width="12.5%"><a href="https://github.com/vNtzYy"><img src="https://avatars.githubusercontent.com/u/111287448?v=4?s=80" width="80px;" alt="vNtzYy"/><br /><sub><b>vNtzYy</b></sub></a><br /><a href="https://github.com/aeon-toolkit/aeon/issues?q=author%3AvNtzYy" title="Bug reports">🐛</a></td>
       <td align="center" valign="top" width="12.5%"><a href="https://github.com/vedazeren"><img src="https://avatars3.githubusercontent.com/u/63582874?v=4?s=80" width="80px;" alt="vedazeren"/><br /><sub><b>vedazeren</b></sub></a><br /><a href="https://github.com/aeon-toolkit/aeon/commits?author=vedazeren" title="Code">💻</a> <a href="https://github.com/aeon-toolkit/aeon/commits?author=vedazeren" title="Tests">⚠️</a></td>
-      <td align="center" valign="top" width="12.5%"><a href="https://github.com/vincent-nich12"><img src="https://avatars3.githubusercontent.com/u/36476633?v=4?s=80" width="80px;" alt="vincent-nich12"/><br /><sub><b>vincent-nich12</b></sub></a><br /><a href="https://github.com/aeon-toolkit/aeon/commits?author=vincent-nich12" title="Code">💻</a></td>
     </tr>
     <tr>
+      <td align="center" valign="top" width="12.5%"><a href="https://github.com/vincent-nich12"><img src="https://avatars3.githubusercontent.com/u/36476633?v=4?s=80" width="80px;" alt="vincent-nich12"/><br /><sub><b>vincent-nich12</b></sub></a><br /><a href="https://github.com/aeon-toolkit/aeon/commits?author=vincent-nich12" title="Code">💻</a></td>
       <td align="center" valign="top" width="12.5%"><a href="https://github.com/vollmersj"><img src="https://avatars2.githubusercontent.com/u/12613127?v=4?s=80" width="80px;" alt="vollmersj"/><br /><sub><b>vollmersj</b></sub></a><br /><a href="https://github.com/aeon-toolkit/aeon/commits?author=vollmersj" title="Documentation">📖</a></td>
       <td align="center" valign="top" width="12.5%"><a href="https://github.com/xiaobenbenecho"><img src="https://avatars.githubusercontent.com/u/17461849?v=4?s=80" width="80px;" alt="xiaobenbenecho"/><br /><sub><b>xiaobenbenecho</b></sub></a><br /><a href="https://github.com/aeon-toolkit/aeon/commits?author=xiaobenbenecho" title="Code">💻</a></td>
       <td align="center" valign="top" width="12.5%"><a href="https://github.com/xiaopu222"><img src="https://avatars.githubusercontent.com/u/118000577?v=4?s=80" width="80px;" alt="xiaopu222"/><br /><sub><b>xiaopu222</b></sub></a><br /><a href="https://github.com/aeon-toolkit/aeon/commits?author=xiaopu222" title="Documentation">📖</a></td>
diff --git a/aeon/networks/__init__.py b/aeon/networks/__init__.py
index 2087b4e6..dd7b90f8 100644
--- a/aeon/networks/__init__.py
+++ b/aeon/networks/__init__.py
@@ -14,9 +14,12 @@ __all__ = [
     "AEFCNNetwork",
     "AEResNetNetwork",
     "LITENetwork",
+    "AEAttentionBiGRUNetwork",
     "AEDRNNNetwork",
     "AEBiGRUNetwork",
 ]
+
+from aeon.networks._ae_abgru import AEAttentionBiGRUNetwork
 from aeon.networks._ae_bgru import AEBiGRUNetwork
 from aeon.networks._ae_drnn import AEDRNNNetwork
 from aeon.networks._ae_fcn import AEFCNNetwork
diff --git a/aeon/networks/_ae_abgru.py b/aeon/networks/_ae_abgru.py
new file mode 100644
index 00000000..ea331396
--- /dev/null
+++ b/aeon/networks/_ae_abgru.py
@@ -0,0 +1,181 @@
+"""Implements an Auto-Encoder based on Attention Bidirectional GRUs."""
+
+from aeon.networks.base import BaseDeepLearningNetwork
+
+
+class AEAttentionBiGRUNetwork(BaseDeepLearningNetwork):
+    """
+    A class to implement an Auto-Encoder based on Attention Bidirectional GRUs.
+
+    Parameters
+    ----------
+        latent_space_dim : int, default=128
+            Dimension of the latent space.
+        temporal_latent_space : bool, default=False
+            Flag to choose whether the latent space is an MTS or Euclidean space.
+        n_layers_encoder : int, default=None
+            Number of Attention BiGRU layers in the encoder.
+            If None, one layer will be used.
+        n_layers_decoder : int, default=None
+            Number of Attention BiGRU layers in the decoder.
+            If None, one layer will be used.
+        activation_encoder : Union[list, str], default="relu"
+            Activation function(s) to use in each layer of the encoder.
+            Can be a single string or a list.
+        activation_decoder : Union[list, str], default="relu"
+            Activation function(s) to use in each layer of the decoder.
+            Can be a single string or a list.
+
+    References
+    ----------
+    .. [1] Ienco, D., & Interdonato, R. (2020). Deep multivariate time series
+    embedding clustering via attentive-gated autoencoder. In Advances in Knowledge
+    Discovery and Data Mining: 24th Pacific-Asia Conference, PAKDD 2020, Singapore,
+    May 11-14, 2020, Proceedings, Part I 24 (pp. 318-329). Springer International
+    Publishing.
+
+    """
+
+    _config = {
+        "python_dependencies": ["tensorflow"],
+        "python_version": "<3.12",
+        "structure": "auto-encoder",
+    }
+
+    def __init__(
+        self,
+        latent_space_dim=None,
+        temporal_latent_space=False,
+        n_layers_encoder=1,
+        n_layers_decoder=1,
+        activation_encoder="relu",
+        activation_decoder="relu",
+    ):
+        super().__init__()
+
+        self.latent_space_dim = latent_space_dim
+        self.temporal_latent_space = temporal_latent_space
+        self.activation_encoder = activation_encoder
+        self.activation_decoder = activation_decoder
+        self.n_layers_encoder = n_layers_encoder
+        self.n_layers_decoder = n_layers_decoder
+
+    def build_network(self, input_shape, **kwargs):
+        """Construct a network and return its input and output layers.
+
+        Arguments
+        ---------
+        input_shape : tuple of shape = (n_timepoints (m), n_channels (d))
+            The shape of the data fed into the input layer.
+        kwargs : dict
+            Contains additional keyword arguments to be passed to the
+            `build_network` method like `num_input_samples`.
+
+        Returns
+        -------
+        encoder : a keras Model.
+        decoder : a keras Model.
+        """
+        import tensorflow as tf
+
+        if isinstance(self.activation_encoder, str):
+            self._activation_encoder = [
+                self.activation_encoder for _ in range(self.n_layers_encoder)
+            ]
+        else:
+            self._activation_encoder = self.activation_encoder
+            assert isinstance(self.activation_encoder, list)
+            assert len(self.activation_encoder) == self.n_layers_encoder
+
+        if isinstance(self.activation_decoder, str):
+            self._activation_decoder = [
+                self.activation_decoder for _ in range(self.n_layers_decoder)
+            ]
+        else:
+            self._activation_decoder = self.activation_decoder
+            assert isinstance(self.activation_decoder, list)
+            assert len(self.activation_decoder) == self.n_layers_decoder
+
+        if not isinstance(self.n_layers_encoder, int):
+            raise ValueError("Number of layers of the encoder must be an integer.")
+
+        if not isinstance(self.n_layers_decoder, int):
+            raise ValueError("Number of layers of the decoder must be an integer.")
+
+        input_layer = tf.keras.layers.Input(input_shape)
+        x = input_layer
+
+        if self.latent_space_dim is None:
+            if "num_input_samples" in kwargs.keys():
+                self.n_filters_RNN = 64
+                if kwargs["input_samples"] > 250:
+                    self.n_filters_RNN = 512
+            else:
+                self.latent_space_dim = 64
+                self.n_filters_RNN = self.latent_space_dim
+        elif self.latent_space_dim is not None:
+            self.n_filters_RNN = self.latent_space_dim
+
+        for i in range(self.n_layers_encoder):
+            forward_layer = tf.keras.layers.GRU(
+                self.n_filters_RNN,
+                activation=self._activation_encoder[i],
+                return_sequences=True,
+            )(x)
+            backward_layer = tf.keras.layers.GRU(
+                self.n_filters_RNN,
+                activation=self._activation_encoder[i],
+                return_sequences=True,
+                go_backwards=True,
+            )(x)
+
+            query = tf.keras.layers.Dense(self.n_filters_RNN)(forward_layer)
+            key = tf.keras.layers.Dense(self.n_filters_RNN)(backward_layer)
+            value = tf.keras.layers.Dense(self.n_filters_RNN)(backward_layer)
+
+            attention_layer = tf.keras.layers.Attention()([query, key, value])
+            x = tf.keras.layers.Dense(self.n_filters_RNN, activation="sigmoid")(
+                attention_layer
+            )
+            x = x * attention_layer
+
+        if not self.temporal_latent_space:
+            shape_before_flatten = x.shape[1:]
+            x = tf.keras.layers.Flatten()(x)
+            x = tf.keras.layers.Dense(self.latent_space_dim)(x)
+        elif self.temporal_latent_space:
+            x = tf.keras.layers.Conv1D(filters=self.latent_space_dim, kernel_size=1)(x)
+
+        encoder = tf.keras.models.Model(inputs=input_layer, outputs=x, name="encoder")
+
+        if not self.temporal_latent_space:
+            decoder_inputs = tf.keras.layers.Input(
+                shape=(self.latent_space_dim,), name="decoder_input"
+            )
+            x = tf.keras.layers.RepeatVector(input_shape[0], name="repeat_vector")(
+                decoder_inputs
+            )
+        else:
+            decoder_inputs = tf.keras.layers.Input(
+                shape=shape_before_flatten, name="decoder_input"
+            )
+            x = decoder_inputs
+
+        for i in range(self.n_layers_decoder - 1, -1, -1):
+            x = tf.keras.layers.Bidirectional(
+                tf.keras.layers.GRU(
+                    units=self.n_filters_RNN // 2,
+                    activation=self._activation_decoder[i],
+                    return_sequences=True,
+                ),
+                name=f"decoder_bgru_{i+1}",
+            )(x)
+
+        decoder_outputs = tf.keras.layers.TimeDistributed(
+            tf.keras.layers.Dense(input_shape[1]), name="decoder_output"
+        )(x)
+        decoder = tf.keras.models.Model(
+            inputs=decoder_inputs, outputs=decoder_outputs, name="decoder"
+        )
+
+        return encoder, decoder
diff --git a/aeon/transformations/series/__init__.py b/aeon/transformations/series/__init__.py
index c07f3c6a..5097207d 100644
--- a/aeon/transformations/series/__init__.py
+++ b/aeon/transformations/series/__init__.py
@@ -5,6 +5,7 @@ __all__ = [
     "BaseSeriesTransformer",
     "ClearSkyTransformer",
     "ClaSPTransformer",
+    "DFTSeriesTransformer",
     "Dobin",
     "MatrixProfileSeriesTransformer",
     "StatsModelsACF",
@@ -26,6 +27,7 @@ from aeon.transformations.series._bkfilter import BKFilter
 from aeon.transformations.series._boxcox import BoxCoxTransformer
 from aeon.transformations.series._clasp import ClaSPTransformer
 from aeon.transformations.series._clear_sky import ClearSkyTransformer
+from aeon.transformations.series._dft import DFTSeriesTransformer
 from aeon.transformations.series._dobin import Dobin
 from aeon.transformations.series._matrix_profile import MatrixProfileSeriesTransformer
 from aeon.transformations.series._pca import PCASeriesTransformer
diff --git a/aeon/transformations/series/_dft.py b/aeon/transformations/series/_dft.py
new file mode 100644
index 00000000..1f603c39
--- /dev/null
+++ b/aeon/transformations/series/_dft.py
@@ -0,0 +1,88 @@
+"""Discrete Fourier Approximation filter transformation."""
+
+__maintainer__ = ["Cyril-Meyer"]
+__all__ = ["DFTSeriesTransformer"]
+
+
+import numpy as np
+
+from aeon.transformations.series.base import BaseSeriesTransformer
+
+
+class DFTSeriesTransformer(BaseSeriesTransformer):
+    """Filter a times series using Discrete Fourier Approximation (DFT).
+
+    Parameters
+    ----------
+    r : float
+        Proportion of Fourier terms to retain [0, 1]
+
+    sort : bool
+        Sort the Fourier terms by amplitude to keep most important terms
+
+    Notes
+    -----
+    More information of the NumPy FFT functions used
+    https://numpy.org/doc/stable/reference/routines.fft.html
+
+    References
+    ----------
+    .. [1] Cooley, J.W., & Tukey, J.W. (1965).
+       An algorithm for the machine calculation of complex Fourier series.
+       Mathematics of Computation, 19, 297-301.
+
+    Examples
+    --------
+    >>> import numpy as np
+    >>> from aeon.transformations.series._dft import DFTSeriesTransformer
+    >>> X = np.random.random((2, 100)) # Random series length 100
+    >>> dft = DFTSeriesTransformer()
+    >>> X_ = dft.fit_transform(X)
+    >>> X_.shape
+    (2, 100)
+    """
+
+    _tags = {
+        "capability:multivariate": True,
+        "X_inner_type": "np.ndarray",
+        "fit_is_empty": True,
+    }
+
+    def __init__(self, r=0.5, sort=False):
+        self.r = r
+        self.sort = sort
+        super().__init__(axis=1)
+
+    def _transform(self, X, y=None):
+        """Transform X and return a transformed version.
+
+        Parameters
+        ----------
+        X : np.ndarray
+            time series in shape (n_channels, n_timepoints)
+        y : ignored argument for interface compatibility
+
+        Returns
+        -------
+        transformed version of X
+        """
+        # Compute DFT
+        dft = np.fft.fft(X)
+
+        # Mask array of terms to keep and number of terms to keep
+        mask = np.zeros_like(dft, dtype=bool)
+        keep = max(int(self.r * dft.shape[1]), 1)
+
+        # If sort is set, sort the indices by the decreasing dft amplitude
+        if self.sort:
+            sorted_indices = np.argsort(np.abs(dft))[:, ::-1]
+            for i in range(dft.shape[0]):
+                mask[i, sorted_indices[i, 0:keep]] = True
+        # Else, keep the first terms
+        else:
+            mask[:, 0:keep] = True
+
+        # Invert DFT with masked terms
+        X_ = np.fft.ifft(dft * mask).real
+
+        return X_
diff --git a/docs/api_reference/transformations.rst b/docs/api_reference/transformations.rst
index a1b614a6..d34cc96d 100644
--- a/docs/api_reference/transformations.rst
+++ b/docs/api_reference/transformations.rst
@@ -186,6 +186,14 @@ Filtering and denoising
 
     BKFilter
 
+.. currentmodule:: aeon.transformations.series._dft
+
+.. autosummary::
+    :toctree: auto_generated/
+    :template: class.rst
+
+    DFTSeriesTransformer
+
 Slope
 ~~~~~~~~~~~~~~~~~~~~~~
 


Test patch:
diff --git a/aeon/transformations/series/tests/test_dft.py b/aeon/transformations/series/tests/test_dft.py
new file mode 100644
index 00000000..6ffe04b7
--- /dev/null
+++ b/aeon/transformations/series/tests/test_dft.py
@@ -0,0 +1,37 @@
+"""Tests for DFT transformation."""
+
+__maintainer__ = []
+
+import numpy as np
+import pytest
+
+
+@pytest.mark.parametrize("r", [0.00, 0.25, 0.50, 0.75, 1.00])
+@pytest.mark.parametrize("sort", [True, False])
+def test_dft(r, sort):
+    """Test the functionality of DFT transformation."""
+    n_samples = 100
+    t = np.linspace(0, 10, n_samples)
+    x1 = (
+        0.5 * np.sin(2 * np.pi * 1 * t)
+        + 0.2 * np.sin(2 * np.pi * 5 * t)
+        + 0.1 * np.sin(2 * np.pi * 10 * t)
+    )
+    x2 = (
+        0.4 * np.sin(2 * np.pi * 1.5 * t)
+        + 0.3 * np.sin(2 * np.pi * 4 * t)
+        + 0.1 * np.sin(2 * np.pi * 8 * t)
+    )
+    x12 = np.array([x1, x2])
+    x12r = x12 + np.random.random((2, n_samples)) * 0.25
+
+    from aeon.transformations.series._dft import DFTSeriesTransformer
+
+    dft = DFTSeriesTransformer(r=r, sort=sort)
+    x_1 = dft.fit_transform(x1)
+    x_2 = dft.fit_transform(x2)
+    x_12 = dft.fit_transform(x12)
+    dft.fit_transform(x12r)
+
+    np.testing.assert_almost_equal(x_1[0], x_12[0], decimal=4)
+    np.testing.assert_almost_equal(x_2[0], x_12[1], decimal=4)