Repository: aeon-toolkit/aeon
Pull request #2187 (2024-10-13 18:20:50)

[ENH] RotationForest coverage
more make work from me for a few lines of coverage. @MatthewMiddlehurst Im not sure when this would happen, any ideas on which classifier would do this and under what circumstances? 

```python
        probas = clf.predict_proba(X_t)

        if probas.shape[1] != self.n_classes_:
            new_probas = np.zeros((probas.shape[0], self.n_classes_))
            for i, cls in enumerate(clf.classes_):
                cls_idx = self._class_dictionary[cls]
                new_probas[:, cls_idx] = probas[:, i]
            probas = new_probas
```


aeon-actions-bot[bot]: 
## Thank you for contributing to `aeon`

I did not find any labels to add based on the title. Please add the [ENH], [MNT], [BUG], [DOC], [REF], [DEP] and/or [GOV] tags to your pull requests titles. For now you can add the labels manually.
I have added the following labels to this PR based on the changes made: **[ $\\color{#BCAE15}{\\textsf{classification}}$ ]**. Feel free to change these if they do not properly represent the PR.

The [Checks](https://github.com/aeon-toolkit/aeon/pull/2187/checks) tab will show the status of our automated tests. You can click on individual test runs in the tab or "Details" in the panel below to see more information if there is a failure.

If our `pre-commit` code quality check fails, any trivial fixes will automatically be pushed to your PR unless it is a draft.

Don't hesitate to ask questions on the `aeon` [Slack](https://join.slack.com/t/aeon-toolkit/shared_invite/zt-22vwvut29-HDpCu~7VBUozyfL_8j3dLA) channel if you have any.

### PR CI actions

These checkboxes will add labels to enable/disable CI functionality for this PR. This may not take effect immediately, and a new commit may be required to run the new configuration.

- [ ] Run `pre-commit` checks for all files
- [ ] Run `mypy` typecheck tests
- [ ] Run all `pytest` tests and configurations
- [ ] Run all notebook example tests
- [ ] Run numba-disabled `codecov` tests
- [ ] Stop automatic `pre-commit` fixes (always disabled for drafts)
- [ ] Disable numba cache loading
- [ ] Push an empty commit to re-run CI checks
    
baraline: I think the only thing that could look like this is the binary case where estimators produce only probas for class 1 and we want it for class 0 and 1.

Otherwise the estimators is just ignoring/adding classes which seems wrong? Unless we are in some kind of unsupervised problem such as clustering where we create clusters on the go with updates? 
MatthewMiddlehurst: Answered this in a DM, but I believe it was a workaround for bagging with very small datasets (i.e. so small you are losing classes when you select your data for the tree).
MatthewMiddlehurst: Does this PR still need more changes?

Patch:
diff --git a/README.md b/README.md
index 9a81698b..82674523 100644
--- a/README.md
+++ b/README.md
@@ -55,14 +55,13 @@ can be found [here](https://www.aeon-toolkit.org/en/stable/developer_guide/dev_i
 
 The best place to get started for all `aeon` packages is our [getting started guide](https://www.aeon-toolkit.org/en/stable/getting_started.html).
 
-Below we provide a quick example of how to use `aeon` for forecasting,
-classification and clustering.
+Below we provide a quick example of how to use `aeon` for classification and clustering.
 
 ### Classification
 
 *It's worth mentioning that the classifier used in the example can easily be
 swapped out for a regressor, and the labels for numeric targets. This flexibility
-allowing for seamless adaptation to different tasks and datasets while preserving
+allows for seamless adaptation to different tasks and datasets while preserving
 API consistency.*
 
 ```python
diff --git a/aeon/base/_base_collection.py b/aeon/base/_base_collection.py
index da64b2f7..d3d298e8 100644
--- a/aeon/base/_base_collection.py
+++ b/aeon/base/_base_collection.py
@@ -55,7 +55,7 @@ class BaseCollectionEstimator(BaseAeonEstimator):
         Parameters
         ----------
         X : collection
-            See aeon.utils.registry.COLLECTIONS_DATA_TYPES for details
+            See aeon.utils.COLLECTIONS_DATA_TYPES for details
             on aeon supported data structures.
         store_metadata : bool, default=True
             Whether to store metadata about X in self.metadata_.
@@ -107,7 +107,7 @@ class BaseCollectionEstimator(BaseAeonEstimator):
         Parameters
         ----------
         X : data structure
-           See aeon.utils.registry.COLLECTIONS_DATA_TYPES for details
+           See aeon.utils.COLLECTIONS_DATA_TYPES for details
            on aeon supported data structures.
 
         Returns
@@ -174,7 +174,7 @@ class BaseCollectionEstimator(BaseAeonEstimator):
         Parameters
         ----------
         X : data structure
-            Must be of type aeon.utils.registry.COLLECTIONS_DATA_TYPES.
+            Must be of type aeon.utils.COLLECTIONS_DATA_TYPES.
 
         Returns
         -------
diff --git a/aeon/clustering/deep_learning/_ae_fcn.py b/aeon/clustering/deep_learning/_ae_fcn.py
index 00758178..70b55bb4 100644
--- a/aeon/clustering/deep_learning/_ae_fcn.py
+++ b/aeon/clustering/deep_learning/_ae_fcn.py
@@ -5,6 +5,7 @@ __all__ = ["AEFCNClusterer"]
 
 import gc
 import os
+import sys
 import time
 from copy import deepcopy
 
@@ -66,6 +67,10 @@ class AEFCNClusterer(BaseDeepClusterer):
     verbose : boolean, default = False
         Whether to output extra information.
     loss : string, default="mean_squared_error"
+        Fit parameter for the keras model. "multi_rec" for multiple mse loss.
+        Multiple mse loss computes mean squared error between all embeddings
+        of encoder layers with the corresponding reconstructions of the
+        decoder layers.
         Fit parameter for the keras model.
     metrics : keras metrics, default = ["mean_squared_error"]
         will be set to mean_squared_error as default if None
@@ -296,18 +301,29 @@ class AEFCNClusterer(BaseDeepClusterer):
                 file_name=self.file_name_,
             )
 
-        self.history = self.training_model_.fit(
-            X,
-            X,
-            batch_size=mini_batch_size,
-            epochs=self.n_epochs,
-            verbose=self.verbose,
-            callbacks=self.callbacks_,
-        )
+        if not self.loss == "multi_rec":
+            self.history = self.training_model_.fit(
+                X,
+                X,
+                batch_size=mini_batch_size,
+                epochs=self.n_epochs,
+                verbose=self.verbose,
+                callbacks=self.callbacks_,
+            )
+
+        elif self.loss == "multi_rec":
+            self.history = self._fit_multi_rec_model(
+                autoencoder=self.training_model_,
+                inputs=X,
+                outputs=X,
+                batch_size=mini_batch_size,
+                epochs=self.n_epochs,
+            )
 
         try:
             self.model_ = tf.keras.models.load_model(
-                self.file_path + self.file_name_ + ".keras", compile=False
+                self.file_path + self.file_name_ + ".keras",
+                compile=False,
             )
             if not self.save_best_model:
                 os.remove(self.file_path + self.file_name_ + ".keras")
@@ -326,6 +342,132 @@ class AEFCNClusterer(BaseDeepClusterer):
         latent_space = self.model_.layers[1].predict(X)
         return self._estimator.score(latent_space)
 
+    def _fit_multi_rec_model(
+        self,
+        autoencoder,
+        inputs,
+        outputs,
+        batch_size,
+        epochs,
+    ):
+        import tensorflow as tf
+
+        train_dataset = tf.data.Dataset.from_tensor_slices((inputs, outputs))
+        train_dataset = train_dataset.shuffle(buffer_size=1024).batch(batch_size)
+
+        if isinstance(self.optimizer_, str):
+            self.optimizer_ = tf.keras.optimizers.get(self.optimizer_)
+
+        history = {"loss": []}
+
+        def layerwise_mse_loss(autoencoder, inputs, outputs):
+            def loss(y_true, y_pred):
+                # Calculate MSE for each layer in the encoder and decoder
+                mse = 0
+
+                _encoder_intermediate_outputs = (
+                    []
+                )  # Store embeddings of each layer in the Encoder
+                _decoder_intermediate_outputs = (
+                    []
+                )  # Store embeddings of each layer in the Decoder
+
+                encoder = autoencoder.layers[1]  # Returns Functional API Models.
+                decoder = autoencoder.layers[2]  # Returns Functional API Models.
+
+                # Run the models since the below given loop misses the latent space
+                # layer which doesn't contribute to the loss.
+                logits = encoder(inputs)
+                __dec_outputs = decoder(logits)
+
+                # Encoder
+                for i in range(self.n_layers):
+                    _activation_layer = encoder.get_layer(f"__act_encoder_block{i}")
+                    _model = tf.keras.models.Model(
+                        inputs=encoder.input, outputs=_activation_layer.output
+                    )
+                    __output = _model(inputs, training=True)
+                    _encoder_intermediate_outputs.append(__output)
+
+                # Decoder
+                for i in range(self.n_layers):
+                    _activation_layer = decoder.get_layer(f"__act_decoder_block{i}")
+                    _model = tf.keras.models.Model(
+                        inputs=decoder.input, outputs=_activation_layer.output
+                    )
+                    __output = _model(logits, training=True)
+                    _decoder_intermediate_outputs.append(__output)
+
+                if not (
+                    len(_encoder_intermediate_outputs)
+                    == len(_decoder_intermediate_outputs)
+                ):
+                    raise ValueError("The Auto-Encoder must be symmetric in nature.")
+
+                # # Append normal mean_squared_error
+
+                for enc_output, dec_output in zip(
+                    _encoder_intermediate_outputs, _decoder_intermediate_outputs
+                ):
+                    mse += tf.keras.backend.mean(
+                        tf.keras.backend.square(enc_output - dec_output)
+                    )
+
+                inputs_casted = tf.cast(inputs, dtype=tf.float64)
+                __dec_outputs_casted = tf.cast(__dec_outputs, dtype=tf.float64)
+                return tf.cast(mse, dtype=tf.float64) + tf.cast(
+                    tf.reduce_mean(tf.square(inputs_casted - __dec_outputs_casted)),
+                    dtype=tf.float64,
+                )
+
+            return loss
+
+        # Initialize callbacks
+        for callback in self.callbacks_:
+            callback.set_model(autoencoder)
+            callback.on_train_begin()
+
+        for epoch in range(epochs):
+            epoch_loss = 0
+            num_batches = 0
+            for step, (x_batch_train, y_batch_train) in enumerate(train_dataset):
+                with tf.GradientTape() as tape:
+                    # Calculate the actual loss by calling the loss function
+                    loss_func = layerwise_mse_loss(
+                        autoencoder=autoencoder,
+                        inputs=x_batch_train,
+                        outputs=y_batch_train,
+                    )
+                    loss_value = loss_func(y_batch_train, autoencoder(x_batch_train))
+
+                grads = tape.gradient(loss_value, autoencoder.trainable_weights)
+                self.optimizer_.apply_gradients(
+                    zip(grads, autoencoder.trainable_weights)
+                )
+
+                epoch_loss += float(loss_value)
+                num_batches += 1
+
+                # Update callbacks on batch end
+                for callback in self.callbacks_:
+                    callback.on_batch_end(step, {"loss": float(loss_value)})
+
+            epoch_loss /= num_batches
+            history["loss"].append(epoch_loss)
+
+            sys.stdout.write(
+                "Training loss at epoch %d: %.4f\n" % (epoch, float(epoch_loss))
+            )
+
+            for callback in self.callbacks_:
+                callback.on_epoch_end(epoch, {"loss": float(epoch_loss)})
+
+        # Finalize callbacks
+        for callback in self.callbacks_:
+            callback.on_train_end()
+
+        return history
+
     @classmethod
     def _get_test_params(cls, parameter_set="default"):
         """Return testing parameter settings for the estimator.
diff --git a/aeon/networks/__init__.py b/aeon/networks/__init__.py
index d5cdb360..5d8a87f2 100644
--- a/aeon/networks/__init__.py
+++ b/aeon/networks/__init__.py
@@ -13,12 +13,12 @@ __all__ = [
     "AEFCNNetwork",
     "AEResNetNetwork",
     "LITENetwork",
+    "DCNNNetwork",
     "AEDCNNNetwork",
     "AEAttentionBiGRUNetwork",
     "AEDRNNNetwork",
     "AEBiGRUNetwork",
 ]
-
 from aeon.networks._ae_abgru import AEAttentionBiGRUNetwork
 from aeon.networks._ae_bgru import AEBiGRUNetwork
 from aeon.networks._ae_dcnn import AEDCNNNetwork
@@ -26,6 +26,7 @@ from aeon.networks._ae_drnn import AEDRNNNetwork
 from aeon.networks._ae_fcn import AEFCNNetwork
 from aeon.networks._ae_resnet import AEResNetNetwork
 from aeon.networks._cnn import TimeCNNNetwork
+from aeon.networks._dcnn import DCNNNetwork
 from aeon.networks._encoder import EncoderNetwork
 from aeon.networks._fcn import FCNNetwork
 from aeon.networks._inception import InceptionNetwork
diff --git a/aeon/networks/_ae_fcn.py b/aeon/networks/_ae_fcn.py
index 1c1c2bef..e64569e0 100644
--- a/aeon/networks/_ae_fcn.py
+++ b/aeon/networks/_ae_fcn.py
@@ -199,7 +199,9 @@ class AEFCNNetwork(BaseDeepLearningNetwork):
             )(x)
 
             conv = tf.keras.layers.BatchNormalization()(conv)
-            conv = tf.keras.layers.Activation(activation=self._activation[i])(conv)
+            conv = tf.keras.layers.Activation(
+                activation=self._activation[i], name=f"__act_encoder_block{i}"
+            )(conv)
 
             x = conv
 
@@ -251,7 +253,9 @@ class AEFCNNetwork(BaseDeepLearningNetwork):
             )(x)
 
             conv = tf.keras.layers.BatchNormalization()(conv)
-            conv = tf.keras.layers.Activation(activation=self._activation[i])(conv)
+            conv = tf.keras.layers.Activation(
+                activation=self._activation[i], name=f"__act_decoder_block{i}"
+            )(conv)
 
             x = conv
 
diff --git a/aeon/networks/_dcnn.py b/aeon/networks/_dcnn.py
new file mode 100644
index 00000000..243340c3
--- /dev/null
+++ b/aeon/networks/_dcnn.py
@@ -0,0 +1,167 @@
+"""Dilated Convolutional Nerual Networks (DCNN) Model."""
+
+__maintainer__ = []
+
+from aeon.networks.base import BaseDeepLearningNetwork
+
+
+class DCNNNetwork(BaseDeepLearningNetwork):
+    """Establish the network structure for a DCNN-Model.
+
+    Dilated Convolutional Neural Network based Model
+    for low-rank embeddings.
+
+    Parameters
+    ----------
+    latent_space_dim: int, default=128
+        Dimension of the models's latent space.
+    n_layers: int, default=4
+        Number of convolution layers.
+    kernel_size: Union[int, List[int]], default=3
+        Size of the 1D Convolutional Kernel. Defaults
+        to a list of three's for `n_layers` elements.
+    activation: Union[str, List[str]], default="relu"
+        The activation function used by convolution layers.
+        Defaults to a list of "relu" for `n_layers` elements.
+    n_filters: Union[int, List[int]], default=None
+        Number of filters used in convolution layers. Defaults
+        to a list of multiple's of 32 for `n_layers` elements.
+    dilation_rate: Union[int, List[int]], default=None
+        The dilation rate for convolution. Defaults to a list of
+        powers of 2 for `n_layers` elements.
+    padding: Union[str, List[str]], default="causal"
+        Padding to be used in each DCNN Layer. Defaults to a list
+        of causal paddings for `n_layers` elements.
+
+    References
+    ----------
+    .. [1] Franceschi, J. Y., Dieuleveut, A., & Jaggi, M. (2019).
+    Unsupervised scalable representation learning for multivariate
+    time series. Advances in neural information processing systems, 32.
+    """
+
+    _config = {
+        "python_dependencies": ["tensorflow"],
+        "python_version": "<3.12",
+        "structure": "encoder",
+    }
+
+    def __init__(
+        self,
+        latent_space_dim=128,
+        n_layers=4,
+        kernel_size=3,
+        activation="relu",
+        n_filters=None,
+        dilation_rate=None,
+        padding="causal",
+    ):
+        super().__init__()
+
+        self.latent_space_dim = latent_space_dim
+        self.kernel_size = kernel_size
+        self.n_filters = n_filters
+        self.n_layers = n_layers
+        self.dilation_rate = dilation_rate
+        self.activation = activation
+        self.padding = padding
+
+    def build_network(self, input_shape):
+        """Construct a network and return its input and output layers.
+
+        Parameters
+        ----------
+        input_shape : tuple of shape = (n_timepoints (m), n_channels (d))
+            The shape of the data fed into the input layer.
+
+        Returns
+        -------
+        model : a keras Model.
+        """
+        import tensorflow as tf
+
+        if self.n_filters is None:
+            self._n_filters = [32 * i for i in range(1, self.n_layers + 1)]
+        elif isinstance(self.n_filters, int):
+            self._n_filters = [self.n_filters for _ in range(self.n_layers)]
+        elif isinstance(self.n_filters, list):
+            self._n_filters = self.n_filters
+            assert len(self.n_filters) == self.n_layers
+
+        if self.dilation_rate is None:
+            self._dilation_rate = [
+                2**layer_num for layer_num in range(1, self.n_layers + 1)
+            ]
+        elif isinstance(self.dilation_rate, int):
+            self._dilation_rate = [self.dilation_rate for _ in range(self.n_layers)]
+        else:
+            self._dilation_rate = self.dilation_rate
+            assert isinstance(self.dilation_rate, list)
+            assert len(self.dilation_rate) == self.n_layers
+
+        if self.kernel_size is None:
+            self._kernel_size = [3 for _ in range(self.n_layers)]
+        elif isinstance(self.kernel_size, int):
+            self._kernel_size = [self.kernel_size for _ in range(self.n_layers)]
+        elif isinstance(self.kernel_size, list):
+            self._kernel_size = self.kernel_size
+            assert len(self.kernel_size) == self.n_layers
+
+        if self.activation is None:
+            self._activation = ["relu" for _ in range(self.n_layers)]
+        elif isinstance(self.activation, str):
+            self._activation = [self.activation for _ in range(self.n_layers)]
+        elif isinstance(self.activation, list):
+            self._activation = self.activation
+            assert len(self._activation) == self.n_layers
+
+        if self.padding is None:
+            self._padding = ["causal" for _ in range(self.n_layers)]
+        elif isinstance(self.padding, str):
+            self._padding = [self.padding for _ in range(self.n_layers)]
+        elif isinstance(self.padding, list):
+            self._padding = self.padding
+            assert len(self._padding) == self.n_layers
+
+        input_layer = tf.keras.layers.Input(input_shape)
+
+        x = input_layer
+        for i in range(0, self.n_layers):
+            x = self._dcnn_layer(
+                x,
+                self._n_filters[i],
+                self._dilation_rate[i],
+                _activation=self._activation[i],
+                _kernel_size=self._kernel_size[i],
+                _padding=self._padding[i],
+            )
+
+        x = tf.keras.layers.GlobalMaxPool1D()(x)
+        output_layer = tf.keras.layers.Dense(self.latent_space_dim)(x)
+
+        return input_layer, output_layer
+
+    def _dcnn_layer(
+        self, _inputs, _n_filters, _dilation_rate, _activation, _kernel_size, _padding
+    ):
+        import tensorflow as tf
+
+        _add = tf.keras.layers.Conv1D(_n_filters, kernel_size=1)(_inputs)
+        x = tf.keras.layers.Conv1D(
+            _n_filters,
+            kernel_size=_kernel_size,
+            dilation_rate=_dilation_rate,
+            padding=_padding,
+            kernel_regularizer="l2",
+        )(_inputs)
+        x = tf.keras.layers.Conv1D(
+            _n_filters,
+            kernel_size=_kernel_size,
+            dilation_rate=_dilation_rate,
+            padding="causal",
+            kernel_regularizer="l2",
+            activation=_activation,
+        )(x)
+        output = tf.keras.layers.Add()([x, _add])
+        output = tf.keras.layers.Activation(_activation)(output)
+        return output
diff --git a/aeon/transformations/collection/feature_based/_catch22.py b/aeon/transformations/collection/feature_based/_catch22.py
index 4db6ff16..9727d0be 100644
--- a/aeon/transformations/collection/feature_based/_catch22.py
+++ b/aeon/transformations/collection/feature_based/_catch22.py
@@ -7,6 +7,7 @@ __maintainer__ = []
 __all__ = ["Catch22"]
 
 import math
+import warnings
 
 import numpy as np
 from joblib import Parallel, delayed
@@ -16,6 +17,7 @@ from aeon.transformations.collection.base import BaseCollectionTransformer
 from aeon.utils.numba.general import z_normalise_series, z_normalise_series_with_mean
 from aeon.utils.numba.stats import mean, numba_max, numba_min
 from aeon.utils.validation import check_n_jobs
+from aeon.utils.validation._dependencies import _check_soft_dependencies
 
 feature_names = [
     "DN_HistogramMode_5",
@@ -219,65 +221,75 @@ class Catch22(BaseCollectionTransformer):
 
         threads_to_use = check_n_jobs(self.n_jobs)
 
+        features = [
+            Catch22._DN_HistogramMode_5,
+            Catch22._DN_HistogramMode_10,
+            Catch22._CO_f1ecac,
+            Catch22._CO_FirstMin_ac,
+            Catch22._CO_HistogramAMI_even_2_5,
+            Catch22._CO_trev_1_num,
+            Catch22._MD_hrv_classic_pnn40,
+            Catch22._SB_BinaryStats_mean_longstretch1,
+            Catch22._SB_TransitionMatrix_3ac_sumdiagcov,
+            Catch22._PD_PeriodicityWang_th0_01,
+            Catch22._CO_Embed2_Dist_tau_d_expfit_meandiff,
+            Catch22._IN_AutoMutualInfoStats_40_gaussian_fmmi,
+            Catch22._FC_LocalSimple_mean1_tauresrat,
+            Catch22._DN_OutlierInclude_p_001_mdrmd,
+            Catch22._DN_OutlierInclude_n_001_mdrmd,
+            Catch22._SP_Summaries_welch_rect_area_5_1,
+            Catch22._SB_BinaryStats_diff_longstretch0,
+            Catch22._SB_MotifThree_quantile_hh,
+            Catch22._SC_FluctAnal_2_rsrangefit_50_1_logi_prop_r1,
+            Catch22._SC_FluctAnal_2_dfa_50_1_2_logi_prop_r1,
+            Catch22._SP_Summaries_welch_rect_centroid,
+            Catch22._FC_LocalSimple_mean3_stderr,
+        ]
+
+        use_pycatch22_transform = False
         if self.use_pycatch22:
-            import pycatch22
-
-            features = [
-                pycatch22.DN_HistogramMode_5,
-                pycatch22.DN_HistogramMode_10,
-                pycatch22.CO_f1ecac,
-                pycatch22.CO_FirstMin_ac,
-                pycatch22.CO_HistogramAMI_even_2_5,
-                pycatch22.CO_trev_1_num,
-                pycatch22.MD_hrv_classic_pnn40,
-                pycatch22.SB_BinaryStats_mean_longstretch1,
-                pycatch22.SB_TransitionMatrix_3ac_sumdiagcov,
-                pycatch22.PD_PeriodicityWang_th0_01,
-                pycatch22.CO_Embed2_Dist_tau_d_expfit_meandiff,
-                pycatch22.IN_AutoMutualInfoStats_40_gaussian_fmmi,
-                pycatch22.FC_LocalSimple_mean1_tauresrat,
-                pycatch22.DN_OutlierInclude_p_001_mdrmd,
-                pycatch22.DN_OutlierInclude_n_001_mdrmd,
-                pycatch22.SP_Summaries_welch_rect_area_5_1,
-                pycatch22.SB_BinaryStats_diff_longstretch0,
-                pycatch22.SB_MotifThree_quantile_hh,
-                pycatch22.SC_FluctAnal_2_rsrangefit_50_1_logi_prop_r1,
-                pycatch22.SC_FluctAnal_2_dfa_50_1_2_logi_prop_r1,
-                pycatch22.SP_Summaries_welch_rect_centroid,
-                pycatch22.FC_LocalSimple_mean3_stderr,
-            ]
-        else:
-            features = [
-                Catch22._DN_HistogramMode_5,
-                Catch22._DN_HistogramMode_10,
-                Catch22._CO_f1ecac,
-                Catch22._CO_FirstMin_ac,
-                Catch22._CO_HistogramAMI_even_2_5,
-                Catch22._CO_trev_1_num,
-                Catch22._MD_hrv_classic_pnn40,
-                Catch22._SB_BinaryStats_mean_longstretch1,
-                Catch22._SB_TransitionMatrix_3ac_sumdiagcov,
-                Catch22._PD_PeriodicityWang_th0_01,
-                Catch22._CO_Embed2_Dist_tau_d_expfit_meandiff,
-                Catch22._IN_AutoMutualInfoStats_40_gaussian_fmmi,
-                Catch22._FC_LocalSimple_mean1_tauresrat,
-                Catch22._DN_OutlierInclude_p_001_mdrmd,
-                Catch22._DN_OutlierInclude_n_001_mdrmd,
-                Catch22._SP_Summaries_welch_rect_area_5_1,
-                Catch22._SB_BinaryStats_diff_longstretch0,
-                Catch22._SB_MotifThree_quantile_hh,
-                Catch22._SC_FluctAnal_2_rsrangefit_50_1_logi_prop_r1,
-                Catch22._SC_FluctAnal_2_dfa_50_1_2_logi_prop_r1,
-                Catch22._SP_Summaries_welch_rect_centroid,
-                Catch22._FC_LocalSimple_mean3_stderr,
-            ]
+            if _check_soft_dependencies("pycatch22", severity="none"):
+                import pycatch22
+
+                features = [
+                    pycatch22.DN_HistogramMode_5,
+                    pycatch22.DN_HistogramMode_10,
+                    pycatch22.CO_f1ecac,
+                    pycatch22.CO_FirstMin_ac,
+                    pycatch22.CO_HistogramAMI_even_2_5,
+                    pycatch22.CO_trev_1_num,
+                    pycatch22.MD_hrv_classic_pnn40,
+                    pycatch22.SB_BinaryStats_mean_longstretch1,
+                    pycatch22.SB_TransitionMatrix_3ac_sumdiagcov,
+                    pycatch22.PD_PeriodicityWang_th0_01,
+                    pycatch22.CO_Embed2_Dist_tau_d_expfit_meandiff,
+                    pycatch22.IN_AutoMutualInfoStats_40_gaussian_fmmi,
+                    pycatch22.FC_LocalSimple_mean1_tauresrat,
+                    pycatch22.DN_OutlierInclude_p_001_mdrmd,
+                    pycatch22.DN_OutlierInclude_n_001_mdrmd,
+                    pycatch22.SP_Summaries_welch_rect_area_5_1,
+                    pycatch22.SB_BinaryStats_diff_longstretch0,
+                    pycatch22.SB_MotifThree_quantile_hh,
+                    pycatch22.SC_FluctAnal_2_rsrangefit_50_1_logi_prop_r1,
+                    pycatch22.SC_FluctAnal_2_dfa_50_1_2_logi_prop_r1,
+                    pycatch22.SP_Summaries_welch_rect_centroid,
+                    pycatch22.FC_LocalSimple_mean3_stderr,
+                ]
+
+                use_pycatch22_transform = True
+            else:
+                warnings.warn(
+                    "pycatch22 not installed, but 'self.use_pycatch22' is set to True."
+                    "Please install pycatch22. Aeon catch22 will be used.",
+                    stacklevel=2,
+                )
 
         c22_list = Parallel(
             n_jobs=threads_to_use, backend=self.parallel_backend, prefer="threads"
         )(
             delayed(
                 self._transform_case_pycatch22
-                if self.use_pycatch22
+                if use_pycatch22_transform
                 else self._transform_case
             )(
                 X[i],
diff --git a/aeon/utils/validation/collection.py b/aeon/utils/validation/collection.py
index 5a5eb62f..e52ab73c 100644
--- a/aeon/utils/validation/collection.py
+++ b/aeon/utils/validation/collection.py
@@ -82,7 +82,7 @@ def get_n_cases(X):
     Parameters
     ----------
     X : collection
-        See aeon.utils.registry.COLLECTIONS_DATA_TYPES for details.
+        See aeon.utils.COLLECTIONS_DATA_TYPES for details.
 
     Returns
     -------
@@ -103,7 +103,7 @@ def get_n_timepoints(X):
     Parameters
     ----------
     X : collection
-        See aeon.utils.registry.COLLECTIONS_DATA_TYPES for details.
+        See aeon.utils.COLLECTIONS_DATA_TYPES for details.
 
     Returns
     -------
@@ -129,7 +129,7 @@ def get_n_channels(X):
     Parameters
     ----------
     X : collection
-        See aeon.utils.registry.COLLECTIONS_DATA_TYPES for details.
+        See aeon.utils.COLLECTIONS_DATA_TYPES for details.
 
     Returns
     -------
@@ -171,7 +171,7 @@ def get_type(X):
     Parameters
     ----------
     X : collection
-        See aeon.utils.registry.COLLECTIONS_DATA_TYPES for details.
+        See aeon.utils.COLLECTIONS_DATA_TYPES for details.
 
     Returns
     -------
@@ -241,7 +241,7 @@ def is_equal_length(X):
     Parameters
     ----------
     X : collection
-        See aeon.utils.registry.COLLECTIONS_DATA_TYPES for details.
+        See aeon.utils.COLLECTIONS_DATA_TYPES for details.
 
     Returns
     -------
diff --git a/examples/transformations/catch22.ipynb b/examples/transformations/catch22.ipynb
index 6bdd09a5..a551f67e 100644
--- a/examples/transformations/catch22.ipynb
+++ b/examples/transformations/catch22.ipynb
@@ -6,44 +6,43 @@
    "source": [
     "# The Canonical Time-series Characteristics (catch22) transform\n",
     "\n",
-    "catch22\\[1\\] is a collection of 22 time series features extracted from the 7000+ present in the _hctsa_ \\[2\\]\\[3\\] toolbox.\n",
+    "Catch22\\[1\\] is a collection of 22 time series features extracted from the 7000+ present in the _hctsa_ \\[2\\]\\[3\\] toolbox.\n",
     "A hierarchical clustering was performed on the correlation matrix of features that performed better than random chance to remove redundancy.\n",
     "These clusters were sorted by balanced accuracy using a decision tree classifier and a single feature was selected from the 22 clusters formed, taking into account balanced accuracy results, computational efficiency and interpretability.\n",
+    "More about the individual features of catch22 can be learned in the [Gitbook](https://time-series-features.gitbook.io/catch22/information-about-catch22/feature-descriptions) of the original creators.\n",
     "\n",
-    "In this notebook, we will demonstrate how to use the catch22 transformer on the ItalyPowerDemand univariate and BasicMotions multivariate datasets. We also show catch22 used for classification with a random forest classifier.\n",
-    "\n",
-    "#### References:\n",
-    "\n",
-    "\\[1\\] Lubba, C. H., Sethi, S. S., Knaute, P., Schultz, S. R., Fulcher, B. D., & Jones, N. S. (2019). catch22: CAnonical Time-series CHaracteristics. Data Mining and Knowledge Discovery, 33(6), 1821-1852.\n",
-    "\n",
-    "\\[2\\] Fulcher, B. D., & Jones, N. S. (2017). hctsa: A computational framework for automated time-series phenotyping using massive feature extraction. Cell systems, 5(5), 527-531.\n",
-    "\n",
-    "\\[3\\] Fulcher, B. D., Little, M. A., & Jones, N. S. (2013). Highly comparative time-series analysis: the empirical structure of time series and their methods. Journal of the Royal Society Interface, 10(83), 20130048."
+    "In this notebook, we will demonstrate how to use aeon's catch22 transformer on the ItalyPowerDemand univariate and BasicMotions multivariate datasets. We will go through the parameters of catch22 and how changing the default values may change results. Catch22 has also been used inside of [classification](../classification/feature_based.ipynb)."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 1. Transformation"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Catch22 is a feature based transformer that extracts 22 features from a time series. The input data can be both univariate and multivariate, without the need to reshape the data. It is most commonly used for interpretability of each time series data. Additionally, as the data of a time series will be reduced to 22 data values, it will increase computational efficiency of machine learning tasks such as clustering, classification, etc."
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## 1. Imports"
+    "### 1.1 Import Data and Catch22"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 18,
-   "metadata": {
-    "execution": {
-     "iopub.execute_input": "2020-12-19T14:30:07.306937Z",
-     "iopub.status.busy": "2020-12-19T14:30:07.306390Z",
-     "iopub.status.idle": "2020-12-19T14:30:08.036353Z",
-     "shell.execute_reply": "2020-12-19T14:30:08.036857Z"
-    }
-   },
+   "execution_count": 7,
+   "metadata": {},
    "outputs": [],
    "source": [
-    "from sklearn import metrics\n",
+    "import numpy as np\n",
     "\n",
-    "from aeon.classification.feature_based import Catch22Classifier\n",
     "from aeon.datasets import load_basic_motions, load_italy_power_demand\n",
     "from aeon.transformations.collection.feature_based import Catch22"
    ]
@@ -52,69 +51,65 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## 2. Load data"
+    "### 1.2 Load Data"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 19,
-   "metadata": {
-    "execution": {
-     "iopub.execute_input": "2020-12-19T14:30:08.041533Z",
-     "iopub.status.busy": "2020-12-19T14:30:08.041060Z",
-     "iopub.status.idle": "2020-12-19T14:30:08.210768Z",
-     "shell.execute_reply": "2020-12-19T14:30:08.211258Z"
-    }
-   },
+   "execution_count": 3,
+   "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "(67, 1, 24) (67,) (50, 1, 24) (50,)\n",
-      "(40, 6, 100) (40,) (40, 6, 100) (40,)\n"
+      "Italy Power Demand (Univariate):  (67, 1, 24) (67,) (1029, 1, 24) (1029,)\n",
+      "Load Basic Motions (Multivarite):  (40, 6, 100) (40,) (40, 6, 100) (40,)\n"
      ]
     }
    ],
    "source": [
     "IPD_X_train, IPD_y_train = load_italy_power_demand(split=\"train\")\n",
     "IPD_X_test, IPD_y_test = load_italy_power_demand(split=\"test\")\n",
-    "IPD_X_test = IPD_X_test[:50]\n",
-    "IPD_y_test = IPD_y_test[:50]\n",
     "\n",
-    "print(IPD_X_train.shape, IPD_y_train.shape, IPD_X_test.shape, IPD_y_test.shape)\n",
+    "print(\n",
+    "    \"Italy Power Demand (Univariate): \",\n",
+    "    IPD_X_train.shape,\n",
+    "    IPD_y_train.shape,\n",
+    "    IPD_X_test.shape,\n",
+    "    IPD_y_test.shape,\n",
+    ")\n",
     "\n",
     "BM_X_train, BM_y_train = load_basic_motions(split=\"train\")\n",
-    "BM_X_test, BM_y_test = load_basic_motions(\n",
-    "    split=\"test\",\n",
-    ")\n",
+    "BM_X_test, BM_y_test = load_basic_motions(split=\"test\")\n",
     "\n",
-    "print(BM_X_train.shape, BM_y_train.shape, BM_X_test.shape, BM_y_test.shape)"
+    "print(\n",
+    "    \"Load Basic Motions (Multivarite): \",\n",
+    "    BM_X_train.shape,\n",
+    "    BM_y_train.shape,\n",
+    "    BM_X_test.shape,\n",
+    "    BM_y_test.shape,\n",
+    ")"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## 3. catch22 transform\n",
-    "\n",
-    "### Univariate\n",
-    "\n",
-    "The catch22 features are provided in the form of a transformer, `Catch22`.\n",
-    "From this the transformed data can be used for a variety of time series analysis tasks."
+    "### 1.3 Transform the Data"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### Univariate"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 20,
-   "metadata": {
-    "execution": {
-     "iopub.execute_input": "2020-12-19T14:30:08.215545Z",
-     "iopub.status.busy": "2020-12-19T14:30:08.215049Z",
-     "iopub.status.idle": "2020-12-19T14:30:08.222937Z",
-     "shell.execute_reply": "2020-12-19T14:30:08.223422Z"
-    }
-   },
+   "execution_count": 4,
+   "metadata": {},
    "outputs": [
     {
      "name": "stdout",
@@ -135,134 +130,248 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Multivariate\n",
-    "\n",
-    "Transformation of multivariate data is supported by `Catch22`.\n",
-    "The default procedure will concatenate each column prior to transformation."
+    "#### Multivariate"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Do note that the result of the shape won't be (X , 22). This is because it's a multivariate dataset, and therefore the feature vector will be of size 22 times the number of channels. "
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 21,
-   "metadata": {
-    "execution": {
-     "iopub.execute_input": "2020-12-19T14:30:08.264541Z",
-     "iopub.status.busy": "2020-12-19T14:30:08.264050Z",
-     "iopub.status.idle": "2020-12-19T14:30:08.266022Z",
-     "shell.execute_reply": "2020-12-19T14:30:08.266517Z"
-    }
-   },
+   "execution_count": 5,
+   "metadata": {},
    "outputs": [
     {
-     "data": {
-      "text/html": [
-       "<style>#sk-container-id-4 {color: black;background-color: white;}#sk-container-id-4 pre{padding: 0;}#sk-container-id-4 div.sk-toggleable {background-color: white;}#sk-container-id-4 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-4 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-4 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-4 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-4 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-4 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-4 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-4 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-4 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-4 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-4 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-4 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-4 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-4 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-4 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-4 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-4 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-4 div.sk-item {position: relative;z-index: 1;}#sk-container-id-4 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-4 div.sk-item::before, #sk-container-id-4 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-4 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-4 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-4 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-4 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-4 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-4 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-4 div.sk-label-container {text-align: center;}#sk-container-id-4 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-4 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-4\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>Catch22()</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-4\" type=\"checkbox\" checked><label for=\"sk-estimator-id-4\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">Catch22</label><div class=\"sk-toggleable__content\"><pre>Catch22()</pre></div></div></div></div></div>"
-      ],
-      "text/plain": [
-       "Catch22()"
-      ]
-     },
-     "execution_count": 21,
-     "metadata": {},
-     "output_type": "execute_result"
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "(40, 132)\n"
+     ]
     }
    ],
    "source": [
     "c22_mv = Catch22()\n",
-    "c22_mv.fit(BM_X_train, BM_y_train)"
+    "data = c22_mv.fit_transform(BM_X_train, BM_y_train)\n",
+    "transformed_data_mv = c22_uv.transform(BM_X_train)\n",
+    "print(transformed_data_mv.shape)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 2. Parameters"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Aeon's catch22 includes a lot options for users need compared to the original catch22 implementation which we will talk about in section 2.4. Few of the parameters are shown below with examples, specifically the ones that change affect the output. More can be found in [catch22's documentation](../../docs/api_reference/transformations.rst)."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### 2.1 Features"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Catch22 takes 22 distinct features from a time series. Sometimes you may not need all the features extracted by catch22, instead you may only need some very specific features. By defining an array containing strings of features, only those specified features will be extracted. The order of these features do matter, as that will be the order of the output. Aeon's [catch22's documentation](../../docs/api_reference/transformations.rst) specifies a list of the 22 features for extraction."
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 22,
-   "metadata": {
-    "execution": {
-     "iopub.execute_input": "2020-12-19T14:30:08.271483Z",
-     "iopub.status.busy": "2020-12-19T14:30:08.270986Z",
-     "iopub.status.idle": "2020-12-19T14:30:08.413472Z",
-     "shell.execute_reply": "2020-12-19T14:30:08.413974Z"
-    }
-   },
+   "execution_count": 6,
+   "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "(40, 132)\n"
+      "(67, 3)\n"
+     ]
+    },
+    {
+     "ename": "ValueError",
+     "evalue": "Invalid feature selection.",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[1;31mValueError\u001b[0m                                Traceback (most recent call last)",
+      "Cell \u001b[1;32mIn[6], line 11\u001b[0m\n\u001b[0;32m      9\u001b[0m c22_short \u001b[38;5;241m=\u001b[39m Catch22(features\u001b[38;5;241m=\u001b[39mfeatures_short)\n\u001b[0;32m     10\u001b[0m c22_short\u001b[38;5;241m.\u001b[39mfit(IPD_X_train, IPD_y_train)\n\u001b[1;32m---> 11\u001b[0m transformed_data_short \u001b[38;5;241m=\u001b[39m \u001b[43mc22_short\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mtransform\u001b[49m\u001b[43m(\u001b[49m\u001b[43mIPD_X_train\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m     12\u001b[0m \u001b[38;5;28mprint\u001b[39m(transformed_data_short\u001b[38;5;241m.\u001b[39mshape)\n",
+      "File \u001b[1;32md:\\AeonProject\\aeon\\.venv\\Lib\\site-packages\\aeon\\transformations\\collection\\base.py:157\u001b[0m, in \u001b[0;36mBaseCollectionTransformer.transform\u001b[1;34m(self, X, y)\u001b[0m\n\u001b[0;32m    154\u001b[0m X_inner \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_preprocess_collection(X, store_metadata\u001b[38;5;241m=\u001b[39m\u001b[38;5;28;01mFalse\u001b[39;00m)\n\u001b[0;32m    155\u001b[0m y_inner \u001b[38;5;241m=\u001b[39m y\n\u001b[1;32m--> 157\u001b[0m Xt \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_transform\u001b[49m\u001b[43m(\u001b[49m\u001b[43mX\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mX_inner\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43my\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43my_inner\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m    159\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m Xt\n",
+      "File \u001b[1;32md:\\AeonProject\\aeon\\.venv\\Lib\\site-packages\\aeon\\transformations\\collection\\feature_based\\_catch22.py:182\u001b[0m, in \u001b[0;36mCatch22._transform\u001b[1;34m(self, X, y)\u001b[0m\n\u001b[0;32m    165\u001b[0m \u001b[38;5;250m\u001b[39m\u001b[38;5;124;03m\"\"\"Transform X into the catch22 features.\u001b[39;00m\n\u001b[0;32m    166\u001b[0m \n\u001b[0;32m    167\u001b[0m \u001b[38;5;124;03mParameters\u001b[39;00m\n\u001b[1;32m   (...)\u001b[0m\n\u001b[0;32m    178\u001b[0m \u001b[38;5;124;03m    The catch22 features for each dimension.\u001b[39;00m\n\u001b[0;32m    179\u001b[0m \u001b[38;5;124;03m\"\"\"\u001b[39;00m\n\u001b[0;32m    180\u001b[0m n_cases \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mlen\u001b[39m(X)\n\u001b[1;32m--> 182\u001b[0m f_idx \u001b[38;5;241m=\u001b[39m \u001b[43m_verify_features\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mfeatures\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mcatch24\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m    184\u001b[0m threads_to_use \u001b[38;5;241m=\u001b[39m check_n_jobs(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mn_jobs)\n\u001b[0;32m    186\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39muse_pycatch22:\n",
+      "File \u001b[1;32md:\\AeonProject\\aeon\\.venv\\Lib\\site-packages\\aeon\\transformations\\collection\\feature_based\\_catch22.py:1300\u001b[0m, in \u001b[0;36m_verify_features\u001b[1;34m(features, catch24)\u001b[0m\n\u001b[0;32m   1298\u001b[0m         f_idx\u001b[38;5;241m.\u001b[39mappend(\u001b[38;5;241m23\u001b[39m)\n\u001b[0;32m   1299\u001b[0m     \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m-> 1300\u001b[0m         \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mInvalid feature selection.\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[0;32m   1301\u001b[0m \u001b[38;5;28;01melif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(f, \u001b[38;5;28mint\u001b[39m):\n\u001b[0;32m   1302\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m f \u001b[38;5;241m>\u001b[39m\u001b[38;5;241m=\u001b[39m \u001b[38;5;241m0\u001b[39m \u001b[38;5;129;01mand\u001b[39;00m f \u001b[38;5;241m<\u001b[39m \u001b[38;5;241m22\u001b[39m:\n",
+      "\u001b[1;31mValueError\u001b[0m: Invalid feature selection."
      ]
     }
    ],
    "source": [
-    "transformed_data_mv = c22_mv.transform(BM_X_train)\n",
-    "print(transformed_data_mv.shape)"
+    "features_long = [\"DN_HistogramMode_5\", \"CO_f1ecac\", \"FC_LocalSimple_mean3_stderr\"]\n",
+    "features_short = [\"mode_5\", \"acf_timescale\", \"forecast_error\"]\n",
+    "\n",
+    "c22_long = Catch22(features=features_long)\n",
+    "c22_long.fit(IPD_X_train, IPD_y_train)\n",
+    "transformed_data_long = c22_long.transform(IPD_X_train)\n",
+    "print(transformed_data_long.shape)\n",
+    "\n",
+    "c22_short = Catch22(features=features_short)\n",
+    "c22_short.fit(IPD_X_train, IPD_y_train)\n",
+    "transformed_data_short = c22_short.transform(IPD_X_train)\n",
+    "print(transformed_data_short.shape)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### 2.2 Catch24"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## 4. catch22 Forest Classifier\n",
+    "Catch24 extracts 24 features from a time series. The 24 features consist of the 22 features from catch22 with the addition of the mean and standard deviation of the time series. More features does not strictly define better results, as it may increase run time and overfit the data in certain time series tasks. In certain tasks, catch24 may outperform catch22. For example in \\[4\\], catch24 significally outperformed catch22 in cross-domain anomaly detection.\n",
     "\n",
-    "For classification tasks the default classifier to use with the catch22 features is random forest classifier.\n",
-    "An implementation making use of the `RandomForestClassifier` from sklearn built on catch22 features is provided in the form on the `Catch22Classifier` for ease of use."
+    "Catch22 extracts the most important features for machine learning tasks and therefore is more widely used."
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 23,
-   "metadata": {
-    "execution": {
-     "iopub.execute_input": "2020-12-19T14:30:08.431962Z",
-     "iopub.status.busy": "2020-12-19T14:30:08.419431Z",
-     "iopub.status.idle": "2020-12-19T14:30:08.535295Z",
-     "shell.execute_reply": "2020-12-19T14:30:08.535836Z"
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "(67, 24)\n"
+     ]
     }
-   },
+   ],
+   "source": [
+    "c24 = Catch22(catch24=True)\n",
+    "data_c24 = c24.fit_transform(IPD_X_train)\n",
+    "print(data_c24.shape)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### 2.3 Replace NaNs"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "You may find that some time series cannot extract certain features from it. This may happen when division by zero occurs, or the input value is zero. Simply, it means we cannot extract the feature from the time series. However, we may still want a number for calculations and therefore 'replace_nans' allows us to replace NaN with zero."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
    "outputs": [
     {
-     "data": {
-      "text/html": [
-       "<style>#sk-container-id-5 {color: black;background-color: white;}#sk-container-id-5 pre{padding: 0;}#sk-container-id-5 div.sk-toggleable {background-color: white;}#sk-container-id-5 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-5 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-5 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-5 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-5 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-5 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-5 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-5 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-5 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-5 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-5 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-5 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-5 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-5 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-5 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-5 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-5 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-5 div.sk-item {position: relative;z-index: 1;}#sk-container-id-5 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-5 div.sk-item::before, #sk-container-id-5 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-5 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-5 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-5 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-5 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-5 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-5 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-5 div.sk-label-container {text-align: center;}#sk-container-id-5 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-5 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-5\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>Catch22Classifier(random_state=0)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-5\" type=\"checkbox\" checked><label for=\"sk-estimator-id-5\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">Catch22Classifier</label><div class=\"sk-toggleable__content\"><pre>Catch22Classifier(random_state=0)</pre></div></div></div></div></div>"
-      ],
-      "text/plain": [
-       "Catch22Classifier(random_state=0)"
-      ]
-     },
-     "execution_count": 23,
-     "metadata": {},
-     "output_type": "execute_result"
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Data with NaN: [       nan        nan 1.         0.         0.         6.\n",
+      " 6.         0.                nan 0.         0.         0.\n",
+      " 3.         0.         1.         1.60943791 1.                nan\n",
+      "        nan        nan 0.08       0.        ]\n",
+      "\n",
+      "Data with no NaN:  [0.         0.         1.         0.         0.         6.\n",
+      " 6.         0.         0.         0.         0.         0.\n",
+      " 3.         0.         1.         1.60943791 1.         0.\n",
+      " 0.         0.         0.08       0.        ]\n"
+     ]
     }
    ],
    "source": [
-    "c22f = Catch22Classifier(random_state=0)\n",
-    "c22f.fit(IPD_X_train, IPD_y_train)"
+    "training_data = np.array([[0, 0, 0, 0, 0, 0]])\n",
+    "\n",
+    "c22_nan = Catch22()\n",
+    "data_nan = c22_nan.fit_transform(training_data)\n",
+    "print(f\"Data with NaN: {data_nan[0]}\\n\")\n",
+    "\n",
+    "c22_no_nan = Catch22(replace_nans=True)\n",
+    "data_no_nan = c22_no_nan.fit_transform(training_data)\n",
+    "print(\"Data with no NaN: \", data_no_nan[0])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### 2.4 Pycatch22"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Pycatch22 is the original implementation of catch22 based on \\[1\\]. Aeon allows you to use pycatch22 by setting the parameter 'use_pycatch22' to true. The difference of the two is that pycatch22 uses C as their backend while python uses the Numba library, which assembles python code into C. Aeon also regularly maintains their catch22 library, and therefore there should be barely any discrepancy between outputs. Pycatch22 has a few issues with their implementation such as at times struggling to run on windows. If you are using the aeon library for a certain task, but want to use pycatch22 for transformation of the data, it is recommended to use aeon's catch22 with the parameter 'use_pycatch22' set to true. If you do that, you may encounter a warning that pycatch22 has not been installed and therefore will use aeon's catch22, if that happens just install the pycatch22 library.\n",
+    "\n",
+    "Currently, pycatch22 has an issue where the output features extracted using Python yield different values compared to those extracted using the native C code. Aeon's catch22 implementation extracts the same results as pycatch22's C code. Therefore, the extracted results may differ."
    ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
-   "metadata": {
-    "execution": {
-     "iopub.execute_input": "2020-12-19T14:30:08.553299Z",
-     "iopub.status.busy": "2020-12-19T14:30:08.552508Z",
-     "iopub.status.idle": "2020-12-19T14:30:08.561331Z",
-     "shell.execute_reply": "2020-12-19T14:30:08.561821Z"
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Pycatch22 :  [-0.57058807 -0.73624268  4.          0.625      -0.45833333  2.45190656\n",
+      "  6.          0.42507544  0.58904862  0.92048041  0.11344743  0.37262397\n",
+      "  3.          0.86956522  6.          1.81200059  0.75        0.15104572\n",
+      "  0.          0.          0.04        0.        ]\n",
+      "aeon catch22 :  [ 0.09203038 -0.73624265  7.          0.625      -0.45833333  3.\n",
+      "  6.          0.42507544  0.58904862  0.8982969   0.11344743  0.37262397\n",
+      "  3.          0.86956522  4.          1.83902118  0.75        0.15104572\n",
+      "         nan         nan  0.06666667  0.        ]\n"
+     ]
     }
-   },
-   "outputs": [],
+   ],
    "source": [
-    "c22f_preds = c22f.predict(IPD_X_test)\n",
-    "print(\"C22F Accuracy: \" + str(metrics.accuracy_score(IPD_y_test, c22f_preds)))"
+    "py22 = Catch22(use_pycatch22=True)\n",
+    "data_py22 = py22.fit_transform(IPD_X_test)\n",
+    "print(f\"Pycatch22 : {data_py22[667]}\\n\")\n",
+    "\n",
+    "py22 = Catch22()\n",
+    "data_py22 = py22.fit_transform(IPD_X_test)\n",
+    "print(\"aeon catch22 : \", data_py22[667])"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### References\n",
     "\n",
-    "[1] Carl H Lubba, Sarab S Sethi, Philip Knaute, Simon R Schultz, Ben D Fulcher*, Nick S Jones*. catch22: CAnonical Time-series CHaracteristics (2019)\n",
-    "\n"
+    "## 3. References:\n",
+    "\n",
+    "\\[1\\] Lubba, C. H., Sethi, S. S., Knaute, P., Schultz, S. R., Fulcher, B. D., & Jones, N. S. (2019). catch22: CAnonical Time-series CHaracteristics. Data Mining and Knowledge Discovery, 33(6), 1821-1852.\n",
+    "\n",
+    "\\[2\\] Fulcher, B. D., & Jones, N. S. (2017). hctsa: A computational framework for automated time-series phenotyping using massive feature extraction. Cell systems, 5(5), 527-531.\n",
+    "\n",
+    "\\[3\\] Fulcher, B. D., Little, M. A., & Jones, N. S. (2013). Highly comparative time-series analysis: the empirical structure of time series and their methods. Journal of the Royal Society Interface, 10(83), 20130048.\n",
+    "\n",
+    "\\[4\\] Agrahari, R., Nicholson, M., Conran, C., Assem, H. and Kelleher, J.D., 2022. Assessing feature representations for instance-based cross-domain anomaly detection in cloud services univariate time series data. IoT, 3(1), pp.123-144."
    ]
   }
  ],
@@ -282,7 +391,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.5"
+   "version": "3.11.9"
   },
   "toc": {
    "base_numbering": 1,


Test patch:
diff --git a/aeon/classification/sklearn/tests/test_rotation_forest_classifier.py b/aeon/classification/sklearn/tests/test_rotation_forest_classifier.py
index 605203d6..bbaea821 100644
--- a/aeon/classification/sklearn/tests/test_rotation_forest_classifier.py
+++ b/aeon/classification/sklearn/tests/test_rotation_forest_classifier.py
@@ -1,10 +1,12 @@
 """Rotation Forest test code."""
 
 import numpy as np
+import pytest
 from sklearn.metrics import accuracy_score
 
 from aeon.classification.sklearn import RotationForestClassifier
 from aeon.datasets import load_unit_test
+from aeon.testing.data_generation import make_example_3d_numpy
 
 
 def test_rotf_output():
@@ -81,3 +83,19 @@ def test_rotf_fit_predict():
     y_proba = rotf.predict_proba(X_train)
     assert isinstance(y_proba, np.ndarray)
     assert y_proba.shape == (len(y_train), 2)
+
+
+def test_rotf_input():
+    """Test RotF with incorrect input."""
+    rotf = RotationForestClassifier()
+    X2 = rotf._check_X(np.random.random((10, 1, 100)))
+    assert X2.shape == (10, 100)
+    with pytest.raises(
+        ValueError, match="RotationForestClassifier is not a time series classifier"
+    ):
+        rotf._check_X(np.random.random((10, 10, 100)))
+    with pytest.raises(
+        ValueError, match="RotationForestClassifier is not a time series classifier"
+    ):
+        rotf._check_X([[1, 2, 3], [4, 5], [6, 7, 8]])
+    X, y = make_example_3d_numpy()
diff --git a/aeon/clustering/deep_learning/tests/test_clusterer_features.py b/aeon/clustering/deep_learning/tests/test_clusterer_features.py
new file mode 100644
index 00000000..71e57619
--- /dev/null
+++ b/aeon/clustering/deep_learning/tests/test_clusterer_features.py
@@ -0,0 +1,23 @@
+"""Tests whether various clusterer params work well."""
+
+import numpy as np
+import pytest
+
+from aeon.clustering.deep_learning import AEFCNClusterer
+from aeon.utils.validation._dependencies import _check_soft_dependencies
+
+
+@pytest.mark.skipif(
+    not _check_soft_dependencies(["tensorflow"], severity="none"),
+    reason="Tensorflow soft dependency not found.",
+)
+def test_multi_rec_fcn():
+    """Tests whether multi-rec loss works fine or not."""
+    X = np.random.random((100, 5, 2))
+    clst = AEFCNClusterer(
+        n_clusters=2, n_epochs=10, n_filters=[2, 3, 4], loss="multi_rec"
+    )
+    clst.fit(X)
+    assert (
+        clst.history["loss"][0] > clst.history["loss"][9]
+    )  # Check if loss is decreasing.
diff --git a/aeon/networks/tests/test_dcnn.py b/aeon/networks/tests/test_dcnn.py
new file mode 100644
index 00000000..70ba3517
--- /dev/null
+++ b/aeon/networks/tests/test_dcnn.py
@@ -0,0 +1,50 @@
+"""Tests for the DCNN Model."""
+
+import random
+
+import pytest
+
+from aeon.networks import DCNNNetwork
+from aeon.utils.validation._dependencies import _check_soft_dependencies
+
+
+@pytest.mark.skipif(
+    not _check_soft_dependencies(["tensorflow"], severity="none"),
+    reason="Tensorflow soft dependency unavailable.",
+)
+@pytest.mark.parametrize(
+    "latent_space_dim,n_layers",
+    [
+        (32, 1),
+        (128, 2),
+        (256, 3),
+        (64, 4),
+    ],
+)
+def test_dcnnnetwork_init(latent_space_dim, n_layers):
+    """Test whether DCNNNetwork initializes correctly for various parameters."""
+    dcnnnet = DCNNNetwork(
+        latent_space_dim=latent_space_dim,
+        n_layers=n_layers,
+        activation=random.choice(["relu", "tanh"]),
+        n_filters=[random.choice([50, 25, 100]) for _ in range(n_layers)],
+    )
+    model = dcnnnet.build_network((1000, 5))
+    assert model is not None
+
+
+@pytest.mark.skipif(
+    not _check_soft_dependencies(["tensorflow"], severity="none"),
+    reason="Tensorflow soft dependency unavailable.",
+)
+@pytest.mark.parametrize("activation", ["relu", "tanh"])
+def test_dcnnnetwork_activations(activation):
+    """Test whether DCNNNetwork initializes correctly with different activations."""
+    dcnnnet = DCNNNetwork(
+        latent_space_dim=64,
+        n_layers=2,
+        activation=activation,
+        n_filters=[50, 50],
+    )
+    model = dcnnnet.build_network((150, 5))
+    assert model is not None