Repository: aeon-toolkit/aeon
Pull request #576 (2023-07-17 20:12:22)

[ENH] centralise and tidy up the converters
This PR has been split into two and is part of a sequence of four. 

1. https://github.com/aeon-toolkit/aeon/pull/606 implements utility functions to test data characteristics and perform conversions, all still built on datatypes. Basically duplicates some functionality currently done by multiple calls to checkisscitype
2. This PR pulls all the conversions for collections into _convert_collections, and it also provides the missing conversions (listed in NOT_IMPLEMENTED_CONVERSIONS)

Note  these are not yet called, it is a set up before revising classifier base then adding a new base class. this PR pulls the converters used by the collections estimators into one place, simplifies things in relation to datatypes and provides comprehensive conversion. Note that conversions of some of the more unusual types are not done as efficiently as they could be, since I have just combined two other conversions. Given we allow seven different data structures and some of them are never used, it seemed reasonable to me, I can create an issue to reimplement after this is in.


Next steps are:

3. Tidying up classification base class to use these 
https://github.com/aeon-toolkit/aeon/pull/551

4. generalise to a common base class inbetween BaseEstimator and BaseClassifier/Regressor/Clusterer/CollectionsTransformer to share the same conversion normalisation. 


#### Reference Issues/PRs

part of #42 see #557


## Did you add any tests for the change?

yes I think I covered it all with tests

#### Any other comments?

Wasnt sure where to put these

### PR checklist

<!--
Please go through the checklist below. Please feel free to remove points if they are not applicable.
-->

##### For all contributions
- [ ] I've added myself to the [list of contributors](https://github.com/aeon-toolkit/aeon/blob/main/.all-contributorsrc).
- [ ] Optionally, I've updated aeon's [CODEOWNERS](https://github.com/aeon-toolkit/aeon/blob/main/CODEOWNERS) to receive notifications about future changes to these files.
- [ ] The PR title starts with either [ENH], [MNT], [DOC], or [BUG] indicating whether the PR topic is related to enhancement, maintenance, documentation, or bug.
- [ ] For new estimators, I've added the estimator to the online documentation.


<!--
Thanks for contributing!
-->


review-notebook-app[bot]: Check out this pull request on&nbsp; <a href="https://app.reviewnb.com/aeon-toolkit/aeon/pull/576"><img align="absmiddle"  alt="ReviewNB" height="28" class="BotMessageButtonImage" src="https://raw.githubusercontent.com/ReviewNB/support/master/images/button_reviewnb.png"/></a> 

 See visual diffs & provide feedback on Jupyter Notebooks. 

---

 <i>Powered by <a href='https://www.reviewnb.com/?utm_source=gh'>ReviewNB</a></i>
TonyBagnall: > Is this duplicating the other conversion functionality? I do not see other stuff being removed and am wondering if thats just a todo.

it is duplicating in preparation for use in base classifier. I am not going to remove the existing ones to safely eventually deprecate datatypes. My plan is to isolate it first.
TonyBagnall: none of this is being used yet, should probably have kept it draft, but good spots all round
MatthewMiddlehurst: It can still be converted to one if its not ready, unsure at what stage it is at 🙂 
TonyBagnall: now the fit overriding classifiers are gone I will flip this around, do the base classifier first then tidy up the conversions I think

Patch:
diff --git a/.readthedocs.yml b/.readthedocs.yml
index 93a7044b..1b016c16 100644
--- a/.readthedocs.yml
+++ b/.readthedocs.yml
@@ -9,11 +9,11 @@ python:
       path: .
       extra_requirements:
         - docs
+
 build:
-  os: ubuntu-20.04
+  os: ubuntu-22.04
   tools:
-    python: "3.9"
+    python: "3.10"
 
 sphinx:
   configuration: docs/conf.py
-#  fail_on_warning: True
diff --git a/aeon/clustering/clara.py b/aeon/clustering/clara.py
index fbc93e78..023a21d6 100644
--- a/aeon/clustering/clara.py
+++ b/aeon/clustering/clara.py
@@ -87,9 +87,9 @@ class TimeSeriesCLARA(BaseClusterer):
     >>> X_train, y_train = load_basic_motions(split="TRAIN")[0:10]
     >>> X_test, y_test = load_basic_motions(split="TEST")[0:10]
     >>> # Example of PAM clustering
-    >>> km = TimeSeriesCLARA(n_clusters=3, distance="dtw", random_state=1)
+    >>> km = TimeSeriesCLARA(n_clusters=3, distance="euclidean", random_state=1)
     >>> km.fit(X_train)
-    TimeSeriesCLARA(distance='dtw', n_clusters=3, random_state=1)
+    TimeSeriesCLARA(distance='euclidean', n_clusters=3, random_state=1)
     >>> preds = km.predict(X_test)
 
     References
diff --git a/aeon/clustering/clarans.py b/aeon/clustering/clarans.py
index 9c91f7d0..093f521e 100644
--- a/aeon/clustering/clarans.py
+++ b/aeon/clustering/clarans.py
@@ -81,9 +81,9 @@ class TimeSeriesCLARANS(TimeSeriesKMedoids):
     >>> X_train, y_train = load_basic_motions(split="TRAIN")[0:10]
     >>> X_test, y_test = load_basic_motions(split="TEST")[0:10]
     >>> # Example of PAM clustering
-    >>> km = TimeSeriesCLARANS(n_clusters=3, distance="dtw", random_state=1)
+    >>> km = TimeSeriesCLARANS(n_clusters=3, distance="euclidean", random_state=1)
     >>> km.fit(X_train)
-    TimeSeriesCLARANS(distance='dtw', n_clusters=3, random_state=1)
+    TimeSeriesCLARANS(distance='euclidean', n_clusters=3, random_state=1)
     >>> preds = km.predict(X_test)
 
     References
diff --git a/aeon/clustering/k_medoids.py b/aeon/clustering/k_medoids.py
index 565eb9c0..5d3ace0c 100644
--- a/aeon/clustering/k_medoids.py
+++ b/aeon/clustering/k_medoids.py
@@ -100,9 +100,9 @@ class TimeSeriesKMedoids(BaseClusterer):
     >>> X_train, y_train = load_basic_motions(split="TRAIN")[0:10]
     >>> X_test, y_test = load_basic_motions(split="TEST")[0:10]
     >>> # Example of PAM clustering
-    >>> km = TimeSeriesKMedoids(n_clusters=3, distance="dtw", random_state=1)
+    >>> km = TimeSeriesKMedoids(n_clusters=3, distance="euclidean", random_state=1)
     >>> km.fit(X_train)
-    TimeSeriesKMedoids(distance='dtw', n_clusters=3, random_state=1)
+    TimeSeriesKMedoids(distance='euclidean', n_clusters=3, random_state=1)
     >>> pam_pred = km.predict(X_test)
 
     # Example of alternate clustering
diff --git a/aeon/utils/validation/__init__.py b/aeon/utils/validation/__init__.py
index 0ffa9b67..7fb31ad3 100644
--- a/aeon/utils/validation/__init__.py
+++ b/aeon/utils/validation/__init__.py
@@ -28,7 +28,6 @@ import numpy as np
 import pandas as pd
 
 from aeon.utils.validation.collection import (
-    equal_length,
     get_n_cases,
     get_type,
     has_missing,
diff --git a/aeon/utils/validation/_convert_collection.py b/aeon/utils/validation/_convert_collection.py
new file mode 100644
index 00000000..3eafbb43
--- /dev/null
+++ b/aeon/utils/validation/_convert_collection.py
@@ -0,0 +1,613 @@
+# -*- coding: utf-8 -*-
+"""Collection data converters.
+
+This contains all functions to convert supported collection data types.
+
+String identifier meanings (from aeon.utils.validation.collection import
+COLLECTIONS_DATA_TYPES) :
+numpy3D     : 2D numpy array of time series shape (n_cases,  n_channels, n_timepoints)
+np-list     : list of 2D numpy arrays shape (n_channels, n_timepoints_i)
+df-list     : list of 2D pandas dataframes shape (n_channels, n_timepoints_i)
+numpyflat   : 2D numpy array of univariate time series shape (n_cases, n_timepoints)
+pd-wide     : pd.DataFrame of univariate time series shape (n_cases, n_timepoints)
+nested_univ : pd.DataFrame shape (n_cases, n_channels) each cell a pd.Series
+pd-multiindex : d.DataFrame with multi-index,
+
+For the seven supported, this gives 42 different converters.
+Rather than use them directly, we recommend using the conversion dictionary
+convert_dictionary in the collections file.
+legacy code supported "dask_panel" but it is not actually used anywhere.
+"""
+import numpy as np
+import pandas as pd
+
+
+def _nested_univ_is_equal(X):
+    """Check whether series are unequal length."""
+    length = X.iloc[0, 0].size
+    for i in range(1, X.shape[0]):
+        if X.iloc[i, 0].size != length:
+            return False
+    return True
+
+
+def _is_nested_univ_dataframe(X):
+    """Check if X is nested dataframe."""
+    # Otherwise check all entries are pd.Series
+    if not isinstance(X, pd.DataFrame):
+        return False
+    for _, series in X.items():
+        for cell in series:
+            if not isinstance(cell, pd.Series):
+                return False
+    return True
+
+
+def _is_pd_wide(X):
+    """Check whether the input nested DataFrame is "pd-wide" type."""
+    # only test is if all values are float. This from chatgpt seems stupid
+    if isinstance(X, pd.DataFrame) and not isinstance(X.index, pd.MultiIndex):
+        if _is_nested_univ_dataframe(X):
+            return False
+        float_cols = X.select_dtypes(include=[float]).columns
+        for col in float_cols:
+            if not np.issubdtype(X[col].dtype, np.floating):
+                return False
+        return True
+    return False
+
+
+def convert_identity(X):
+    """Convert identity."""
+    return X
+
+
+numpy3D_error = (
+    "Error: Input should be 3-dimensional NumPy array with shape ("
+    "n_instances, n_channels, n_timepoints)."
+)
+
+
+def _from_numpy3d_to_np_list(X):
+    """Convert 3D np.ndarray to a list of 2D numpy.
+
+    Converts 3D numpy array (n_instances, n_channels, n_timepoints) to
+    a 2D list length [n_instances] each of shape (n_channels, n_timepoints)
+
+    Parameters
+    ----------
+    X : np.ndarray
+        The input array with shape (n_instances, n_channels, n_timepoints)
+
+    Returns
+    -------
+    list : list [n_instances] np.ndarray
+        A list of np.ndarray
+    """
+    if not isinstance(X, np.ndarray) and X.ndim != 3:
+        raise TypeError(numpy3D_error)
+    np_list = []
+    for arr in X:
+        np_list.append(arr)
+    return np_list
+
+
+def _from_numpy3d_to_df_list(X):
+    """Convert 3D np.darray to a list of dataframes in wide format.
+
+    Converts 3D numpy array (n_instances, n_channels, n_timepoints) to
+    a 2D list length [n_instances] of pd.DataFrames shape (n_channels, n_timepoints)
+
+    Parameters
+    ----------
+    X : np.ndarray
+        The input array with shape (n_instances, n_channels, n_timepoints)
+
+    Returns
+    -------
+    df : pd.DataFrame
+
+    Raise
+    -----
+    TypeError if X not 3D numpy array
+    """
+    if not isinstance(X, np.ndarray) and X.ndim != 3:
+        raise TypeError(numpy3D_error)
+    df_list = []
+    for x in X:
+        df_list.append(pd.DataFrame(np.transpose(x)))
+    return df_list
+
+
+def _from_numpy3d_to_pd_wide(X):
+    """Convert 3D np.darray to a list of dataframes in wide format.
+
+    Only valid with univariate time series. Converts 3D numpy array (n_instances, 1,
+    n_timepoints) to a dataframe (n_instances, n_timepoints)
+
+    Parameters
+    ----------
+    X : np.ndarray
+        The input array with shape (n_instances, n_channels, n_timepoints)
+
+    Returns
+    -------
+    df : a dataframe (n_instances,n_channels*n_timepoints)
+
+    Raise
+    -----
+    TypeError if X not 3D numpy array
+    """
+    if not isinstance(X, np.ndarray) and X.ndim != 3:
+        raise TypeError(numpy3D_error)
+    X_flat = _from_numpy3d_to_numpyflat(X)
+    return pd.DataFrame(X_flat)
+
+
+def _from_numpy3d_to_numpyflat(X):
+    """Convert 3D np.darray to a 2D np.ndarray."""
+    if not isinstance(X, np.ndarray) and X.ndim != 3:
+        raise TypeError(numpy3D_error)
+    X_flat = X.reshape(X.shape[0], X.shape[1] * X.shape[2])
+    return X_flat
+
+
+def _from_numpy3d_to_pd_multiindex(X):
+    """Convert numpy3D collection to pandas multi-index collection."""
+    if not isinstance(X, np.ndarray) and X.ndim != 3:
+        raise TypeError(numpy3D_error)
+
+    n_instances, n_channels, n_timepoints = X.shape
+    multi_index = pd.MultiIndex.from_product(
+        [range(n_instances), range(n_channels), range(n_timepoints)],
+        names=["instances", "columns", "timepoints"],
+    )
+
+    X_mi = pd.DataFrame({"X": X.flatten()}, index=multi_index)
+    X_mi = X_mi.unstack(level="columns")
+    X_mi.columns = [f"var_{i}" for i in range(n_channels)]
+    return X_mi
+
+
+def _from_numpy3d_to_nested_univ(X):
+    """Convert numpy3D collection to nested_univ pd.DataFrame."""
+    if not isinstance(X, np.ndarray) and X.ndim != 3:
+        raise TypeError(numpy3D_error)
+    n_instances, n_channels, n_timepoints = X.shape
+    array_type = X.dtype
+    container = pd.Series
+    column_names = [f"var_{i}" for i in range(n_channels)]
+    column_list = []
+    for j, column in enumerate(column_names):
+        nested_column = (
+            pd.DataFrame(X[:, j, :])
+            .apply(lambda x: [container(x, dtype=array_type)], axis=1)
+            .str[0]
+            .rename(column)
+        )
+        column_list.append(nested_column)
+    df = pd.concat(column_list, axis=1)
+    return df
+
+
+np_list_error = "input should be a list of 2D np.ndarray"
+
+
+def _from_np_list_to_numpy3d(X):
+    """Convert from a list of 2D numpy to 3d numpy."""
+    if not isinstance(X, list):
+        raise TypeError(np_list_error)
+    for i in range(1, len(X)):
+        if len(X[i][0]) != len(X[0][0]):
+            raise TypeError(
+                "Error time series not equal length, cannot convert to numpy3D"
+            )
+    return np.array(X)
+
+
+def _from_np_list_to_df_list(X):
+    """Convert from a list of 2D numpy to list of dataframes."""
+    if not isinstance(X, list) or not isinstance(X[0], np.ndarray) or X[0].ndim != 2:
+        raise TypeError(np_list_error)
+    n_cases = len(X)
+    df_list = []
+    for i in range(n_cases):
+        df_list.append(pd.DataFrame(np.transpose(X[i])))
+    return df_list
+
+
+def _from_np_list_to_nested_univ(X, store=None):
+    """Convert from a a list of 2D numpy to nested pd.DataFrame."""
+    if not isinstance(X, list) or not isinstance(X[0], np.ndarray) or X[0].ndim != 2:
+        raise TypeError(np_list_error)
+    n_cases = len(X)
+    n_channels = X[0].shape[0]
+    df = pd.DataFrame(index=range(n_cases), columns=range(n_channels))
+    for i in range(n_cases):
+        for j in range(n_channels):
+            data = pd.Series(X[i][j])
+            df.iloc[i][j] = data
+    return df
+
+
+def _from_np_list_to_numpyflat(X):
+    if not isinstance(X, list) or not isinstance(X[0], np.ndarray):
+        raise TypeError(np_list_error)
+    X_3d = _from_np_list_to_numpy3d(X)
+    return _from_numpy3d_to_numpyflat(X_3d)
+
+
+def _from_np_list_to_pd_wide(X):
+    X = _from_np_list_to_numpyflat(X)
+    return pd.DataFrame(X)
+
+
+def _from_np_list_to_pd_multiindex(X):
+    X_df = _from_np_list_to_df_list(X)
+    return _from_df_list_to_pd_multiindex(X_df)
+
+
+df_list_error = (
+    "Input should be 2-dimensional np.ndarray with shape (n_instances, "
+    "n_timepoints).",
+)
+
+
+def _from_df_list_to_np_list(X):
+    list = []
+    n_cases = len(X)
+    for i in range(n_cases):
+        list.append(np.transpose(np.array(X[i])))
+    return list
+
+
+def _from_df_list_to_numpy3d(X):
+    n = len(X[0])
+    cols = set(X[0].columns)
+
+    for i in range(len(X)):
+        if not n == len(X[i]) or not set(X[i].columns) == cols:
+            raise TypeError("elements of obj must have same length and columns")
+    nump3D = np.array([x.to_numpy().transpose() for x in X])
+    return nump3D
+
+
+def _from_df_list_to_numpyflat(X):
+    if not _equal_length(X, "df-list"):
+        raise TypeError(
+            f"{type(X)} does not store equal length series, "
+            f"cannot convert to  numpy flat"
+        )
+    np_list = _from_df_list_to_np_list(X)
+    return _from_np_list_to_numpyflat(np_list)
+
+
+def _from_df_list_to_pd_wide(X):
+    if not _equal_length(X, "df-list"):
+        raise TypeError(
+            f"{type(X)} does not store equal length series, "
+            f"cannot convert to  pd wide"
+        )
+    np_list = _from_df_list_to_np_list(X)
+    return _from_np_list_to_pd_wide(np_list)
+
+
+def _from_df_list_to_nested_univ(X):
+    np_list = _from_df_list_to_np_list(X)
+    return _from_np_list_to_nested_univ(np_list)
+
+
+def _from_df_list_to_pd_multiindex(X):
+    n = len(X)
+    mi = pd.concat(X, axis=0, keys=range(n), names=["instances", "timepoints"])
+    return mi
+
+
+numpyflat_error = (
+    "Input should be 2-dimensional np.ndarray with shape (n_instances, "
+    "n_timepoints).",
+)
+
+
+def _from_numpyflat_to_numpy3d(X):
+    if X.ndim != 2:
+        raise TypeError(numpyflat_error)
+    X_3d = X.reshape(X.shape[0], 1, X.shape[1])
+    return X_3d
+
+
+def _from_numpyflat_to_np_list(X):
+    if not isinstance(X, np.ndarray) or X.ndim != 2:
+        raise TypeError(numpyflat_error)
+    X_3d = X.reshape(X.shape[0], 1, X.shape[1])
+    X_list = []
+    for x in X_3d:
+        X_list.append(x)
+    return X_list
+
+
+def _from_numpyflat_to_df_list(X):
+    """Convert numpyflat collection to list of dataframe shape (1,n_timepoints).
+
+    Parameters
+    ----------
+    X : np.ndarray
+        2-dimensional np.ndarray (n_instances, n_timepoints)
+
+    Returns
+    -------
+    X_mi : list
+        list of pd.DataFrame (n_instances, n_timepoints)
+    """
+    if not isinstance(X, np.ndarray) or X.ndim != 2:
+        raise TypeError(numpyflat_error)
+    X_3d = X.reshape(X.shape[0], 1, X.shape[1])
+    X_list = []
+    for x in X_3d:
+        X_list.append(pd.DataFrame(np.transpose(x)))
+    return X_list
+
+
+def _from_numpyflat_to_pd_wide(X):
+    if not isinstance(X, np.ndarray) or X.ndim != 2:
+        raise TypeError(numpyflat_error)
+    return pd.DataFrame(X)
+
+
+def _from_numpyflat_to_nested_univ(X):
+    """Convert numpyflat to nested_univ format pd.DataFrame with a single column.
+
+    Parameters
+    ----------
+    X : np.ndarray shape (n_cases, n_timepoints)
+
+    Returns
+    -------
+    Xt : pd.DataFrame
+        DataFrame with a single column of pd.Series
+    """
+    if not isinstance(X, np.ndarray) or X.ndim != 2:
+        raise TypeError(numpyflat_error)
+    container = pd.Series
+    n_instances, n_timepoints = X.shape
+    time_index = np.arange(n_timepoints)
+    kwargs = {"index": time_index}
+
+    Xt = pd.DataFrame(
+        pd.Series([container(X[i, :], **kwargs) for i in range(n_instances)])
+    )
+    return Xt
+
+
+def _from_numpyflat_to_pd_multiindex(X):
+    X_3d = _from_numpyflat_to_numpy3d(X)
+    return _from_numpy3d_to_pd_multiindex(X_3d)
+
+
+def _from_pd_wide_to_numpy3d(X):
+    X = X.to_numpy()
+    return _from_numpyflat_to_numpy3d(X)
+
+
+def _from_pd_wide_to_np_list(X):
+    X = X.to_numpy()
+    return _from_numpyflat_to_np_list(X)
+
+
+def _from_pd_wide_to_df_list(X):
+    X = X.to_numpy()
+    return _from_numpyflat_to_df_list(X)
+
+
+def _from_pd_wide_to_numpyflat(X):
+    return X.to_numpy()
+
+
+def _from_pd_wide_to_nested_univ(X):
+    X = X.to_numpy()
+    return _from_numpyflat_to_nested_univ(X)
+
+
+def _pd_wide_to_pd_multiindex(X):
+    X_3d = _from_pd_wide_to_numpy3d(X)
+    return _from_numpy3d_to_pd_multiindex(X_3d)
+
+
+def _from_nested_univ_to_numpy3d(X):
+    """Convert nested Panel to 3D numpy Panel.
+
+    Needs to check equal length, but this is legacy only.
+    """
+    if not _nested_univ_is_equal(X):
+        raise TypeError(
+            "Error, nested univ contains unequal length series, "
+            "cannot convert to numpy3D"
+        )
+
+    def _convert_series_cell_to_numpy(cell):
+        if isinstance(cell, pd.Series):
+            return cell.to_numpy()
+        else:
+            return cell
+
+    X_3d = np.stack(
+        X.applymap(_convert_series_cell_to_numpy)
+        .apply(lambda row: np.stack(row), axis=1)
+        .to_numpy()
+    )
+    return X_3d
+
+
+def _from_nested_univ_to_np_list(X):
+    df_list = _from_nested_univ_to_df_list(X)
+    np_list = _from_df_list_to_np_list(df_list)
+    return np_list
+
+
+def _from_nested_univ_to_pd_multiindex(X):
+    X_mi = pd.DataFrame()
+    instance_index = "instances"
+    time_index = "timepoints"
+    X_cols = X.columns
+    nested_cols = [c for c in X_cols if isinstance(X[[c]].iloc[0, 0], pd.Series)]
+    non_nested_cols = list(set(X_cols).difference(nested_cols))
+    for c in nested_cols:
+        X_col = X[[c]].explode(c)
+        X_col = X_col.infer_objects()
+        idx_df = X[[c]].applymap(lambda x: x.index).explode(c)
+        index = pd.MultiIndex.from_arrays([idx_df.index, idx_df[c].values])
+        index = index.set_names([instance_index, time_index])
+        X_col.index = index
+        X_mi[[c]] = X_col
+    for c in non_nested_cols:
+        for ix in X.index:
+            X_mi.loc[ix, c] = X[[c]].loc[ix].iloc[0]
+        X_mi[[c]] = X_mi[[c]].convert_dtypes()
+
+    return X_mi
+
+
+def _from_nested_univ_to_df_list(X):
+    # this is not already implemented, so chain two conversions
+    X_multi = _from_nested_univ_to_pd_multiindex(X)
+    return _from_pd_multiindex_to_df_list(X_multi)
+
+
+def _from_nested_univ_to_numpyflat(X):
+    if not _nested_univ_is_equal(X):
+        raise TypeError("Cannot convert unequal length series to numpyflat")
+    # convert nested data into tabular data
+    if isinstance(X, pd.Series):
+        Xt = np.array(X.tolist())
+
+    elif isinstance(X, pd.DataFrame):
+        try:
+            Xt = np.hstack([X.iloc[:, i].tolist() for i in range(X.shape[1])])
+
+        # except strange key error for specific case
+        except KeyError:
+            if (X.shape == (1, 1)) and (X.iloc[0, 0].shape == (1,)):
+                # in fact only breaks when an additional condition is met,
+                Xt = X.iloc[0, 0].values
+            else:
+                raise
+    #        if Xt.ndim != 2:
+    #            raise TypeError("Cannot convert nested_univ to numpyflat")
+    else:
+        raise TypeError(
+            f"Expected input is pandas Series or pandas DataFrame, "
+            f"but found: {type(X)}"
+        )
+    return Xt
+
+
+def _from_nested_univ_to_pd_wide(X):
+    npflat = _from_nested_univ_to_numpy3d(X)
+    return _from_numpy3d_to_pd_wide(npflat)
+
+
+def _from_pd_multiindex_to_df_list(X):
+    instance_index = X.index.levels[0]
+    Xlist = [X.loc[i].rename_axis(None) for i in instance_index]
+    return Xlist
+
+
+def _from_pd_multiindex_to_np_list(X):
+    """Convert from a nested pd.DataFrame to a list of 2D numpy."""
+    df_list = _from_pd_multiindex_to_df_list(X)
+    return _from_df_list_to_np_list(df_list)
+
+
+def _from_pd_multiindex_to_numpy3d(X):
+    df_list = _from_pd_multiindex_to_df_list(X)
+    return _from_df_list_to_numpy3d(df_list)
+
+
+def _from_pd_multiindex_to_numpyflat(X):
+    df_list = _from_pd_multiindex_to_df_list(X)
+    return _from_df_list_to_numpyflat(df_list)
+
+
+def _from_pd_multiindex_to_pd_wide(X):
+    df_list = _from_pd_multiindex_to_df_list(X)
+    return _from_df_list_to_pd_wide(df_list)
+
+
+def _from_pd_multiindex_to_nested_univ(X):
+    """Convert a pandas DataFrame witha multi-index to a nested DataFrame."""
+    instance_index = 0
+
+    # get number of distinct cases (note: a case may have 1 or many dimensions)
+    instance_idxs = X.index.get_level_values(instance_index).unique()
+
+    x_nested = pd.DataFrame()
+
+    # Loop the dimensions (columns) of multi-index DataFrame
+    for _label, _series in X.items():  # noqa
+        # Slice along the instance dimension to return list of series for each case
+        # Note: if you omit .rename_axis the returned DataFrame
+        #       prints time axis dimension at the start of each cell,
+        #       but this doesn't affect the values.
+        dim_list = [
+            _series.xs(instance_idx, level=instance_index).rename_axis(None)
+            for instance_idx in instance_idxs
+        ]
+        x_nested[_label] = pd.Series(dim_list)
+    x_nested = pd.DataFrame(x_nested).set_axis(instance_idxs)
+
+    col_msg = "Multi-index and nested DataFrames should have same columns names"
+    assert (x_nested.columns == X.columns).all(), col_msg
+
+    return x_nested
+
+
+def _equal_length(X, input_type):
+    """Test if X contains equal length time series.
+
+    Assumes input_type is a valid type (COLLECTIONS_DATA_TYPES).
+
+    Parameters
+    ----------
+    X : data structure.
+    input_type : string, one of COLLECTIONS_DATA_TYPES
+
+    Returns
+    -------
+    boolean: True if all series in X are equal length, False otherwise
+
+    Raises
+    ------
+    ValueError if input_type equals "dask_panel" or not in COLLECTIONS_DATA_TYPES.
+
+    Example
+    -------
+    >>> _equal_length( np.zeros(shape=(10, 3, 20)), "numpy3D")
+    True
+    """
+    always_equal = {"numpy3D", "numpyflat", "pd-wide"}
+    if input_type in always_equal:
+        return True
+    # np-list are shape (n_channels, n_timepoints)
+    if input_type == "np-list":
+        first = X[0].shape[1]
+        for i in range(1, len(X)):
+            if X[i].shape[1] != first:
+                return False
+        return True
+    # df-list are shape (n_timepoints, n_channels)
+    if input_type == "df-list":
+        first = X[0].shape[0]
+        for i in range(1, len(X)):
+            if X[i].shape[0] != first:
+                return False
+        return True
+    if input_type == "nested_univ":  # Nested univariate or hierachical
+        return _nested_univ_is_equal(X)
+    if input_type == "pd-multiindex":
+        # TEMPORARY: WORK OUT HOW TO TEST THESE
+        return True
+    #        raise ValueError(" Multi index not supported here ")
+    if input_type == "dask_panel":
+        raise ValueError(" DASK panel not supported here ")
+    raise ValueError(f" unknown input type {input_type}")
+    return False
diff --git a/aeon/utils/validation/collection.py b/aeon/utils/validation/collection.py
index 6a5a3836..51f3880b 100644
--- a/aeon/utils/validation/collection.py
+++ b/aeon/utils/validation/collection.py
@@ -8,12 +8,15 @@ __author__ = ["TonyBagnall"]
 import numpy as np
 import pandas as pd
 
+import aeon.utils.validation._convert_collection as conv
+from aeon.utils.validation._convert_collection import _equal_length
+
 COLLECTIONS_DATA_TYPES = [
     "numpy3D",  # 3D np.ndarray of format (n_cases, n_channels, n_timepoints)
     "np-list",  # python list of 2D numpy array of length [n_cases],
     # each of shape (n_channels, n_timepoints_i)
     "df-list",  # python list of 2D pd.DataFrames of length [n_cases], each a of
-    # shape (n_channels, n_timepoints_i)
+    # shape (n_timepoints_i, n_channels)
     "numpyflat",  # 2D np.ndarray of shape (n_cases, n_channels*n_timepoints)
     "pd-wide",  # 2D pd.DataFrame of shape (n_cases, n_channels*n_timepoints)
     "nested_univ",  # pd.DataFrame (n_cases, n_channels) with each cell a pd.Series,
@@ -21,6 +24,109 @@ COLLECTIONS_DATA_TYPES = [
     # To add "dask_panel": but not currently used anywhere
 ]
 
+convert_dictionary = dict()
+# assign identity function to type conversion to self
+for x in COLLECTIONS_DATA_TYPES:
+    convert_dictionary[(x, x)] = conv.convert_identity
+# numpy3D -> *
+convert_dictionary[("numpy3D", "np-list")] = conv._from_numpy3d_to_np_list
+convert_dictionary[("numpy3D", "df-list")] = conv._from_numpy3d_to_df_list
+convert_dictionary[("numpy3D", "pd-wide")] = conv._from_numpy3d_to_pd_wide
+convert_dictionary[("numpy3D", "numpyflat")] = conv._from_numpy3d_to_numpyflat
+convert_dictionary[("numpy3D", "nested_univ")] = conv._from_numpy3d_to_nested_univ
+convert_dictionary[("numpy3D", "pd-multiindex")] = conv._from_numpy3d_to_pd_multiindex
+# np-list-> *
+convert_dictionary[("np-list", "numpy3D")] = conv._from_np_list_to_numpy3d
+convert_dictionary[("np-list", "df-list")] = conv._from_np_list_to_df_list
+convert_dictionary[("np-list", "pd-wide")] = conv._from_np_list_to_pd_wide
+convert_dictionary[("np-list", "numpyflat")] = conv._from_np_list_to_numpyflat
+convert_dictionary[("np-list", "nested_univ")] = conv._from_np_list_to_nested_univ
+convert_dictionary[("np-list", "pd-multiindex")] = conv._from_np_list_to_pd_multiindex
+# df-list-> *
+convert_dictionary[("df-list", "numpy3D")] = conv._from_df_list_to_numpy3d
+convert_dictionary[("df-list", "np-list")] = conv._from_df_list_to_np_list
+convert_dictionary[("df-list", "pd-wide")] = conv._from_df_list_to_pd_wide
+convert_dictionary[("df-list", "numpyflat")] = conv._from_df_list_to_numpyflat
+convert_dictionary[("df-list", "nested_univ")] = conv._from_df_list_to_nested_univ
+convert_dictionary[("df-list", "pd-multiindex")] = conv._from_df_list_to_pd_multiindex
+# numpyflat -> *: NOTE ASSUMES n_channels == 1 for this conversion.
+convert_dictionary[("numpyflat", "numpy3D")] = conv._from_numpyflat_to_numpy3d
+convert_dictionary[("numpyflat", "np-list")] = conv._from_numpyflat_to_np_list
+convert_dictionary[("numpyflat", "df-list")] = conv._from_numpyflat_to_df_list
+convert_dictionary[("numpyflat", "pd-wide")] = conv._from_numpyflat_to_pd_wide
+convert_dictionary[("numpyflat", "nested_univ")] = conv._from_numpyflat_to_nested_univ
+convert_dictionary[
+    ("numpyflat", "pd-multiindex")
+] = conv._from_numpyflat_to_pd_multiindex
+# pd-wide -> *: NOTE ASSUMES n_channels == 1 for this conversion.
+convert_dictionary[("pd-wide", "numpy3D")] = conv._from_pd_wide_to_numpy3d
+convert_dictionary[("pd-wide", "np-list")] = conv._from_pd_wide_to_np_list
+convert_dictionary[("pd-wide", "df-list")] = conv._from_pd_wide_to_df_list
+convert_dictionary[("pd-wide", "numpyflat")] = conv._from_pd_wide_to_numpyflat
+convert_dictionary[("pd-wide", "nested_univ")] = conv._from_pd_wide_to_nested_univ
+convert_dictionary[("pd-wide", "pd-multiindex")] = conv._pd_wide_to_pd_multiindex
+# nested_univ -> *
+convert_dictionary[("nested_univ", "numpy3D")] = conv._from_nested_univ_to_numpy3d
+convert_dictionary[("nested_univ", "np-list")] = conv._from_nested_univ_to_np_list
+convert_dictionary[("nested_univ", "df-list")] = conv._from_nested_univ_to_df_list
+convert_dictionary[("nested_univ", "pd-wide")] = conv._from_nested_univ_to_pd_wide
+convert_dictionary[("nested_univ", "numpyflat")] = conv._from_nested_univ_to_numpyflat
+convert_dictionary[
+    ("nested_univ", "pd-multiindex")
+] = conv._from_nested_univ_to_pd_multiindex
+# pd_multiindex -> *
+convert_dictionary[("pd-multiindex", "numpy3D")] = conv._from_pd_multiindex_to_numpy3d
+convert_dictionary[("pd-multiindex", "np-list")] = conv._from_pd_multiindex_to_np_list
+convert_dictionary[("pd-multiindex", "df-list")] = conv._from_pd_multiindex_to_df_list
+convert_dictionary[("pd-multiindex", "pd-wide")] = conv._from_pd_multiindex_to_pd_wide
+convert_dictionary[
+    ("pd-multiindex", "numpyflat")
+] = conv._from_pd_multiindex_to_numpyflat
+convert_dictionary[
+    ("pd-multiindex", "nested_univ")
+] = conv._from_pd_multiindex_to_nested_univ
+
+
+def convertX(X, output_type):
+    """Convert from one of collections compatible data structure to another.
+
+    See aeon.utils.validation.collections.COLLECTIONS_DATA_TYPE for the list.
+
+    Parameters
+    ----------
+    X : data structure.
+    output_type : string, one of COLLECTIONS_DATA_TYPES
+
+    Returns
+    -------
+    Data structure conforming to "to_type"
+
+    Raises
+    ------
+    TypeError if
+        X pd.ndarray but wrong dimension
+        X is list but not of np.ndarray or p.DataFrame.
+        X is a pd.DataFrame of non float primitives.
+
+    Example
+    -------
+    >>> from aeon.utils.validation.collection import convertX, get_type
+    >>> X=convertX(np.zeros(shape=(10, 3, 20)), "np-list")
+    >>> type(X)
+    <class 'list'>
+    >>> get_type(X)
+    'np-list'
+    """
+    input_type = get_type(X)
+    if (input_type, output_type) not in convert_dictionary.keys():
+        raise TypeError(
+            f"Attempting to convert from {input_type} to {output_type} "
+            f"but this is not a valid conversion. See "
+            f"aeon.utils.validation.collections.COLLECTIONS_DATA_TYPE "
+            f"for the list of valid collections"
+        )
+    return convert_dictionary[(input_type, output_type)](X)
+
 
 def resolve_equal_length_inner_type(inner_type):
     """Hierarchy of preference for internal supported types for equal length."""
@@ -60,57 +166,6 @@ def resolve_unequal_length_inner_type(inner_type):
     )
 
 
-def convertX(X, output_type):
-    """Convert from one of collections compatible data structure to another.
-
-    See aeon.utils.validation.collections.COLLECTIONS_DATA_TYPE for the list.
-
-    Parameters
-    ----------
-    X : data structure.
-    output_type : string, one of COLLECTIONS_DATA_TYPES
-
-    Returns
-    -------
-    Data structure conforming to "to_type"
-
-    Raises
-    ------
-    ValueError if
-        X pd.ndarray but wrong dimension
-        X is list but not of np.ndarray or p.DataFrame.
-        X is a pd.DataFrame of non float primitives.
-
-    Example
-    -------
-    >>> from aeon.utils.validation.collection import convertX, get_type
-    >>> X=convertX(np.zeros(shape=(10, 3, 20)), "np-list")
-    >>> type(X)
-    <class 'list'>
-    >>> get_type(X)
-    'np-list'
-    """
-    # Temporarily retain the current conversion
-    from aeon.datatypes import convert_to
-
-    input_type = get_type(X)
-    if output_type not in COLLECTIONS_DATA_TYPES:
-        raise ValueError(
-            f"Error with convertX, trying to convert to {output_type} "
-            f"which is not a valid collection type: {COLLECTIONS_DATA_TYPES}"
-        )
-    # Temporary fix because numpyflat does not work with old conversions
-    if input_type == "numpyflat":
-        X = X.reshape(X.shape[0], 1, X.shape[1])
-    else:
-        X = convert_to(
-            X,
-            to_type=output_type,
-            as_scitype="Panel",
-        )
-    return X
-
-
 def get_n_cases(X):
     """Return the number of cases in a collectiom.
 
@@ -159,26 +214,31 @@ def get_type(X):
         elif X.ndim == 2:
             return "numpyflat"
         else:
-            raise ValueError("ERROR np.ndarray must be either 2D or 3D")
+            raise ValueError(
+                f"ERROR np.ndarray must be either 2D or 3D but found " f"{X.ndim}"
+            )
     elif isinstance(X, list):  # np-list or df-list
         if isinstance(X[0], np.ndarray):  # if one a numpy they must all be 2D numpy
             for a in X:
                 if not (isinstance(a, np.ndarray) and a.ndim == 2):
-                    raise ValueError("ERROR np-list np.ndarray must be either 2D or 3D")
+                    raise TypeError(
+                        f"ERROR np-list np.ndarray must be either 2D or "
+                        f"3D, found {a.ndim}"
+                    )
             return "np-list"
         elif isinstance(X[0], pd.DataFrame):
             for a in X:
                 if not isinstance(a, pd.DataFrame):
-                    raise ValueError("ERROR df-list must only contain pd.DataFrame")
+                    raise TypeError("ERROR df-list must only contain pd.DataFrame")
             return "df-list"
     elif isinstance(X, pd.DataFrame):  # Nested univariate, hierachical or pd-wide
-        if _is_nested_univ_dataframe(X):
+        if conv._is_nested_univ_dataframe(X):
             return "nested_univ"
         if isinstance(X.index, pd.MultiIndex):
             return "pd-multiindex"
-        elif _is_pd_wide(X):
+        elif conv._is_pd_wide(X):
             return "pd-wide"
-        raise ValueError(
+        raise TypeError(
             "ERROR unknown pd.DataFrame, contains non float values, "
             "not hierarchical nor is it nested pd.Series"
         )
@@ -206,60 +266,11 @@ def is_equal_length(X):
 
     Example
     -------
+    >>> from aeon.utils.validation.collection import is_equal_length
     >>> is_equal_length( np.zeros(shape=(10, 3, 20)))
     True
     """
-    return equal_length(X, get_type(X))
-
-
-def equal_length(X, input_type):
-    """Test if X contains equal length time series.
-
-    Assumes input_type is a valid type (COLLECTIONS_DATA_TYPES).
-
-    Parameters
-    ----------
-    X : data structure.
-    input_type : string, one of COLLECTIONS_DATA_TYPES
-
-    Returns
-    -------
-    boolean: True if all series in X are equal length, False otherwise
-
-    Raises
-    ------
-    ValueError if input_type equals "dask_panel" or not in COLLECTIONS_DATA_TYPES.
-
-    Example
-    -------
-    >>> equal_length( np.zeros(shape=(10, 3, 20)), "numpy3D")
-    True
-    """
-    always_equal = {"numpy3D", "numpyflat", "pd-wide"}
-    if input_type in always_equal:
-        return True
-    if input_type == "np-list":
-        first = X[0].shape[1]
-        for i in range(1, len(X)):
-            if X[i].shape[1] != first:
-                return False
-        return True
-    if input_type == "df-list":
-        first = X[0].shape[1]
-        for i in range(1, len(X)):
-            if X[i].shape[1] != first:
-                return False
-        return True
-    if input_type == "nested_univ":  # Nested univariate or hierachical
-        return _nested_univ_is_equal(X)
-    if input_type == "pd-multiindex":
-        # TEMPORARY: WORK OUT HOW TO TEST THESE
-        return True
-    #        raise ValueError(" Multi index not supported here ")
-    if input_type == "dask_panel":
-        raise ValueError(" DASK panel not supported here ")
-    raise ValueError(f" unknown input type {input_type}")
-    return False
+    return _equal_length(X, get_type(X))
 
 
 def has_missing(X):
@@ -318,42 +329,11 @@ def is_univariate(X):
         return True
     if type == "numpy3D" or type == "nested_univ":
         return X.shape[1] == 1
-    if type == "df-list" or type == "np-list":
+    # df list (n_timepoints, n_channels)
+    if type == "df-list":
+        return X[0].shape[1] == 1
+    # np list (n_channels, n_timepoints)
+    if type == "np-list":
         return X[0].shape[0] == 1
     if type == "pd-multiindex":
         return X.columns.shape[0] == 1
-
-
-def _nested_univ_is_equal(X):
-    """Check whether series are unequal length."""
-    length = X.iloc[0, 0].size
-    for series in X.iloc[0]:
-        if series.size != length:
-            return False
-    return True
-
-
-def _is_nested_univ_dataframe(X):
-    """Check if X is nested dataframe."""
-    # Otherwise check all entries are pd.Series
-    if not isinstance(X, pd.DataFrame):
-        return False
-    for _, series in X.items():
-        for cell in series:
-            if not isinstance(cell, pd.Series):
-                return False
-    return True
-
-
-def _is_pd_wide(X):
-    """Check whether the input nested DataFrame is "pd-wide" type."""
-    # only test is if all values are float. This from chatgpt seems stupid
-    if isinstance(X, pd.DataFrame) and not isinstance(X.index, pd.MultiIndex):
-        if _is_nested_univ_dataframe(X):
-            return False
-        float_cols = X.select_dtypes(include=[float]).columns
-        for col in float_cols:
-            if not np.issubdtype(X[col].dtype, np.floating):
-                return False
-        return True
-    return False
diff --git a/build_tools/run_examples.sh b/build_tools/run_examples.sh
index d7633467..fbb1d9d6 100755
--- a/build_tools/run_examples.sh
+++ b/build_tools/run_examples.sh
@@ -10,6 +10,10 @@ excluded=(
   "examples/benchmarking/results_loading.ipynb"
 )
 
+shopt -s lastpipe
+notebooks=()
+runtimes=()
+
 # Loop over all notebooks in the examples directory.
 find "examples/" -name "*.ipynb" -print0 |
   while IFS= read -r -d "" notebook; do
@@ -19,6 +23,16 @@ find "examples/" -name "*.ipynb" -print0 |
     # Run the notebook.
     else
       echo "Running: $notebook"
+
+      start=$(date +%s)
       $CMD "$notebook"
+      end=$(date +%s)
+
+      notebooks+=("$notebook")
+      runtimes+=($((end-start)))
     fi
   done
+
+# print runtimes and notebooks
+echo "Runtimes:"
+paste <(printf "%s\n" "${runtimes[@]}") <(printf "%s\n" "${notebooks[@]}")
diff --git a/docs/_static/css/fields.css b/docs/_static/css/fields.css
deleted file mode 100644
index f3f17365..00000000
--- a/docs/_static/css/fields.css
+++ /dev/null
@@ -1,83 +0,0 @@
-
-/*
-body {
-    font-family: Helvetica, Arial, sans-serif !important;
-    font-style: 100%;
-}
-*/
-
-table.docutils tr {
-    border-style: solid none solid none;
-    border-width: 1px 0 1px 0;
-    border-color: #ddd;
-}
-
-div.body p, div.body dd, div.body li {
-    line-height: 1.5em;
-}
-
-div.body {
-    font-size: 0.9em;
-}
-
-div.body p {
-    margin-top: 1.2em;
-    margin-bottom: .1em;
-}
-
-div.body p, div.body dd, div.body li {
-    line-height: 1.5em;
-}
-
-table.docutils td, table.docutils th {
-    padding: 1px 8px 1px 5px !important;
-    border-top: 0;
-    border-left: 0;
-    border-right: 0;
-    border-bottom: 1px solid #aaa;
-}
-
-.field-odd, .field-even {
-    background-color: #fff;
-}
-
-.field-name {
-    background-color: #F0F7FA;
-}
-
-.field-body {
-    background-color: #fff;
-}
-
-th.field-name {
-    white-space: nowrap;
-}
-
-.field-odd, .field-even {
-    background-color: #fff;
-}
-
-.field-name {
-    background-color: #F0F7FA;
-}
-
-.field-body {
-    background-color: #fff;
-}
-
-h2 > a.reference {
-    font-weight: bold;
-    color: #2878A2;
-    text-decoration: none;
-    word-wrap: break-word;
-    font-family: Arial, sans-serif;
-}
-
-a.reference > code.xref {
-    background: transparent;
-    border: none;
-    font-size: 1.1em;
-    font-family: monospace;
-    padding: 2px 4px;
-    color: #2980B9 !important;
-}
diff --git a/docs/conf.py b/docs/conf.py
index 53e1d302..5db60d05 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -4,7 +4,6 @@
 
 import os
 import sys
-from importlib import import_module
 
 import aeon
 
@@ -46,6 +45,7 @@ extensions = [
     "sphinx_design",
     "sphinx_issues",
     "sphinx_copybutton",
+    "sphinx_remove_toctrees",
     "myst_parser",
 ]
 
@@ -119,6 +119,10 @@ issues_github_path = "aeon-toolkit/aeon"
 # sphinx-copybutton configuration
 copybutton_exclude = ".linenos, .gp, .go"
 
+# sphinx-remove-toctrees configuration
+# see https://github.com/pradyunsg/furo/pull/674
+remove_from_toctrees = ["api_reference/auto_generated/*"]
+
 # MyST Parser configuration
 
 # When building HTML using the sphinx.ext.mathjax (enabled by default),
@@ -260,9 +264,6 @@ html_favicon = "images/logo/aeon-favicon.ico"
 # so a file named "default.css" will overwrite the builtin "default.css".
 html_static_path = ["_static"]
 html_css_files = ["css/custom.css"]
-html_js_files = [
-    "js/dynamic_table.js",
-]
 
 html_show_sourcelink = False
 
@@ -314,99 +315,6 @@ texinfo_documents = [
     ),
 ]
 
-
-def _make_estimator_overview(app):
-    """Make estimator overview table."""
-    import pandas as pd
-
-    from aeon.registry import all_estimators
-
-    def _process_author_info(author_info):
-        """
-        Process author information from source code files.
-
-        Parameters
-        ----------
-        author_info : str
-            Author information string from source code files.
-
-        Returns
-        -------
-        author_info : str
-            Preprocessed author information.
-
-        Notes
-        -----
-        A list of author names is turned into a string.
-        Multiple author names will be separated by a comma,
-        with the final name always preceded by "&".
-        """
-        if isinstance(author_info, list):
-            if len(author_info) > 1:
-                return ", ".join(author_info[:-1]) + " & " + author_info[-1]
-            else:
-                return author_info[0]
-        else:
-            return author_info
-
-    def _does_not_start_with_underscore(input_string):
-        return not input_string.startswith("_")
-
-    # creates dataframe as df
-    COLNAMES = ["Class Name", "Estimator Type", "Authors"]
-
-    df = pd.DataFrame([], columns=COLNAMES)
-
-    for modname, modclass in all_estimators():
-        algorithm_type = "::".join(str(modclass).split(".")[1:-2])
-        try:
-            author_info = _process_author_info(modclass.__author__)
-        except AttributeError:
-            try:
-                author_info = _process_author_info(
-                    import_module(modclass.__module__).__author__
-                )
-            except AttributeError:
-                author_info = "no author info"
-
-        # includes part of class string
-        modpath = str(modclass)[8:-2]
-        path_parts = modpath.split(".")
-        # joins strings excluding starting with '_'
-        clean_path = ".".join(list(filter(_does_not_start_with_underscore, path_parts)))
-        # adds html link reference
-        modname = str(
-            '<a href="https://www.aeon-toolkit.org/en/latest/api_reference'
-            + "/auto_generated/"
-            + clean_path
-            + '.html">'
-            + modname
-            + "</a>"
-        )
-
-        record = pd.DataFrame([modname, algorithm_type, author_info], index=COLNAMES).T
-        df = pd.concat([df, record], ignore_index=True)
-    with open("estimator_overview_table.md", "w") as file:
-        df.to_markdown(file, index=False)
-
-
-def setup(app):
-    """Set up sphinx builder.
-
-    Parameters
-    ----------
-    app : Sphinx application object
-    """
-
-    def adds(pth):
-        print("Adding stylesheet: %s" % pth)  # noqa: T201, T001
-        app.add_css_file(pth)
-
-    adds("fields.css")  # for parameters, etc.
-
-    app.connect("builder-inited", _make_estimator_overview)
-
-
 # -- Extension configuration -------------------------------------------------
 
 # -- Options for nbsphinx extension ---------------------------------------
diff --git a/docs/index.md b/docs/index.md
index 0573cdc3..6fa42efe 100644
--- a/docs/index.md
+++ b/docs/index.md
@@ -252,4 +252,5 @@ about.md
 glossary.md
 changelog.md
 related_software
+papers_using_aeon.md
 ```
diff --git a/docs/papers_using_aeon.md b/docs/papers_using_aeon.md
new file mode 100644
index 00000000..6c90c572
--- /dev/null
+++ b/docs/papers_using_aeon.md
@@ -0,0 +1,4 @@
+# Papers using Aeon
+
+
+This is a list of papers that use Aeon. If you have a paper that uses Aeon, please add it to this list by sending a pull request. Please include a hyperlink to the paper and a link to the code in your personal GitHub or other repository.
diff --git a/pyproject.toml b/pyproject.toml
index a9f594a6..62789964 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -112,6 +112,7 @@ docs = [
     "sphinx-design>=0.3.0",
     "sphinx-copybutton",
     "sphinx>=5.0.0,<6", # sphinx has <6 because of shinx-design
+    "sphinx-remove-toctrees",
     "tabulate",
     "sphinxext-opengraph",
 ]


Test patch:
diff --git a/aeon/utils/validation/tests/test_collection.py b/aeon/utils/validation/tests/test_collection.py
index 8c984223..a3d6dabb 100644
--- a/aeon/utils/validation/tests/test_collection.py
+++ b/aeon/utils/validation/tests/test_collection.py
@@ -5,15 +5,15 @@ import pandas as pd
 import pytest
 
 from aeon.datasets import make_example_multi_index_dataframe
-
-# from aeon.datasets._data_generators import make_example_multi_index_dataframe
 from aeon.utils._testing.tests.test_collection import make_nested_dataframe_data
-from aeon.utils.validation.collection import (  # _nested_uni_is_equal,; ,
-    COLLECTIONS_DATA_TYPES,
+from aeon.utils.validation._convert_collection import (
+    _equal_length,
     _is_nested_univ_dataframe,
     _is_pd_wide,
+)
+from aeon.utils.validation.collection import (
+    COLLECTIONS_DATA_TYPES,
     convertX,
-    equal_length,
     get_n_cases,
     get_type,
     has_missing,
@@ -28,7 +28,7 @@ for _ in range(10):
     np_list.append(np.random.random(size=(1, 20)))
 df_list = []
 for _ in range(10):
-    df_list.append(pd.DataFrame(np.random.random(size=(1, 20))))
+    df_list.append(pd.DataFrame(np.random.random(size=(20, 1))))
 nested, _ = make_nested_dataframe_data(n_cases=10)
 multiindex = make_example_multi_index_dataframe(
     n_instances=10, n_channels=1, n_timepoints=20
@@ -48,7 +48,7 @@ for i in range(10):
     np_list_uneq.append(np.random.random(size=(1, 20 + i)))
 df_list_uneq = []
 for i in range(10):
-    df_list_uneq.append(pd.DataFrame(np.random.random(size=(1, 20 + i))))
+    df_list_uneq.append(pd.DataFrame(np.random.random(size=(20 + i, 1))))
 
 nested_univ_uneq = pd.DataFrame(dtype=float)
 instance_list = []
@@ -56,7 +56,7 @@ for i in range(0, 10):
     instance_list.append(pd.Series(np.random.randn(20 + i)))
 nested_univ_uneq["channel0"] = instance_list
 
-UNEQUAL_LENGTH_DATA_EXAMPLES = {
+UNEQUAL_LENGTH_UNIVARIATE = {
     "np-list": np_list_uneq,
     "df-list": df_list_uneq,
     "nested_univ": nested_univ_uneq,
@@ -66,19 +66,19 @@ for _ in range(10):
     np_list_multi.append(np.random.random(size=(2, 20)))
 df_list_multi = []
 for _ in range(10):
-    df_list_multi.append(pd.DataFrame(np.random.random(size=(2, 20))))
+    df_list_multi.append(pd.DataFrame(np.random.random(size=(20, 2))))
 multi = make_example_multi_index_dataframe(
     n_instances=10, n_channels=2, n_timepoints=20
 )
 
 nested_univ_multi = pd.DataFrame(dtype=float)
 instance_list = []
-for i in range(0, 10):
-    instance_list.append(pd.Series(np.random.randn(20 + i)))
+for _ in range(0, 10):
+    instance_list.append(pd.Series(np.random.randn(20)))
 nested_univ_multi["channel0"] = instance_list
 instance_list = []
-for i in range(0, 10):
-    instance_list.append(pd.Series(np.random.randn(20 + i)))
+for _ in range(0, 10):
+    instance_list.append(pd.Series(np.random.randn(20)))
 nested_univ_multi["channel1"] = instance_list
 
 
@@ -89,42 +89,37 @@ EQUAL_LENGTH_MULTIVARIATE = {
     "nested_univ": nested_univ_multi,
     "pd-multiindex": multi,
 }
-# NotImplementedError: no conversion defined from type np-list to numpy3D under the
-# datatypes definitions. SOON TO CHANGE!
-
-NOT_IMPLEMENTED_CONVERSIONS = [
-    ("np-list", "numpy3D"),
-    ("pd-wide", "numpy3D"),
-    ("numpyflat", "np-list"),
-    ("pd-wide", "np-list"),
-    ("numpyflat", "df-list"),
-    ("pd-wide", "df-list"),
-    ("np-list", "numpyflat"),
-    ("numpyflat", "numpyflat"),
-    ("pd-wide", "numpyflat"),
-    ("numpy3D", "pd-wide"),
-    ("pd-wide", "pd-wide"),
-    ("pd-multiindex", "pd-wide"),
-    ("np-list", "pd-wide"),
-    ("df-list", "pd-wide"),
-    ("numpyflat", "pd-wide"),
-    ("numpyflat", "nested_univ"),
-    ("pd-wide", "nested_univ"),
-    ("numpyflat", "pd-multiindex"),
-    ("pd-wide", "pd-multiindex"),
-]
 
 
 @pytest.mark.parametrize("input_data", COLLECTIONS_DATA_TYPES)
 @pytest.mark.parametrize("output_data", COLLECTIONS_DATA_TYPES)
 def test_convertX(input_data, output_data):
     """Test all valid and invalid conversions."""
-    # This should be possible for all, but under the current the following are not
-    # defined
-    if (input_data, output_data) not in NOT_IMPLEMENTED_CONVERSIONS:
-        X = convertX(EQUAL_LENGTH_UNIVARIATE[input_data], output_data)
+    # All should work with univariate equal length
+    X = convertX(EQUAL_LENGTH_UNIVARIATE[input_data], output_data)
+    assert get_type(X) == output_data
+    # Test with multivariate
+    if input_data in EQUAL_LENGTH_MULTIVARIATE:
+        X = convertX(EQUAL_LENGTH_MULTIVARIATE[input_data], output_data)
         assert get_type(X) == output_data
-    # TODO: expand this test when conversions finished.
+    # Test with unequal length
+    if input_data in UNEQUAL_LENGTH_UNIVARIATE:
+        if output_data in UNEQUAL_LENGTH_UNIVARIATE or output_data == "pd-multiindex":
+            X = convertX(UNEQUAL_LENGTH_UNIVARIATE[input_data], output_data)
+            assert get_type(X) == output_data
+        else:
+            with pytest.raises(TypeError):
+                X = convertX(UNEQUAL_LENGTH_UNIVARIATE[input_data], output_data)
+
+
+@pytest.mark.parametrize("input_data", COLLECTIONS_DATA_TYPES)
+def test_convert_df_list(input_data):
+    """Test that df list is correctly transposed."""
+    X = convertX(EQUAL_LENGTH_UNIVARIATE[input_data], "df-list")
+    assert X[0].shape == (20, 1)
+    if input_data in EQUAL_LENGTH_MULTIVARIATE:
+        X = convertX(EQUAL_LENGTH_MULTIVARIATE[input_data], "df-list")
+        assert X[0].shape == (20, 2)
 
 
 def test_resolve_equal_length_inner_type():
@@ -166,7 +161,7 @@ def test_get_type(data):
 @pytest.mark.parametrize("data", COLLECTIONS_DATA_TYPES)
 def test_equal_length(data):
     """Test if equal length series correctly identified."""
-    assert equal_length(EQUAL_LENGTH_UNIVARIATE[data], data)
+    assert _equal_length(EQUAL_LENGTH_UNIVARIATE[data], data)
 
 
 @pytest.mark.parametrize("data", COLLECTIONS_DATA_TYPES)
@@ -178,13 +173,13 @@ def test_is_equal_length(data):
 @pytest.mark.parametrize("data", ["df-list", "np-list"])
 def test_unequal_length(data):
     """Test if unequal length series correctly identified."""
-    assert not equal_length(UNEQUAL_LENGTH_DATA_EXAMPLES[data], data)
+    assert not _equal_length(UNEQUAL_LENGTH_UNIVARIATE[data], data)
 
 
 @pytest.mark.parametrize("data", ["df-list", "np-list"])
 def test_is_unequal_length(data):
     """Test if unequal length series correctly identified."""
-    assert not is_equal_length(UNEQUAL_LENGTH_DATA_EXAMPLES[data])
+    assert not is_equal_length(UNEQUAL_LENGTH_UNIVARIATE[data])
 
 
 @pytest.mark.parametrize("data", COLLECTIONS_DATA_TYPES)