Repository: sktime/sktime

Issue:
[ENH] Initial centroid guess for kmeans
This package is great, thanks for all the hard work.

**Is your feature request related to a problem?**
No

**Describe the solution you'd like**

It would be nice to be able to provide the `TimeSeriesKMeans` class with an initial guess of clusters similar to `KMeans` in `sklearn`, https://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html. In `sklearn` you can pass a `numpy` array representing your initial guess to the `init` argument.

This would mean updating the `init_algorithm` argument for `TimeSeriesKMeans` to accept an array containing the initial centroid guesses. The dimensions of the array would be the same as the `cluster_centers_` attribute.

Alternatively, since an array of centroids is not an algorithm, perhaps it would make more sense add a new argument called `init_centroids` instead of adding it as an option to the `init_algorithm` argument. Or, the `init_algorithm` argument could be renamed to `init` which would bring it more inline with `sklearn`.



Discussion:
Hm, the current `TimeSeriesKMeans` is a de-novo re-implementation, so there are two approaches we could take:

1. change/extend the current `TimeSeriesKMeans`
2. build a direct `sklearn` interface that "inherits" init etc

Let us know which option you would prefer, and whether you would be helping out with this (happy to get you started with the codebase and/or open source contribs)!

### changing current `TimeSeriesKMeans` de-novo re-implementation

The new init algorithm would have to be added in `TimeSeriesLloyds` attribute `_init_algorithms` (line 209).
Adding a "fixed init" strategy should not be too difficult here.
(besides adding, the parameter checks also need to be changed to allow the new input value)

Re renaming, we can do that and it makes sense, but we would need to go through deprecation.
https://www.sktime.net/en/latest/developer_guide/deprecation.html
(e.g., adding the new parameter instead of changing, and raising a warning if the old one is used to tell users to use the new one, then changing the default overide in 0.21.0, then remove the old parameter in 0.22.0)

The algorithm looks ownerless at the moment (needs to be confirmed) so you could make the changes you are requesting.

### direct `sklearn` interface for k-means

An alternative approach would be to write another k-means class which directly interfaces the `sklearn` one and therefore also "inherits" all parameters, algorithms, bugfixes etc.

The pattern to follow here is relatively easy imo, you can take `TimeSeriesDBSCAN` as a template here (the file is copy-pasteable).
The idea is that distance/kernel based `sklearn` estimators can be invoked with distance/kernel matrices, so we compute the latter in `sktime` and then pass them to the former, wrapping a `sklearn` estimator under the hood.

(side note: string-to-distance resolution from registry is missing for a number of estimators currently, incl `TimeSeriersDBSCAN`, but you can leave this for now as it is being dealt with in another PR here: https://github.com/sktime/sktime/pull/4712)

(side note 2: for now, we could copy-paste `TimeSeriesDBSCAN`, but if we do this more often, we may want to refactor to a "common ancestor class" and use the delegator class to be more DRY)
Option 2 sounds good to me and yes I would like to help with this.

I see the dbscan python file https://github.com/sktime/sktime/blob/main/sktime/clustering/dbscan.py.

Would you like this as a new class in https://github.com/sktime/sktime/blob/main/sktime/clustering/k_means.py or would like it to replace the current one?

if you would like a new class, what should the new class be called?
> Option 2 sounds good to me and yes I would like to help with this.

Excellent, much appreciated!

> Would you like this ...

For reasons of not breaking anything for our users, we cannot replace or move the current class (in terms of import location and functionality), at least not right away. I would also recommend to avoid modifying the existing file due to it being coupled to the Lloyds class and `numba` (via the `distances` module).

Now, this results in a slightly hard problem to come up with a non-confusing solution that can be a combination of naming a new file, potentially introducing a folder, and potentially renaming with deprecation.

Thoughts, I cannot come up with a solution that sounds "obvious":
* perhaps a folder `k_means` to replace the current file, and containing the old and the new file (private) and an `__init__` which exports all classes? (it's important to export in a public name module, or the test framework doesn't see it)
* perhaps name it `TimeSeriesKMeansSklearn`?
    * Noting that `tslearn` also has a time series k-means (confusingly, of the same name as the `sktime` native class), this could later be a `TimeSeriesKMeansTslearn`? On a side note, an example for `tslearn` interface pattern is in `kernel_k_means.py`, also copy-pasteable (probably should be a proper adapter instead).

FYI, `sktime`'s policy is currently closer to the "library" paradigm than to sklearn's "opinionated selection" paradigm, especially since there's less of a canon in the time series space. See here:
https://www.sktime.net/en/latest/get_involved/governance.html#algorithm-inclusion-guidelines
(an implication is, we avoid throwing out, rewriting, or replacing "books", instead we work by rearranging bookshelfs if some segments become more popular)

Another example of this are the multiple SARIMAX algorithms we have - `statsmodels`, `statsforecast`, and `pmdarima`. All of these are popular with users, many people having strong opinions on one vs the other, so why play the role of uninvited arbiter, instead of just facilitating use and comparison?
@Alex-JG3, I looked into the code, and the following should be possible in `TimeSeriesKMeans`:

to `init_algorithm` of `TimeSeriesKMeans`, pass a function that returns a 3D `np.array`, (n_instance, n_vars, n_timepoints), these are `n_instance` cluster centers.

What I think *should* be done:

* allow 3D `np.array` as `init_algorithm`
* in `__init__` of `TimeSeriesLloyds`, catch `np.array` and replace by this function
* update the docstring that this is possible.
@fkiraly, yes I see what you mean. I will open a new pull request and try and make these changes.

Resolving patch:
diff --git a/sktime/clustering/k_means.py b/sktime/clustering/k_means.py
--- a/sktime/clustering/k_means.py
+++ b/sktime/clustering/k_means.py
@@ -19,9 +19,15 @@ class TimeSeriesKMeans(TimeSeriesLloyds):
     n_clusters: int, defaults = 8
         The number of clusters to form as well as the number of
         centroids to generate.
-    init_algorithm: str, defaults = 'random'
-        Method for initializing cluster centers. Any of the following are valid:
-        ['kmeans++', 'random', 'forgy']
+    init_algorithm: str, np.ndarray (3d array of shape (n_clusters, n_dimensions,
+        series_length)), defaults = 'random'
+        Method for initializing cluster centers or an array of initial cluster centers.
+        If string, any of the following strings are valid:
+            ['kmeans++', 'random', 'forgy'].
+        If 3D np.ndarray, initializes cluster centers with the provided array. The array
+            must have shape (n_clusters, n_dimensions, series_length) and the number of
+            clusters in the array must be the same as what is provided to the n_clusters
+            argument.
     metric: str or Callable, defaults = 'dtw'
         Distance metric to compute similarity between time series. Any of the following
         are valid: ['dtw', 'euclidean', 'erp', 'edr', 'lcss', 'squared', 'ddtw', 'wdtw',
@@ -204,10 +210,20 @@ def get_test_params(cls, parameter_set="default"):
             `MyClass(**params)` or `MyClass(**params[i])` creates a valid test instance.
             `create_test_instance` uses the first (or only) dictionary in `params`
         """
-        return {
+        params1 = {
             "n_clusters": 2,
             "metric": "euclidean",
+            "init_algorithm": "random",
             "n_init": 1,
             "max_iter": 10,
             "random_state": 0,
         }
+        params2 = {
+            "n_clusters": 3,
+            "metric": "dtw",
+            "init_algorithm": np.random.normal(size=(3, 4, 10)),
+            "n_init": 1,
+            "max_iter": 15,
+            "random_state": 1,
+        }
+        return [params1, params2]
diff --git a/sktime/clustering/k_medoids.py b/sktime/clustering/k_medoids.py
--- a/sktime/clustering/k_medoids.py
+++ b/sktime/clustering/k_medoids.py
@@ -19,9 +19,15 @@ class TimeSeriesKMedoids(TimeSeriesLloyds):
     n_clusters: int, defaults = 8
         The number of clusters to form as well as the number of
         centroids to generate.
-    init_algorithm: str, defaults = 'forgy'
-        Method for initializing cluster centers. Any of the following are valid:
-        ['kmeans++', 'random', 'forgy']
+    init_algorithm: str, np.ndarray (3d array of shape (n_clusters, n_dimensions,
+        series_length)), defaults = 'forgy'
+        Method for initializing cluster centers or an array of initial cluster centers.
+        If string, any of the following strings are valid:
+            ['kmeans++', 'random', 'forgy'].
+        If 3D np.ndarray, initializes cluster centers with the provided array. The array
+            must have shape (n_clusters, n_dimensions, series_length) and the number of
+            clusters in the array must be the same as what is provided to the n_clusters
+            argument.
     metric: str or Callable, defaults = 'dtw'
         Distance metric to compute similarity between time series. Any of the following
         are valid: ['dtw', 'euclidean', 'erp', 'edr', 'lcss', 'squared', 'ddtw', 'wdtw',
diff --git a/sktime/clustering/partitioning/_lloyds.py b/sktime/clustering/partitioning/_lloyds.py
--- a/sktime/clustering/partitioning/_lloyds.py
+++ b/sktime/clustering/partitioning/_lloyds.py
@@ -162,9 +162,10 @@ class TimeSeriesLloyds(BaseClusterer, ABC):
     n_clusters: int, defaults = 8
         The number of clusters to form as well as the number of
         centroids to generate.
-    init_algorithm: str, defaults = 'forgy'
-        Method for initializing cluster centers. Any of the following are valid:
-        ['kmeans++', 'random', 'forgy']
+    init_algorithm: str, np.ndarray (3d array of shape (n_clusters, n_dimensions,
+        series_length)), defaults = 'forgy'
+        Method for initializing cluster centers or an array of initial cluster centers.
+        Any of the following strings are valid: ['kmeans++', 'random', 'forgy']
     metric: str or Callable, defaults = 'dtw'
         Distance metric to compute similarity between time series. Any of the following
         are valid: ['dtw', 'euclidean', 'erp', 'edr', 'lcss', 'squared', 'ddtw', 'wdtw',
@@ -240,6 +241,7 @@ def __init__(
 
         self._random_state = None
         self._init_algorithm = None
+        self._initial_cluster_centers = None
 
         self._distance_params = distance_params
         if distance_params is None:
@@ -264,14 +266,33 @@ def _check_params(self, X: np.ndarray) -> None:
         self._random_state = check_random_state(self.random_state)
 
         if isinstance(self.init_algorithm, str):
-            self._init_algorithm = self._init_algorithms.get(self.init_algorithm)
-        else:
+            if self.init_algorithm not in self._init_algorithms.keys():
+                raise KeyError(
+                    f"The string provided for init_algorithm: {self.init_algorithm} is "
+                    f"invalid. The following are a list of valid init algorithms "
+                    f"strings: {list(self._init_algorithms.keys())}"
+                )
+            self._init_algorithm = self._init_algorithms[self.init_algorithm]
+        elif isinstance(self.init_algorithm, np.ndarray):
+            if self.init_algorithm.ndim != 3:
+                raise ValueError(
+                    "The array provided to init_algorithm is invalid. It must be 3D."
+                )
+            if self.init_algorithm.shape[0] != self.n_clusters:
+                raise ValueError(
+                    f"The number of centers in init_algorithm and n_clusters must be "
+                    f"the same. The number of centres in init_algorithm is "
+                    f"{self.init_algorithm.shape[0]} but n_clusters is "
+                    f"{self.n_clusters}"
+                )
             self._init_algorithm = self.init_algorithm
-
-        if not isinstance(self._init_algorithm, Callable):
+        elif isinstance(self._init_algorithm, Callable):
+            self._init_algorithm = self.init_algorithm
+        else:
             raise ValueError(
-                f"The value provided for init_algorim: {self.init_algorithm} is "
-                f"invalid. The following are a list of valid init algorithms strings: "
+                f"The value provided for init_algorithm: {self.init_algorithm} is "
+                f"invalid. It must be either a string, callable or 3D numpy array."
+                f" The following are a list of valid init algorithms strings: "
                 f"{list(self._init_algorithms.keys())}"
             )
 
@@ -377,12 +398,15 @@ def _fit_one_init(self, X) -> Tuple[np.ndarray, np.ndarray, float, int]:
             Sum of squared distances of samples to their closest cluster center,
             weighted by the sample weights if provided.
         """
-        cluster_centres = self._init_algorithm(
-            X,
-            self.n_clusters,
-            self._random_state,
-            distance_metric=self._distance_metric,
-        )
+        if isinstance(self._init_algorithm, np.ndarray):
+            cluster_centres = self._init_algorithm
+        else:
+            cluster_centres = self._init_algorithm(
+                X,
+                self.n_clusters,
+                self._random_state,
+                distance_metric=self._distance_metric,
+            )
         old_inertia = np.inf
         old_labels = None
         for i in range(self.max_iter):


Test patch: