Repository: sktime/sktime

Issue:
[BUG] sporadic failure in HIVE-COTE2 continuous interval tree subsampling to single class data sample
HIVE-COTE2 sporadically fails on standard test scenarios, as continuous interval tree can end up sub-sampling such that the sub-sampled data contains only one class.

Example run:
https://github.com/alan-turing-institute/sktime/runs/7484288807?check_suite_focus=true

Example traceback:
```
2022-07-24T01:23:32.2345094Z self = ContinuousIntervalTree()
2022-07-24T01:23:32.2345493Z X = array([[ 0.12919995,  0.2       , -0.11130629,  0.25      ,  1.0588584 ,
2022-07-24T01:23:32.2346768Z         -0.25      ,  0.65891194,  0.5       ...     0.75      ,  3.2562025 , -0.5       , -0.08817679,  0.33333334,
2022-07-24T01:23:32.2347199Z          0.05919602,  0.25      ]], dtype=float32)
2022-07-24T01:23:32.2347537Z y = array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0])
2022-07-24T01:23:32.2347774Z 
2022-07-24T01:23:32.2347964Z     def fit(self, X, y):
2022-07-24T01:23:32.2348325Z         """Fit a tree on cases (X,y), where y is the target variable.
2022-07-24T01:23:32.2348672Z     
2022-07-24T01:23:32.2349059Z         Build an information gain based tree for continuous attributes using the
2022-07-24T01:23:32.2349447Z         margin gain metric for ties.
2022-07-24T01:23:32.2349753Z     
2022-07-24T01:23:32.2350037Z         Parameters
2022-07-24T01:23:32.2350336Z         ----------
2022-07-24T01:23:32.2350708Z         X : array-like or sparse matrix of shape = [n_instances,n_attributes]
2022-07-24T01:23:32.2351114Z         The training input samples.
2022-07-24T01:23:32.2351512Z         y : array-like, shape =  [n_instances]    The class labels.
2022-07-24T01:23:32.2351838Z     
2022-07-24T01:23:32.2352115Z         Returns
2022-07-24T01:23:32.2352401Z         -------
2022-07-24T01:23:32.2352691Z         self : object
2022-07-24T01:23:32.2352960Z         """
2022-07-24T01:23:32.2353336Z         if isinstance(X, np.ndarray) and len(X.shape) == 3 and X.shape[1] == 1:
2022-07-24T01:23:32.2353755Z             X = np.reshape(X, (X.shape[0], -1))
2022-07-24T01:23:32.2354134Z         elif not isinstance(X, np.ndarray) or len(X.shape) > 2:
2022-07-24T01:23:32.2354501Z             raise ValueError(
2022-07-24T01:23:32.2354903Z                 "ContinuousIntervalTree is not a time series classifier. "
2022-07-24T01:23:32.2355361Z                 "A valid sklearn input such as a 2d numpy array is required."
2022-07-24T01:23:32.2355802Z                 "Sparse input formats are currently not supported."
2022-07-24T01:23:32.2356157Z             )
2022-07-24T01:23:32.2356518Z         X, y = self._validate_data(X=X, y=y, ensure_min_samples=2)
2022-07-24T01:23:32.2356893Z     
2022-07-24T01:23:32.2357230Z         self.n_instances_, self.n_atts_ = X.shape
2022-07-24T01:23:32.2357574Z         self.classes_ = np.unique(y)
2022-07-24T01:23:32.2357944Z         self.n_classes_ = self.classes_.shape[0]
2022-07-24T01:23:32.2358309Z         self._class_dictionary = {}
2022-07-24T01:23:32.2358691Z         for index, classVal in enumerate(self.classes_):
2022-07-24T01:23:32.2359076Z             self._class_dictionary[classVal] = index
2022-07-24T01:23:32.2359402Z     
2022-07-24T01:23:32.2359699Z         if self.n_classes_ == 1:
2022-07-24T01:23:32.2360076Z >           raise ValueError("fit input y must contain more than one class")
2022-07-24T01:23:32.2360523Z E           ValueError: fit input y must contain more than one class
```


Discussion:
This most likely appeared as a result of making ContinuousIntervalTree sklearn compliant as part of the move from contrib #2951. 

Shame it didn't break any of the tests for that PR, will take a look.

Resolving patch:
diff --git a/sktime/classification/interval_based/_drcif.py b/sktime/classification/interval_based/_drcif.py
--- a/sktime/classification/interval_based/_drcif.py
+++ b/sktime/classification/interval_based/_drcif.py
@@ -571,6 +571,11 @@ def _train_probas_for_estimator(self, y, idx):
 
         indices = range(self.n_instances_)
         subsample = rng.choice(self.n_instances_, size=self.n_instances_)
+
+        # subsample must have at least 2 unique classes
+        while len(np.unique(y[subsample])) == 1:
+            subsample = rng.choice(self.n_instances_, size=self.n_instances_)
+
         oob = [n for n in indices if n not in subsample]
 
         results = np.zeros((self.n_instances_, self.n_classes_))
diff --git a/sktime/classification/kernel_based/_arsenal.py b/sktime/classification/kernel_based/_arsenal.py
--- a/sktime/classification/kernel_based/_arsenal.py
+++ b/sktime/classification/kernel_based/_arsenal.py
@@ -390,6 +390,11 @@ def _train_probas_for_estimator(self, y, idx):
 
         indices = range(self.n_instances_)
         subsample = rng.choice(self.n_instances_, size=self.n_instances_)
+
+        # subsample must have at least 2 unique classes
+        while len(np.unique(y[subsample])) == 1:
+            subsample = rng.choice(self.n_instances_, size=self.n_instances_)
+
         oob = [n for n in indices if n not in subsample]
 
         results = np.zeros((self.n_instances_, self.n_classes_))
diff --git a/sktime/classification/sklearn/_rotation_forest.py b/sktime/classification/sklearn/_rotation_forest.py
--- a/sktime/classification/sklearn/_rotation_forest.py
+++ b/sktime/classification/sklearn/_rotation_forest.py
@@ -451,6 +451,11 @@ def _train_probas_for_estimator(self, y, idx):
 
         indices = range(self.n_instances_)
         subsample = rng.choice(self.n_instances_, size=self.n_instances_)
+
+        # subsample must have at least 2 unique classes
+        while len(np.unique(y[subsample])) == 1:
+            subsample = rng.choice(self.n_instances_, size=self.n_instances_)
+
         oob = [n for n in indices if n not in subsample]
 
         results = np.zeros((self.n_instances_, self.n_classes_))


Test patch: