Repository: scikit-learn/scikit-learn
Pull request: FIX IsolationForest accepts sparse matrix with float value contamination
closes #27626 

This PR makes sure to convert to CSR the validated CSC matrix when contamination is a floating number. We still don't want to call `score_samples` because it will cost an additional validation, more expensive than the expected `X.asformat("csr")`.

Problem statement:
Isolation Forest Bug with Sparse Matrix and Contamination as Float
### Describe the bug

### Environment:

```
Python 3.11 and 3.8
Scikit-learn library 1.3.1
Isolation Forest algorithm
Sparse matrix input (tested csr and csc)
Contamination parameter set as a float
```

### Bug Summary:

When using the Isolation Forest algorithm from the Scikit-learn library with a sparse matrix as input data and explicitly setting the contamination parameter as a float (rather than 'auto'), a bug is encountered that stops the algorithm during the fit operation.

### Expected Behavior:

The Isolation Forest algorithm should run without errors and produce anomaly scores as expected, with the contamination level set according to the specified float value.

### Actual Behavior:

When the contamination parameter is set as a float value, the Isolation Forest algorithm may encounter issues during model fitting or produce unexpected results. The bug might lead to incorrect anomaly detection and, in some cases, throw an error.

### Additional Information:

The bug is not observed when the contamination parameter is set to 'auto' or when dense matrices are used.
The bug's impact may vary depending on the specific version of Scikit-learn being used.
This bug may be related to the handling of sparse data and the contamination parameter in the Isolation Forest implementation.

### Steps/Code to Reproduce

### Not working with contamination as a float
```python
from sklearn.datasets import make_classification
from scipy.sparse import csc_matrix
from sklearn.ensemble import IsolationForest

X, y = make_classification(n_samples=50000, n_features=1000)
X = csc_matrix(X)
X.sort_indices()
IsolationForest(n_estimators=10, max_samples=256, n_jobs=1, contamination=0.1).fit(X)
```

### Properly working with contamination = 'auto'/not specified
```python
from sklearn.datasets import make_classification
from scipy.sparse import csc_matrix
from sklearn.ensemble import IsolationForest

X, y = make_classification(n_samples=50000, n_features=1000)
X = csc_matrix(X)
X.sort_indices()
IsolationForest(n_estimators=10, max_samples=256, n_jobs=1, contamination='auto').fit(X)
```

### Expected Results

The Isolation Forest algorithm should run without errors and produce anomaly scores as expected, with the contamination level set according to the specified float value.

### Actual Results

```pytb
Building estimator 48 of 50 for this parallel run (total 100)...
Building estimator 49 of 50 for this parallel run (total 100)...
Building estimator 50 of 50 for this parallel run (total 100)...
[Parallel(n_jobs=2)]: Done   2 out of   2 | elapsed:    7.0s finished
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Input In [6], in <cell line: 11>()
      8 #Train ML Modell
      9 logging.info('Starting ML Modell Training')
---> 11 distributionforest = algorithm.fit(X)
     13 logging.info('Finished ML Modell Training')
     15 #Store ML Model

File ~/.local/lib/python3.8/site-packages/sklearn/base.py:1152, in _fit_context.<locals>.decorator.<locals>.wrapper(estimator, *args, **kwargs)
   1145     estimator._validate_params()
   1147 with config_context(
   1148     skip_parameter_validation=(
   1149         prefer_skip_nested_validation or global_skip_validation
   1150     )
   1151 ):
-> 1152     return fit_method(estimator, *args, **kwargs)

File ~/.local/lib/python3.8/site-packages/sklearn/ensemble/_iforest.py:349, in IsolationForest.fit(self, X, y, sample_weight)
    344     return self
    346 # Else, define offset_ wrt contamination parameter
    347 # To avoid performing input validation a second time we call
    348 # _score_samples rather than score_samples
--> 349 self.offset_ = np.percentile(self._score_samples(X), 100.0 * self.contamination)
    351 return self

File ~/.local/lib/python3.8/site-packages/sklearn/ensemble/_iforest.py:447, in IsolationForest._score_samples(self, X)
    444 check_is_fitted(self)
    446 # Take the opposite of the scores as bigger is better (here less abnormal)
--> 447 return -self._compute_chunked_score_samples(X)

File ~/.local/lib/python3.8/site-packages/sklearn/ensemble/_iforest.py:477, in IsolationForest._compute_chunked_score_samples(self, X)
    473 scores = np.zeros(n_samples, order="f")
    475 for sl in slices:
    476     # compute score on the slices of test samples:
--> 477     scores[sl] = self._compute_score_samples(X[sl], subsample_features)
    479 return scores

File ~/.local/lib/python3.8/site-packages/sklearn/ensemble/_iforest.py:504, in IsolationForest._compute_score_samples(self, X, subsample_features)
    499 for tree_idx, (tree, features) in enumerate(
    500     zip(self.estimators_, self.estimators_features_)
    501 ):
    502     X_subset = X[:, features] if subsample_features else X
--> 504     leaves_index = tree.apply(X_subset, check_input=False)
    506     depths += (
    507         self._decision_path_lengths[tree_idx][leaves_index]
    508         + self._average_path_length_per_tree[tree_idx][leaves_index]
    509         - 1.0
    510     )
    511 denominator = len(self.estimators_) * average_path_length_max_samples

File ~/.local/lib/python3.8/site-packages/sklearn/tree/_classes.py:553, in BaseDecisionTree.apply(self, X, check_input)
    551 check_is_fitted(self)
    552 X = self._validate_X_predict(X, check_input)
--> 553 return self.tree_.apply(X)

File sklearn/tree/_tree.pyx:839, in sklearn.tree._tree.Tree.apply()

File sklearn/tree/_tree.pyx:842, in sklearn.tree._tree.Tree.apply()

File sklearn/tree/_tree.pyx:895, in sklearn.tree._tree.Tree._apply_sparse_csr()

ValueError: X should be in csr_matrix format, got <class 'scipy.sparse.csc.csc_matrix'>
```

### Versions

```shell
System:
    python: 3.8.10 | packaged by conda-forge | (default, May 11 2021, 07:13:52)  [GCC 9.3.0]
executable: /opt/conda/bin/python
   machine: Linux-4.18.0-372.73.1.el8_6.x86_64-x86_64-with-glibc2.10

Python dependencies:
      sklearn: 1.3.1
          pip: 23.2.1
   setuptools: 59.8.0
        numpy: 1.20.3
        scipy: 1.6.0
       Cython: 0.29.21
       pandas: 2.0.3
   matplotlib: 3.3.4
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /opt/conda/lib/libopenblasp-r0.3.21.so
        version: 0.3.21
threading_layer: pthreads
   architecture: SkylakeX
    num_threads: 16

       user_api: openmp
   internal_api: openmp
         prefix: libgomp
       filepath: /home/jovyan/.local/lib/python3.8/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
    num_threads: 16
```

Hints from issue discussion:
I can reproduce. The thing is that we call `_score_samples` to avoid a second input validation. However, the first validation make sure that we have a CSC matrix while `_score_samples` expect a CSR matrix. We should therefore make the conversion before calling the private function since we still want to avoid all extra validation.
/take

Commit message: add changelog

Patch to review:
diff --git a/doc/whats_new/v1.4.rst b/doc/whats_new/v1.4.rst
index a947dfe225b04..896b018c89620 100644
--- a/doc/whats_new/v1.4.rst
+++ b/doc/whats_new/v1.4.rst
@@ -276,6 +276,10 @@ Changelog
   :class:`ensemble.GradientBoostingRegressor` when trained on sparse data.
   :pr:`26957` by `Thomas Fan`_.
 
+- |Fix| Fixes :class:`ensemble.IsolationForest` when the input is a sparse matrix and
+  `contamination` is set to a float value.
+  :pr:`27645` by :user:`Guillaume Lemaitre <glemaitre>`.
+
 - |API| In :class:`ensemble.AdaBoostClassifier`, the `algorithm` argument `SAMME.R` was
   deprecated and will be removed in 1.6. :pr:`26830` by :user:`Stefanie Senger
   <StefanieSenger>`.
diff --git a/sklearn/ensemble/_iforest.py b/sklearn/ensemble/_iforest.py
index 16d5215b7e0a8..eda98bc06371c 100644
--- a/sklearn/ensemble/_iforest.py
+++ b/sklearn/ensemble/_iforest.py
@@ -348,7 +348,9 @@ def fit(self, X, y=None, sample_weight=None):
 
         # Else, define offset_ wrt contamination parameter
         # To avoid performing input validation a second time we call
-        # _score_samples rather than score_samples
+        # X to a CSR matrix while it was validated earlier as a CSC matrix.
+        if issparse(X):
+            X = X.tocsr()
         self.offset_ = np.percentile(self._score_samples(X), 100.0 * self.contamination)
 
         return self
@@ -433,7 +435,7 @@ def score_samples(self, X):
             The lower, the more abnormal.
         """
         # Check data
-        X = self._validate_data(X, accept_sparse="csr", dtype=np.float32, reset=False)
+        X = self._validate_data(X, accept_sparse="csr", dtype=tree_dtype, reset=False)
 
         return self._score_samples(X)
 
diff --git a/sklearn/ensemble/tests/test_iforest.py b/sklearn/ensemble/tests/test_iforest.py
index 9abe93bd8d115..6736be7023e66 100644
--- a/sklearn/ensemble/tests/test_iforest.py
+++ b/sklearn/ensemble/tests/test_iforest.py
@@ -356,3 +356,23 @@ def test_iforest_preserve_feature_names():
     with warnings.catch_warnings():
         warnings.simplefilter("error", UserWarning)
         model.fit(X)
+
+
+@pytest.mark.parametrize("sparse_container", CSC_CONTAINERS + CSR_CONTAINERS)
+def test_iforest_sparse_input_float_contamination(sparse_container):
+    """Check that `IsolationForest` accepts sparse matrix input and float value for
+    contamination.
+
+    Non-regression test for:
+    https://github.com/scikit-learn/scikit-learn/issues/27626
+    """
+    X, _ = make_classification(n_samples=50, n_features=4, random_state=0)
+    X = sparse_container(X)
+    X.sort_indices()
+    contamination = 0.1
+    iforest = IsolationForest(
+        n_estimators=5, contamination=contamination, random_state=0
+    ).fit(X)
+
+    X_decision = iforest.decision_function(X)
+    assert (X_decision < 0).sum() / X.shape[0] == pytest.approx(contamination)


Code review comments:
diff_hunk: @@ -348,7 +348,9 @@ def fit(self, X, y=None, sample_weight=None):
 
         # Else, define offset_ wrt contamination parameter
         # To avoid performing input validation a second time we call
-        # _score_samples rather than score_samples
+        # X to a CSR matrix while it was validated earlier as a CSC matrix.
path: sklearn/ensemble/_iforest.py
text: @user1:
I have a hard time parsing this. Perhaps?

```
# To avoid performing input validation a second time we call
# _score_samples rather than score_samples.
# _score_samples expects a CSR matrix, so we convert if necessary.
```