Repository: nilearn/nilearn

Issue:
[MAINT] Resolve sklearn 1.4.dev0 related failure of decoder test
The failure still seen here in the numpy2 job: [test_decoder_binary_classification_with_masker_object](https://github.com/nilearn/nilearn/actions/runs/7115012784/job/19370222715?pr=4130)
is actually related to an inconsistency between nilearn and scikit-learn dev version from their nightly releases which we have to use to test against numpy v2.0. All the numpy deprecations so far are dealt with in this PR so I'd rather merge this and deal with the sklearn deprecation separately

_Originally posted by @ymzayek in https://github.com/nilearn/nilearn/issues/4130#issuecomment-1842916077_
            


Discussion:
OK. Do you know whether the sklearn team is working on it ?
Not sure it's a bug from them necessarily. They are now using a private attribute called `_estimator_type` to do some checks when `score` is called from an estimator. This attribute can have the values of `"regressor"` and `"classifier"`. Our Decoder class objects should have type `"classifier"` but since they inherit from `_BaseDecoder` which inherits from sklearn's `LinearRegression` they actually have `_estimator_type="regressor"`. 
@glemaitre wondering if you have any input on the change on the scikit-learn side?
We change the internal scoring with a refactoring. Concretely, we check the `_estimator_type` to ensure that you call a meaningful response method (i.e. `predict`, `predict_proba`, `decision_function`). If you decoded is a `regressor` then, we only accept to call `predict` because we don't have other use case in scikit-learn.

This said, I need to understand if the failure would be a real regression or more a design issue. One thing that might potentially make sense in scikit-learn would be to be slightly more leninent and change the `else` statement with `elif is_regressor(self)` and do not make any check for other models.

I am under the impression that this will not helped in your case however. Could you tell me what is the reason for calling `decision_function` on a regressor?
Looking at the decoder class, I would overwrite `_estimator_type` to `classifier` and for `DecoderRegressor` should use `regressor`.

Perfectly, I assume that `_estimator_type` should be delegated to the `_estimator_type` from the `estimator` attribute using a property. But the fact that you inherit from `LinearRegression` will not allow you to do that.

I don't know enough the code right now to know what is the reason to inherate from `LinearRegression`. I assume that you want to take profit from some functionalities and I am wondering if there not something better to do here. But I need to go through the code to be sure.
I looked a bit closer, I am not sure that this is necessary to inherit indeed. `LinearRegression` only define a `fit` and `__init__` that you are redefining. Then it inherate itself from `LinearModel` but it seems that you are not using the private method define also.

So I am wondering if it would be better to remove the inheritance from `LinearRegression` and instead add:

```python
class Decoder(ClassifierMixin, _BaseDecoder):
    ...

class DecoderRegressor(MultiOutputMixin, RegressorMixin, _BaseDecoder):
    ...
```

On thing weird is to define a public `def decision_function` in `_BaseDecoder` because it is the same than `predict` for regressor. Typically, in scikit-learn we would define `_decision_function` that can be used both in the `Decoder` and `DecoderRegressor` but only the `Decoder` will define `decision_function`.

However, going in this direction will break some backward compatibility but it will be compatible with the scikit-learn rules.

@glemaitre thanks for the feedback! I'm not sure about the original design intention of some of these choices so I'll have to do some tests to see if we can change the inheritance and I'll check on the defined `decision_function` in our code as well.
You can ping on your PR. I don't mind reviewing if you want.

Resolving patch:
diff --git a/nilearn/decoding/decoder.py b/nilearn/decoding/decoder.py
--- a/nilearn/decoding/decoder.py
+++ b/nilearn/decoding/decoder.py
@@ -22,10 +22,10 @@
 import numpy as np
 from joblib import Parallel, delayed
 from sklearn import clone
+from sklearn.base import BaseEstimator, MultiOutputMixin
 from sklearn.dummy import DummyClassifier, DummyRegressor
 from sklearn.linear_model import (
     LassoCV,
-    LinearRegression,
     LogisticRegressionCV,
     RidgeClassifierCV,
     RidgeCV,
@@ -444,7 +444,7 @@ def _parallel_fit(
 
 
 @fill_doc
-class _BaseDecoder(LinearRegression, CacheMixin):
+class _BaseDecoder(CacheMixin, BaseEstimator):
     """A wrapper for popular classification/regression strategies in \
     neuroimaging.
 
@@ -836,8 +836,9 @@ def score(self, X, y, *args):
 
         Parameters
         ----------
-        X : {array-like, sparse matrix}, shape = (n_samples, n_features)
-            Samples.
+        X : Niimg-like
+            See :ref:`extracting_data`.
+            Data on which prediction is to be made.
 
         y : array-like
             Target values.
@@ -870,6 +871,9 @@ def decision_function(self, X):
         y_pred: ndarray, shape (n_samples,)
             Predicted class label per sample.
         """
+        # for backwards compatibility
+        if not isinstance(X, np.ndarray):
+            X = self.masker_.transform(X)
         n_features = self.coef_.shape[1]
         if X.shape[1] != n_features:
             raise ValueError(
@@ -889,8 +893,9 @@ def predict(self, X):
 
         Parameters
         ----------
-        X: {array-like, sparse matrix}, shape = (n_samples, n_features)
-            Samples.
+        X: Niimg-like
+            See :ref:`extracting_data`.
+            Data on which prediction is to be made.
 
         Returns
         -------
@@ -902,8 +907,7 @@ class would be predicted.
         check_is_fitted(self, "coef_")
         check_is_fitted(self, "masker_")
 
-        X = self.masker_.transform(X)
-        n_samples = X.shape[0]
+        n_samples = X.shape[-1]
 
         # Prediction for dummy estimator is different from others as there is
         # no fitted coefficient
@@ -1065,6 +1069,9 @@ def _predict_dummy(self, n_samples):
             )
         return scores.ravel() if scores.shape[1] == 1 else scores
 
+    def _more_tags(self):
+        return {"require_y": True}
+
 
 @fill_doc
 class Decoder(_BaseDecoder):
@@ -1156,6 +1163,8 @@ class Decoder(_BaseDecoder):
     nilearn.decoding.SpaceNetClassifier: Graph-Net and TV-L1 priors/penalties
     """
 
+    _estimator_type = "classifier"
+
     def __init__(
         self,
         estimator="svc",
@@ -1201,7 +1210,7 @@ def __init__(
 
 
 @fill_doc
-class DecoderRegressor(_BaseDecoder):
+class DecoderRegressor(MultiOutputMixin, _BaseDecoder):
     """A wrapper for popular regression strategies in neuroimaging.
 
     The `DecoderRegressor` object supports regression methods.
@@ -1293,6 +1302,8 @@ class DecoderRegressor(_BaseDecoder):
     nilearn.decoding.SpaceNetClassifier: Graph-Net and TV-L1 priors/penalties
     """
 
+    _estimator_type = "regressor"
+
     def __init__(
         self,
         estimator="svr",


Test patch:
diff --git a/nilearn/decoding/tests/test_decoder.py b/nilearn/decoding/tests/test_decoder.py
--- a/nilearn/decoding/tests/test_decoder.py
+++ b/nilearn/decoding/tests/test_decoder.py
@@ -507,10 +507,6 @@ def test_decoder_param_grid_sequence(binary_classification_data):
             assert len(param_list) == n_cv_folds
 
 
-# TODO: remove xfail when resolving nilearn/nilearn#4132
-@pytest.mark.xfail(
-    sklearn.__version__ >= "1.4", reason="Fails on scikit-learn >= 1.4"
-)
 def test_decoder_binary_classification_with_masker_object(
     binary_classification_data,
 ):
@@ -1018,3 +1014,26 @@ def test_decoder_multiclass_warnings(multiclass_data):
             cv=1,
         )
         model.fit(X, y)
+
+
+def test_decoder_tags_classification():
+    """Check value returned by _more_tags."""
+    model = Decoder()
+    assert model._more_tags()["require_y"] is True
+
+
+def test_decoder_tags_regression():
+    """Check value returned by _more_tags."""
+    model = DecoderRegressor()
+    assert model._more_tags()["multioutput"] is True
+
+
+def test_decoder_decision_function(binary_classification_data):
+    """Test decision_function with ndarray. Test for backward compatibility."""
+    X, y, mask = binary_classification_data
+
+    model = Decoder(mask=mask)
+    model.fit(X, y)
+    X = model.masker_.transform(X)
+    assert X.shape[1] == model.coef_.shape[1]
+    model.decision_function(X)