OneVsRestClassifier when all estimators predict a sample belongs to the other classes

### Describe the bug

Hello, I stumbled upon quite a funny case by accident.

In OneVsRestClassifier, each classifier predicts whether a sample belongs to a specific class, or to any of the other class. For instance, if you have 3 classes, you will have 3  binary classifiers:

- the first one says if the sample belongs to class 1 or to one of the two other classes.
- the second one says if the sample belongs to class 2 or to one of the two other classes.
- the third one says if the sample belongs to class 3 or to one of the two other classes.

However, it creates an edge case where all of the estimators of OneVsRestClassifier predict a specific sample belongs to the other classes:

- calling .predict() will mark the sample as belonging to the last class (which is of course wrong since in our example, the 3rd estimator said the sample did not belong in that class).
- calling .predict_proba() will return NaNs values.

### Steps/Code to Reproduce

```python
from sklearn.multiclass import OneVsRestClassifier
from sklearn.base import BaseEstimator, ClassifierMixin
from sklearn.datasets import make_classification

import numpy as np


class MyDumbDumbBinaryClassifier(BaseEstimator, ClassifierMixin):
    def fit(self, X, y):
        self.classes_ = set(y)
        return self

    def predict(self, X):
        return np.array([0 for _ in range(len(X))])

    def predict_proba(self, X):
        ones = np.ones((len(X), len(self.classes_)))
        # the proba of being the positive class is always 0
        ones[:, 1] = 0

        return ones
    

clf = OneVsRestClassifier(MyDumbDumbBinaryClassifier())

X, y = make_classification(n_classes=3, n_informative=5)
clf.fit(X, y)
clf.predict_proba(X)
```

### Expected Results

I guess .predict() should return NaNs, and .predict_proba() should return a vector of 0s for that sample.

### Actual Results

```python
>>> clf.predict(X)

array([2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
       2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
       2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
       2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
       2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2])
```

```python
>>> clf.predict_proba(X)

array([[nan, nan, nan],
       [nan, nan, nan],
       [nan, nan, nan],
       [nan, nan, nan],
       ...
       [nan, nan, nan],
       [nan, nan, nan],
       [nan, nan, nan]])
```


### Versions

```shell
System:
    python: 3.13.2 (v3.13.2:4f8bb3947cf, Feb  4 2025, 11:51:10) [Clang 15.0.0 (clang-1500.3.9.4)]
executable: /Library/Frameworks/Python.framework/Versions/3.13/bin/python3
   machine: macOS-14.4.1-arm64-arm-64bit-Mach-O

Python dependencies:
      sklearn: 1.6.1
          pip: 24.3.1
   setuptools: None
        numpy: 2.2.3
        scipy: 1.15.2
       Cython: None
       pandas: 2.2.3
   matplotlib: 3.10.1
       joblib: 1.4.2
threadpoolctl: 3.6.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libomp
       filepath: /Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/site-packages/sklearn/.dylibs/libomp.dylib
        version: None
```

COMMENT:
> maybe there should be an explicit option (for which finding a name will be the hardest part) in the OneVsRestClassifier ? With this option, we could enable different behaviours for samples that belong to none of the class.

Coming back to this question. Sorry for the delay.
I don't think that we should allow different behaviors. We actually have a defined behavior written in the glossary that covers this case:
```
multiclass classification

    A 2-dimensional array, where the row-wise arg-maximum is the predicted class.
```
So the desired behavior is `ovr.predict(X) == np.argmax(ovr.decision_function(X))`, (similarly with `predict_proba` instead of `decision_function`). This is now the case after https://github.com/scikit-learn/scikit-learn/pull/15504. With that in mind I closing this issue.

Labels: Bug