Repository: scikit-learn/scikit-learn
Pull request: DOC FIx the formula for support_fraction in the covariance estimators
<!--
Thanks for contributing a pull request! Please ensure you have taken a look at
the contribution guidelines: https://github.com/scikit-learn/scikit-learn/blob/main/CONTRIBUTING.md
-->

#### Reference Issues/PRs
<!--
Example: Fixes #1234. See also #3456.
Please use keywords (e.g., Fixes) to create link to the issues or pull requests
you resolved, so that they will automatically be closed when your pull request
is merged. See https://github.com/blog/1506-closing-issues-via-pull-requests
-->
Fixes #26537

#### What does this implement/fix? Explain your changes.
The formula for minimum support_fraction was the formula for n_support
Actual formula for fraction would be divided by n_samples

#### Any other comments?


<!--
Please be aware that we are a loose team of volunteers so patience is
necessary; assistance handling other issues is very welcome. We value
all user contributions, no matter how minor they are. If we are slow to
review, either the pull request needs some benchmarking, tinkering,
convincing, etc. or more likely the reviewers are simply busy. In either
case, we ask for your understanding during the review process.
For more information, see our FAQ on this topic:
http://scikit-learn.org/dev/faq.html#why-is-my-pull-request-not-getting-any-attention.

Thanks for contributing!
-->


Problem statement:
ValueError: The covariance matrix of the support data is equal to 0 - Elliptic Envelope
### Describe the bug

I have been using Elliptic Envelope on a simple time series dataset with default parameters and only setting `contamination` value to `0.5`. However it throws me error like below:

`ValueError: The covariance matrix of the support data is equal to 0, try to increase support_fraction`

Looking at the error, I referred to the documentation and the equation of `support_fraction` does not seem to be clear. It sayd ->  `[n_sample + n_features + 1] / 2`. `Range is (0, 1)`. The value of `n_sample` and `n_features` can never be such that it will fall within 0 and 1. Could you please help in understanding and how this issue can be resolved?

### Steps/Code to Reproduce

```py
model = EllipticEnvelope(contamination=0.5)
model.fit(X)
```

### Expected Results

No error is thrown

### Actual Results

```
File /opt/anaconda3/envs/benchmark/lib/python3.9/site-packages/sklearn/covariance/_elliptic_envelope.py:145, in EllipticEnvelope.fit(self, X, y)
    134 def fit(self, X, y=None):
    135     """Fit the EllipticEnvelope model.
    136 
    137     Parameters
   (...)
    143         Not used, present for API consistency by convention.
    144     """
--> 145     super().fit(X)
    146     self.offset_ = np.percentile(-self.dist_, 100. * self.contamination)
    147     return self

File /opt/anaconda3/envs/benchmark/lib/python3.9/site-packages/sklearn/covariance/_robust_covariance.py:668, in MinCovDet.fit(self, X, y)
    666 self.dist_ = raw_dist
    667 # obtain consistency at normal models
--> 668 self.correct_covariance(X)
    669 # re-weight estimator
    670 self.reweight_covariance(X)

File /opt/anaconda3/envs/benchmark/lib/python3.9/site-packages/sklearn/covariance/_robust_covariance.py:705, in MinCovDet.correct_covariance(self, data)
    703 n_support = np.sum(self.support_)
    704 if n_support < n_samples and np.allclose(self.raw_covariance_, 0):
--> 705     raise ValueError('The covariance matrix of the support data '
    706                      'is equal to 0, try to increase support_fraction')
    707 correction = np.median(self.dist_) / chi2(data.shape[1]).isf(0.5)
    708 covariance_corrected = self.raw_covariance_ * correction

ValueError: The covariance matrix of the support data is equal to 0, try to increase support_fraction
```

### Versions

```shell
0.24.1
```

Hints from issue discussion:
Thank you for reporting this. Could you please extend your "Steps/Code to Reproduce" with a (minimal) `X` that reproduces the problem?
Here are the steps to reproduce it:
```
X = ([[1.04725631],
       [1.84803131],
       [1.29789207],
       [1.59315107],
       [0.79657553],
       [0.        ],
       [0.        ],
       [0.        ],
       [0.        ],
       [0.89970052],
       [0.        ],
       [0.        ],
       [0.        ],
       [0.        ],
       [0.        ],
       [0.        ],
       [0.        ],
       [0.        ],
       [0.        ],
       [0.        ]])
model = EllipticEnvelope(contamination=0.0003)
model.fit(X)
```
@betatim - Would you need anything from my side? Just wanted to follow up and see if you have any interim solution for this.
On further investigation,

In the fast_mcd algorithm if n_features == 1, The raw covariance comes to zero

```python
import numpy as np
from scipy import linalg

X = ([[1.04725631],
       [1.84803131],
       [1.29789207],
       [1.59315107],
       [0.79657553],
       [0.        ],
       [0.        ],
       [0.        ],
       [0.        ],
       [0.89970052],
       [0.        ],
       [0.        ],
       [0.        ],
       [0.        ],
       [0.        ],
       [0.        ],
       [0.        ],
       [0.        ],
       [0.        ],
       [0.        ]])

X = np.asarray(X)
n_samples, n_features = X.shape
n_support = 11

X_sorted = np.sort(np.ravel(X))

diff = X_sorted[n_support:] - X_sorted[: (n_samples - n_support)]
halves_start = np.where(diff == np.min(diff))[0]
# take the middle points' mean to get the robust location estimate
location = (
    0.5
    * (X_sorted[n_support + halves_start] + X_sorted[halves_start]).mean()
)
support = np.zeros(n_samples, dtype=bool)
X_centered = X - location
support[np.argsort(np.abs(X_centered), 0)[:n_support]] = True
covariance = np.asarray([[np.var(X[support])]])
location = np.array([location])
# get precision matrix in an optimized way
precision = linalg.pinvh(covariance)
dist = (np.dot(X_centered, precision) * (X_centered)).sum(axis=1)
```

Now this is due to the support fraction because fast_mcd uses a proportion of points, so since you have not mentioned the support fraction and a lot of values are zero, the cov is zero, which will not allow it to proceed.

With a little hit and trial support fraction that works minimum is 0.7, while the min support fraction calculated is 0.55

`        min_support_fraction = int(np.ceil(0.5 * (n_samples + n_features + 1)))/n_samples
`
The only problem is in the missing documentation where it mentions

support_fraction : float, default=None
        The proportion of points to be included in the support of the raw
        MCD estimate. Default is `None`, which implies that the minimum
        value of `support_fraction` will be used within the algorithm:
        `(n_sample + n_features + 1) / 2`. This parameter must be in the
        range (0, 1).

This is the formula for n_support, not the support_fraction for support_fraction the formula will be,

support_fraction = (n_samples + n_features + 1) / 2*n_samples

I am adding a PR for the same doc change.

Commit message: Merge branch 'main' into Doc_support_fraction

Patch to review:
diff --git a/sklearn/covariance/_elliptic_envelope.py b/sklearn/covariance/_elliptic_envelope.py
index fe109dddd5303..2bf81cd78a22a 100644
--- a/sklearn/covariance/_elliptic_envelope.py
+++ b/sklearn/covariance/_elliptic_envelope.py
@@ -35,7 +35,7 @@ class EllipticEnvelope(OutlierMixin, MinCovDet):
     support_fraction : float, default=None
         The proportion of points to be included in the support of the raw
         MCD estimate. If None, the minimum value of support_fraction will
-        be used within the algorithm: `[n_sample + n_features + 1] / 2`.
+        be used within the algorithm: `[n_samples + n_features + 1] / 2*n_samples`.
         Range is (0, 1).
 
     contamination : float, default=0.1
diff --git a/sklearn/covariance/_robust_covariance.py b/sklearn/covariance/_robust_covariance.py
index a6b32e50a6c1f..5ba8c663ec387 100644
--- a/sklearn/covariance/_robust_covariance.py
+++ b/sklearn/covariance/_robust_covariance.py
@@ -373,7 +373,7 @@ def fast_mcd(
         The proportion of points to be included in the support of the raw
         MCD estimate. Default is `None`, which implies that the minimum
         value of `support_fraction` will be used within the algorithm:
-        `(n_sample + n_features + 1) / 2`. This parameter must be in the
+        `(n_samples + n_features + 1) / 2*n_samples`. This parameter must be in the
         range (0, 1).
 
     cov_computation_method : callable, \
@@ -607,7 +607,7 @@ class MinCovDet(EmpiricalCovariance):
         The proportion of points to be included in the support of the raw
         MCD estimate. Default is None, which implies that the minimum
         value of support_fraction will be used within the algorithm:
-        `(n_sample + n_features + 1) / 2`. The parameter must be in the range
+        `(n_samples + n_features + 1) / 2*n_samples`. The parameter must be in the range
         (0, 1].
 
     random_state : int, RandomState instance or None, default=None


Code review comments:
diff_hunk: @@ -373,7 +373,7 @@ def fast_mcd(
         The proportion of points to be included in the support of the raw
         MCD estimate. Default is `None`, which implies that the minimum
         value of `support_fraction` will be used within the algorithm:
-        `(n_sample + n_features + 1) / 2`. This parameter must be in the
+        `(n_samples + n_features + 1) / 2*n_samples`. This parameter must be in the
path: sklearn/covariance/_robust_covariance.py
text: @user1:

```suggestion
        `(n_samples + n_features + 1) / 2 * n_samples`. This parameter must be in the
```

diff_hunk: @@ -607,7 +607,7 @@ class MinCovDet(EmpiricalCovariance):
         The proportion of points to be included in the support of the raw
         MCD estimate. Default is None, which implies that the minimum
         value of support_fraction will be used within the algorithm:
-        `(n_sample + n_features + 1) / 2`. The parameter must be in the range
+        `(n_samples + n_features + 1) / 2*n_samples`. The parameter must be in the range
path: sklearn/covariance/_robust_covariance.py
text: @user1:

```suggestion
        `(n_samples + n_features + 1) / 2 * n_samples`. The parameter must be in the range
```

diff_hunk: @@ -35,7 +35,7 @@ class EllipticEnvelope(OutlierMixin, MinCovDet):
     support_fraction : float, default=None
         The proportion of points to be included in the support of the raw
         MCD estimate. If None, the minimum value of support_fraction will
-        be used within the algorithm: `[n_sample + n_features + 1] / 2`.
+        be used within the algorithm: `[n_samples + n_features + 1] / 2*n_samples`.
path: sklearn/covariance/_elliptic_envelope.py
text: @user1:

```suggestion
        be used within the algorithm: `(n_samples + n_features + 1) / 2 * n_samples`.
```