Talip7/scikit-learn-issues-multilabel huggingface dataset:
sklearn.preprocessing.StandardScaler() needs a degrees of freedom tunable parameter for scale_

### Describe the workflow you want to enable

I'm scaling a dataset using the standard procedure.

```
from sklearn.preprocessing import StandardScaler

sc = StandardScaler()
sc.fit(data)
sc.transform(data)
```

I'm comparing the results I get with the results from the R function `scale()`. They are subtly different. Tracking down the difference, it turns out sklearn uses a different definition for the standard deviation, one that differs from R in the degrees of freedom assumed. sklearn assumes ddof=0, while R assumes ddof=1.

https://stackoverflow.com/questions/27296387/difference-between-r-scale-and-sklearn-preprocessing-scale/72758497

I do not wish to debate which approach is "more correct". I am fine with sklearn continuing with the current approach as the default. I only want to suggest that, for those who think the slight DoF correction is necessary, `StandardScaler()` should provide an option to tune the degrees of freedom, either in the initial `StandardScaler()` call, or later in the `fit()` call, or wherever it is deemed more appropriate.

### Describe your proposed solution

Basically, leave the default as-is. Introduce a new parameter, let's call it `ddof`, with a default value of 0, but allow the user to specify a different value for it if they need it.

Or have a boolean switch, by default False, which when True enables the Bessel correction, which is the same as using N-1 instead of N for the standard deviation.

I do not know enough about the sklearn internals to judge which solution is better.

### Describe alternatives you've considered, if relevant

This workaround forces `StandardScaler()` to use ddof=1, but it's an extra step and it computes the STD twice. With large datasets, that would slow down the computation and it's wasteful.

```
from sklearn.preprocessing import StandardScaler
import numpy as np

sc = StandardScaler()
sc.fit(data)
sc.scale_ = np.std(data, axis=0, ddof=1).to_list()
sc.transform(data)
```

### Additional context

https://en.wikipedia.org/wiki/Bessel%27s_correction

COMMENT:
`StandardScaler` is using a different [approach](https://www.sci-hub.ru/10.1080/00031305.1983.10483115) to calculate mean and variance to be compatible with batched or streamed data. It's not using the numpy implementation. I don't know if you can simply add the Bessel Correction to the used algorithm.

Labels: New Feature, module:preprocessing, Needs Decision - Include Feature
sklearn.preprocessing.StandardScaler() needs a degrees of freedom tunable parameter for scale_

### Describe the workflow you want to enable

I'm scaling a dataset using the standard procedure.

```
from sklearn.preprocessing import StandardScaler

sc = StandardScaler()
sc.fit(data)
sc.transform(data)
```

I'm comparing the results I get with the results from the R function `scale()`. They are subtly different. Tracking down the difference, it turns out sklearn uses a different definition for the standard deviation, one that differs from R in the degrees of freedom assumed. sklearn assumes ddof=0, while R assumes ddof=1.

https://stackoverflow.com/questions/27296387/difference-between-r-scale-and-sklearn-preprocessing-scale/72758497

I do not wish to debate which approach is "more correct". I am fine with sklearn continuing with the current approach as the default. I only want to suggest that, for those who think the slight DoF correction is necessary, `StandardScaler()` should provide an option to tune the degrees of freedom, either in the initial `StandardScaler()` call, or later in the `fit()` call, or wherever it is deemed more appropriate.

### Describe your proposed solution

Basically, leave the default as-is. Introduce a new parameter, let's call it `ddof`, with a default value of 0, but allow the user to specify a different value for it if they need it.

Or have a boolean switch, by default False, which when True enables the Bessel correction, which is the same as using N-1 instead of N for the standard deviation.

I do not know enough about the sklearn internals to judge which solution is better.

### Describe alternatives you've considered, if relevant

This workaround forces `StandardScaler()` to use ddof=1, but it's an extra step and it computes the STD twice. With large datasets, that would slow down the computation and it's wasteful.

```
from sklearn.preprocessing import StandardScaler
import numpy as np

sc = StandardScaler()
sc.fit(data)
sc.scale_ = np.std(data, axis=0, ddof=1).to_list()
sc.transform(data)
```

### Additional context

https://en.wikipedia.org/wiki/Bessel%27s_correction

COMMENT:
I took a deeper look into it and it actually is possible to add the Bessel Correction. I did a quick and dirty solution for the non sparse case.
I'm going to make a PR with my suggestion (for the non sparse case).

Labels: New Feature, module:preprocessing, Needs Decision - Include Feature
sklearn.preprocessing.StandardScaler() needs a degrees of freedom tunable parameter for scale_

### Describe the workflow you want to enable

I'm scaling a dataset using the standard procedure.

```
from sklearn.preprocessing import StandardScaler

sc = StandardScaler()
sc.fit(data)
sc.transform(data)
```

I'm comparing the results I get with the results from the R function `scale()`. They are subtly different. Tracking down the difference, it turns out sklearn uses a different definition for the standard deviation, one that differs from R in the degrees of freedom assumed. sklearn assumes ddof=0, while R assumes ddof=1.

https://stackoverflow.com/questions/27296387/difference-between-r-scale-and-sklearn-preprocessing-scale/72758497

I do not wish to debate which approach is "more correct". I am fine with sklearn continuing with the current approach as the default. I only want to suggest that, for those who think the slight DoF correction is necessary, `StandardScaler()` should provide an option to tune the degrees of freedom, either in the initial `StandardScaler()` call, or later in the `fit()` call, or wherever it is deemed more appropriate.

### Describe your proposed solution

Basically, leave the default as-is. Introduce a new parameter, let's call it `ddof`, with a default value of 0, but allow the user to specify a different value for it if they need it.

Or have a boolean switch, by default False, which when True enables the Bessel correction, which is the same as using N-1 instead of N for the standard deviation.

I do not know enough about the sklearn internals to judge which solution is better.

### Describe alternatives you've considered, if relevant

This workaround forces `StandardScaler()` to use ddof=1, but it's an extra step and it computes the STD twice. With large datasets, that would slow down the computation and it's wasteful.

```
from sklearn.preprocessing import StandardScaler
import numpy as np

sc = StandardScaler()
sc.fit(data)
sc.scale_ = np.std(data, axis=0, ddof=1).to_list()
sc.transform(data)
```

### Additional context

https://en.wikipedia.org/wiki/Bessel%27s_correction

COMMENT:
@Diadochokinetic that would work. What would also work IMO is a simple boolean flag - whether to enable the Bessel correction or no (with the default no, I'm guessing, for compatibility with existing code).

Anyway, that's just syntax, the important thing is to have that feature in the library. Thank you.

Labels: New Feature, module:preprocessing, Needs Decision - Include Feature
sklearn.preprocessing.StandardScaler() needs a degrees of freedom tunable parameter for scale_

### Describe the workflow you want to enable

I'm scaling a dataset using the standard procedure.

```
from sklearn.preprocessing import StandardScaler

sc = StandardScaler()
sc.fit(data)
sc.transform(data)
```

I'm comparing the results I get with the results from the R function `scale()`. They are subtly different. Tracking down the difference, it turns out sklearn uses a different definition for the standard deviation, one that differs from R in the degrees of freedom assumed. sklearn assumes ddof=0, while R assumes ddof=1.

https://stackoverflow.com/questions/27296387/difference-between-r-scale-and-sklearn-preprocessing-scale/72758497

I do not wish to debate which approach is "more correct". I am fine with sklearn continuing with the current approach as the default. I only want to suggest that, for those who think the slight DoF correction is necessary, `StandardScaler()` should provide an option to tune the degrees of freedom, either in the initial `StandardScaler()` call, or later in the `fit()` call, or wherever it is deemed more appropriate.

### Describe your proposed solution

Basically, leave the default as-is. Introduce a new parameter, let's call it `ddof`, with a default value of 0, but allow the user to specify a different value for it if they need it.

Or have a boolean switch, by default False, which when True enables the Bessel correction, which is the same as using N-1 instead of N for the standard deviation.

I do not know enough about the sklearn internals to judge which solution is better.

### Describe alternatives you've considered, if relevant

This workaround forces `StandardScaler()` to use ddof=1, but it's an extra step and it computes the STD twice. With large datasets, that would slow down the computation and it's wasteful.

```
from sklearn.preprocessing import StandardScaler
import numpy as np

sc = StandardScaler()
sc.fit(data)
sc.scale_ = np.std(data, axis=0, ddof=1).to_list()
sc.transform(data)
```

### Additional context

https://en.wikipedia.org/wiki/Bessel%27s_correction

COMMENT:
@GaelVaroquaux @agramfort do you think this is useful ? Is it relevant in a machine learning context ?

Labels: New Feature, module:preprocessing, Needs Decision - Include Feature
sklearn.preprocessing.StandardScaler() needs a degrees of freedom tunable parameter for scale_

### Describe the workflow you want to enable

I'm scaling a dataset using the standard procedure.

```
from sklearn.preprocessing import StandardScaler

sc = StandardScaler()
sc.fit(data)
sc.transform(data)
```

I'm comparing the results I get with the results from the R function `scale()`. They are subtly different. Tracking down the difference, it turns out sklearn uses a different definition for the standard deviation, one that differs from R in the degrees of freedom assumed. sklearn assumes ddof=0, while R assumes ddof=1.

https://stackoverflow.com/questions/27296387/difference-between-r-scale-and-sklearn-preprocessing-scale/72758497

I do not wish to debate which approach is "more correct". I am fine with sklearn continuing with the current approach as the default. I only want to suggest that, for those who think the slight DoF correction is necessary, `StandardScaler()` should provide an option to tune the degrees of freedom, either in the initial `StandardScaler()` call, or later in the `fit()` call, or wherever it is deemed more appropriate.

### Describe your proposed solution

Basically, leave the default as-is. Introduce a new parameter, let's call it `ddof`, with a default value of 0, but allow the user to specify a different value for it if they need it.

Or have a boolean switch, by default False, which when True enables the Bessel correction, which is the same as using N-1 instead of N for the standard deviation.

I do not know enough about the sklearn internals to judge which solution is better.

### Describe alternatives you've considered, if relevant

This workaround forces `StandardScaler()` to use ddof=1, but it's an extra step and it computes the STD twice. With large datasets, that would slow down the computation and it's wasteful.

```
from sklearn.preprocessing import StandardScaler
import numpy as np

sc = StandardScaler()
sc.fit(data)
sc.scale_ = np.std(data, axis=0, ddof=1).to_list()
sc.transform(data)
```

### Additional context

https://en.wikipedia.org/wiki/Bessel%27s_correction

COMMENT:
I would personally not invest time adding this option. I do not believe it makes a difference in ML problems. We did it so it follows the default of np.std and np.var.
Adding an option means more code, more testing time and its a trade-off of reviewing
time and maintenance.

my 2c

Labels: New Feature, module:preprocessing, Needs Decision - Include Feature
sklearn.preprocessing.StandardScaler() needs a degrees of freedom tunable parameter for scale_

### Describe the workflow you want to enable

I'm scaling a dataset using the standard procedure.

```
from sklearn.preprocessing import StandardScaler

sc = StandardScaler()
sc.fit(data)
sc.transform(data)
```

I'm comparing the results I get with the results from the R function `scale()`. They are subtly different. Tracking down the difference, it turns out sklearn uses a different definition for the standard deviation, one that differs from R in the degrees of freedom assumed. sklearn assumes ddof=0, while R assumes ddof=1.

https://stackoverflow.com/questions/27296387/difference-between-r-scale-and-sklearn-preprocessing-scale/72758497

I do not wish to debate which approach is "more correct". I am fine with sklearn continuing with the current approach as the default. I only want to suggest that, for those who think the slight DoF correction is necessary, `StandardScaler()` should provide an option to tune the degrees of freedom, either in the initial `StandardScaler()` call, or later in the `fit()` call, or wherever it is deemed more appropriate.

### Describe your proposed solution

Basically, leave the default as-is. Introduce a new parameter, let's call it `ddof`, with a default value of 0, but allow the user to specify a different value for it if they need it.

Or have a boolean switch, by default False, which when True enables the Bessel correction, which is the same as using N-1 instead of N for the standard deviation.

I do not know enough about the sklearn internals to judge which solution is better.

### Describe alternatives you've considered, if relevant

This workaround forces `StandardScaler()` to use ddof=1, but it's an extra step and it computes the STD twice. With large datasets, that would slow down the computation and it's wasteful.

```
from sklearn.preprocessing import StandardScaler
import numpy as np

sc = StandardScaler()
sc.fit(data)
sc.scale_ = np.std(data, axis=0, ddof=1).to_list()
sc.transform(data)
```

### Additional context

https://en.wikipedia.org/wiki/Bessel%27s_correction

COMMENT:
> @GaelVaroquaux @agramfort do you think this is useful ?

No, I don't think so. It amounts to changing the features' scale by a
value close to 1 and the same for all features. It will have very little
importance for machine learning.

Labels: New Feature, module:preprocessing, Needs Decision - Include Feature
sklearn.preprocessing.StandardScaler() needs a degrees of freedom tunable parameter for scale_

### Describe the workflow you want to enable

I'm scaling a dataset using the standard procedure.

```
from sklearn.preprocessing import StandardScaler

sc = StandardScaler()
sc.fit(data)
sc.transform(data)
```

I'm comparing the results I get with the results from the R function `scale()`. They are subtly different. Tracking down the difference, it turns out sklearn uses a different definition for the standard deviation, one that differs from R in the degrees of freedom assumed. sklearn assumes ddof=0, while R assumes ddof=1.

https://stackoverflow.com/questions/27296387/difference-between-r-scale-and-sklearn-preprocessing-scale/72758497

I do not wish to debate which approach is "more correct". I am fine with sklearn continuing with the current approach as the default. I only want to suggest that, for those who think the slight DoF correction is necessary, `StandardScaler()` should provide an option to tune the degrees of freedom, either in the initial `StandardScaler()` call, or later in the `fit()` call, or wherever it is deemed more appropriate.

### Describe your proposed solution

Basically, leave the default as-is. Introduce a new parameter, let's call it `ddof`, with a default value of 0, but allow the user to specify a different value for it if they need it.

Or have a boolean switch, by default False, which when True enables the Bessel correction, which is the same as using N-1 instead of N for the standard deviation.

I do not know enough about the sklearn internals to judge which solution is better.

### Describe alternatives you've considered, if relevant

This workaround forces `StandardScaler()` to use ddof=1, but it's an extra step and it computes the STD twice. With large datasets, that would slow down the computation and it's wasteful.

```
from sklearn.preprocessing import StandardScaler
import numpy as np

sc = StandardScaler()
sc.fit(data)
sc.scale_ = np.std(data, axis=0, ddof=1).to_list()
sc.transform(data)
```

### Additional context

https://en.wikipedia.org/wiki/Bessel%27s_correction

COMMENT:
> @GaelVaroquaux @agramfort do you think this is useful ?
> No, I don't think so. It amounts to changing the features' scale by a value close to 1 and the same for all features. It will have very little importance for machine learning.

From a practical ML engineer view, I agree. ddof has very little impact on the results of a ML pipeline. But I do understand that some people want/need that behavior for research purposes. I don't mind spending some time on it, it's a fun task.

Labels: New Feature, module:preprocessing, Needs Decision - Include Feature
sklearn.preprocessing.StandardScaler() needs a degrees of freedom tunable parameter for scale_

### Describe the workflow you want to enable

I'm scaling a dataset using the standard procedure.

```
from sklearn.preprocessing import StandardScaler

sc = StandardScaler()
sc.fit(data)
sc.transform(data)
```

I'm comparing the results I get with the results from the R function `scale()`. They are subtly different. Tracking down the difference, it turns out sklearn uses a different definition for the standard deviation, one that differs from R in the degrees of freedom assumed. sklearn assumes ddof=0, while R assumes ddof=1.

https://stackoverflow.com/questions/27296387/difference-between-r-scale-and-sklearn-preprocessing-scale/72758497

I do not wish to debate which approach is "more correct". I am fine with sklearn continuing with the current approach as the default. I only want to suggest that, for those who think the slight DoF correction is necessary, `StandardScaler()` should provide an option to tune the degrees of freedom, either in the initial `StandardScaler()` call, or later in the `fit()` call, or wherever it is deemed more appropriate.

### Describe your proposed solution

Basically, leave the default as-is. Introduce a new parameter, let's call it `ddof`, with a default value of 0, but allow the user to specify a different value for it if they need it.

Or have a boolean switch, by default False, which when True enables the Bessel correction, which is the same as using N-1 instead of N for the standard deviation.

I do not know enough about the sklearn internals to judge which solution is better.

### Describe alternatives you've considered, if relevant

This workaround forces `StandardScaler()` to use ddof=1, but it's an extra step and it computes the STD twice. With large datasets, that would slow down the computation and it's wasteful.

```
from sklearn.preprocessing import StandardScaler
import numpy as np

sc = StandardScaler()
sc.fit(data)
sc.scale_ = np.std(data, axis=0, ddof=1).to_list()
sc.transform(data)
```

### Additional context

https://en.wikipedia.org/wiki/Bessel%27s_correction

COMMENT:
This sounds like strong arguments to close this issue.
Nevertheless, I want to express my appreciation for opening and discussing this request. Please understand, that the focus of scikit-learn is to be user friendly, production ready and maintainable.

Labels: New Feature, module:preprocessing, Needs Decision - Include Feature
sklearn.preprocessing.StandardScaler() needs a degrees of freedom tunable parameter for scale_

### Describe the workflow you want to enable

I'm scaling a dataset using the standard procedure.

```
from sklearn.preprocessing import StandardScaler

sc = StandardScaler()
sc.fit(data)
sc.transform(data)
```

I'm comparing the results I get with the results from the R function `scale()`. They are subtly different. Tracking down the difference, it turns out sklearn uses a different definition for the standard deviation, one that differs from R in the degrees of freedom assumed. sklearn assumes ddof=0, while R assumes ddof=1.

https://stackoverflow.com/questions/27296387/difference-between-r-scale-and-sklearn-preprocessing-scale/72758497

I do not wish to debate which approach is "more correct". I am fine with sklearn continuing with the current approach as the default. I only want to suggest that, for those who think the slight DoF correction is necessary, `StandardScaler()` should provide an option to tune the degrees of freedom, either in the initial `StandardScaler()` call, or later in the `fit()` call, or wherever it is deemed more appropriate.

### Describe your proposed solution

Basically, leave the default as-is. Introduce a new parameter, let's call it `ddof`, with a default value of 0, but allow the user to specify a different value for it if they need it.

Or have a boolean switch, by default False, which when True enables the Bessel correction, which is the same as using N-1 instead of N for the standard deviation.

I do not know enough about the sklearn internals to judge which solution is better.

### Describe alternatives you've considered, if relevant

This workaround forces `StandardScaler()` to use ddof=1, but it's an extra step and it computes the STD twice. With large datasets, that would slow down the computation and it's wasteful.

```
from sklearn.preprocessing import StandardScaler
import numpy as np

sc = StandardScaler()
sc.fit(data)
sc.scale_ = np.std(data, axis=0, ddof=1).to_list()
sc.transform(data)
```

### Additional context

https://en.wikipedia.org/wiki/Bessel%27s_correction

COMMENT:
Please reconsider.

As machine learning becomes used more frequently, use cases become broader than they may have been in the past.

I will be avoiding the use of StandardScaler() for this reason. 

*Update: To your point below, yes, I understand that scaling is part of preprocessing.  I have substantially shortened this comment in case a longer comment could have caused offense by seeming to suggest otherwise.*

Please consider adding this functionality in an update.

Labels: New Feature, module:preprocessing, Needs Decision - Include Feature