Talip7/scikit-learn-issues-multilabel huggingface dataset:
Mandatory random seeds

### Describe the workflow you want to enable

So far, in methods such as `train_test_split`, it is not mandatory to set a random seed. In the default case, where the seed is `None`, the seed is determined randomly.

Thus, I would like to propose making mandatory setting the random seeds of **all** algorithms, classes and methods that request one.

### Describe your proposed solution

My idea for solving this issue is simple: raising an error if a seed was not set as a parameter for a given algorithm/method/class.

### Describe alternatives you've considered, if relevant

Alternatively, if this could not be implemented, I would propose giving a warning to the user if a seed was not set.

### Additional context

Given all the problems regarding reproducibility in data science and machine learning, I think a good first step in tackling this problem is to make aware to as many developers as possible of the importance of having reproducible results.

And to get reproducible results, it should be mandatory to **always** use non-default random seeds in the code since, in this way, the same seeds will yield the same results, thus achieving fully reproducible results.

COMMENT:
However, I am not against having an easy way to provide a deterministic pipeline. @ogrisel is there a particular danger to having the option available in `set_config` instead of an environment variable (why not both as well)?

Labels: RFC
Mandatory random seeds

### Describe the workflow you want to enable

So far, in methods such as `train_test_split`, it is not mandatory to set a random seed. In the default case, where the seed is `None`, the seed is determined randomly.

Thus, I would like to propose making mandatory setting the random seeds of **all** algorithms, classes and methods that request one.

### Describe your proposed solution

My idea for solving this issue is simple: raising an error if a seed was not set as a parameter for a given algorithm/method/class.

### Describe alternatives you've considered, if relevant

Alternatively, if this could not be implemented, I would propose giving a warning to the user if a seed was not set.

### Additional context

Given all the problems regarding reproducibility in data science and machine learning, I think a good first step in tackling this problem is to make aware to as many developers as possible of the importance of having reproducible results.

And to get reproducible results, it should be mandatory to **always** use non-default random seeds in the code since, in this way, the same seeds will yield the same results, thus achieving fully reproducible results.

COMMENT:
> @ogrisel is there a particular danger to having the option available in set_config instead of an environment variable (why not both as well)?

We could do that but using `set_config` requires editing the source code and directly seeding the numpy rng would be equivalent and more explicit I think.

Labels: RFC
Mandatory random seeds

### Describe the workflow you want to enable

So far, in methods such as `train_test_split`, it is not mandatory to set a random seed. In the default case, where the seed is `None`, the seed is determined randomly.

Thus, I would like to propose making mandatory setting the random seeds of **all** algorithms, classes and methods that request one.

### Describe your proposed solution

My idea for solving this issue is simple: raising an error if a seed was not set as a parameter for a given algorithm/method/class.

### Describe alternatives you've considered, if relevant

Alternatively, if this could not be implemented, I would propose giving a warning to the user if a seed was not set.

### Additional context

Given all the problems regarding reproducibility in data science and machine learning, I think a good first step in tackling this problem is to make aware to as many developers as possible of the importance of having reproducible results.

And to get reproducible results, it should be mandatory to **always** use non-default random seeds in the code since, in this way, the same seeds will yield the same results, thus achieving fully reproducible results.

COMMENT:
> Always fixing the seeds might give a misleading sense of determinism, this is why we intentionally to not hide the inherent randomness of a scikit-learn algorithm.

Also it leads to under-estimating the variance of observed performance scores in evaluations (see https://proceedings.mlsys.org/paper/2021/hash/cfecdb276f634854f3ef915e2e980c31-Abstract.html ), which leads people to selecting wrong models (typically models that are too complex and lead to no performance gains).

Labels: RFC
Mandatory random seeds

### Describe the workflow you want to enable

So far, in methods such as `train_test_split`, it is not mandatory to set a random seed. In the default case, where the seed is `None`, the seed is determined randomly.

Thus, I would like to propose making mandatory setting the random seeds of **all** algorithms, classes and methods that request one.

### Describe your proposed solution

My idea for solving this issue is simple: raising an error if a seed was not set as a parameter for a given algorithm/method/class.

### Describe alternatives you've considered, if relevant

Alternatively, if this could not be implemented, I would propose giving a warning to the user if a seed was not set.

### Additional context

Given all the problems regarding reproducibility in data science and machine learning, I think a good first step in tackling this problem is to make aware to as many developers as possible of the importance of having reproducible results.

And to get reproducible results, it should be mandatory to **always** use non-default random seeds in the code since, in this way, the same seeds will yield the same results, thus achieving fully reproducible results.

COMMENT:
I agree with the concerns expressed regarding accidentally misleading users, either in terms of determinism or in terms of variance. 

> Maybe we could introduce a new random variable `SKLEARN_DEFAULT_RANDOM_STATE_SEED` and use that whenever `check_random_state(None)` is called.

I'm much more comfortable with an opt-in solution like this. Ultimately, it doesn't address the original intent of highlighting the importance of reproducibility to users quite as drastically, but I think its existence (and documentation) would still advance that goal.

Labels: RFC
Mandatory random seeds

### Describe the workflow you want to enable

So far, in methods such as `train_test_split`, it is not mandatory to set a random seed. In the default case, where the seed is `None`, the seed is determined randomly.

Thus, I would like to propose making mandatory setting the random seeds of **all** algorithms, classes and methods that request one.

### Describe your proposed solution

My idea for solving this issue is simple: raising an error if a seed was not set as a parameter for a given algorithm/method/class.

### Describe alternatives you've considered, if relevant

Alternatively, if this could not be implemented, I would propose giving a warning to the user if a seed was not set.

### Additional context

Given all the problems regarding reproducibility in data science and machine learning, I think a good first step in tackling this problem is to make aware to as many developers as possible of the importance of having reproducible results.

And to get reproducible results, it should be mandatory to **always** use non-default random seeds in the code since, in this way, the same seeds will yield the same results, thus achieving fully reproducible results.

COMMENT:
To summarize: The proposal was to disallow `random_state=None`, but the majority of core devs seems to be in favor of keeping it.

I therefore dare to close. Feel free to reopen for the `SKLEARN_DEFAULT_RANDOM_STATE_SEED` or open a new issue for it.

Labels: RFC
⚠️ CI failed on Linux_Nightly.pylatest_pip_scipy_dev ⚠️

**CI failed on [Linux_Nightly.pylatest_pip_scipy_dev](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=47895&view=logs&j=dfe99b15-50db-5d7b-b1e9-4105c42527cf)** (Oct 24, 2022)
- test_function_docstring[sklearn.preprocessing._data.quantile_transform]

COMMENT:
Should be fixed by #24739.

Labels: Needs Triage
⚠️ CI failed on Linux_Nightly.pylatest_pip_scipy_dev ⚠️

**CI failed on [Linux_Nightly.pylatest_pip_scipy_dev](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=47895&view=logs&j=dfe99b15-50db-5d7b-b1e9-4105c42527cf)** (Oct 24, 2022)
- test_function_docstring[sklearn.preprocessing._data.quantile_transform]

COMMENT:
Closing since the CI works on `main`.

Labels: Needs Triage
⚠️ CI failed on Linux.pylatest_pip_openblas_pandas ⚠️

**CI failed on [Linux.pylatest_pip_openblas_pandas](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=47895&view=logs&j=78a0bf4f-79e5-5387-94ec-13e67d216d6e)** (Oct 24, 2022)
- test_function_docstring[sklearn.preprocessing._data.quantile_transform]

COMMENT:
My fault... https://github.com/scikit-learn/scikit-learn/pull/24733#pullrequestreview-1152344273

Labels: Needs Triage
⚠️ CI failed on Linux.pylatest_pip_openblas_pandas ⚠️

**CI failed on [Linux.pylatest_pip_openblas_pandas](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=47895&view=logs&j=78a0bf4f-79e5-5387-94ec-13e67d216d6e)** (Oct 24, 2022)
- test_function_docstring[sklearn.preprocessing._data.quantile_transform]

COMMENT:
Should be fixed by #24739.

Labels: Needs Triage
⚠️ CI failed on Linux.pylatest_pip_openblas_pandas ⚠️

**CI failed on [Linux.pylatest_pip_openblas_pandas](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=47895&view=logs&j=78a0bf4f-79e5-5387-94ec-13e67d216d6e)** (Oct 24, 2022)
- test_function_docstring[sklearn.preprocessing._data.quantile_transform]

COMMENT:
Closing since the CI works on `main`.

Labels: Needs Triage
How observations with sample_weight of zero influence the fit of HistGradientBoostingRegressor

### Describe the bug

Hello,

I am trying to exclude some training observations by giving them a weight of zero through the "sample_weights" argument.

As I understand it, observations with a weight of 0 do not influence the training, so changes in their values should not affect the resulting model. However, this is not what I see if I train two models, each one with different values in the samples with weight of zero.

Does anyone know how exactly the algorithm implements sample weight inside the code?

Thanks a lot!

### Steps/Code to Reproduce

```python
import numpy as np
from sklearn.ensemble import HistGradientBoostingRegressor

rng = np.random.default_rng(12345)
X_train = rng.normal(size=(100, 3))
y_train = rng.normal(loc=10, size=(100, 1)).ravel()
X_test  = rng.normal(size=(5, 3))
weights = np.repeat([0, 1], repeats=[10, 90])

regressor = HistGradientBoostingRegressor(random_state=123)
regressor.fit(X=X_train, y=y_train, sample_weight=weights)
print(regressor.predict(X=X_test))

X_train_2 = X_train.copy()
X_train_2[:10, :] = 50000
regressor = HistGradientBoostingRegressor(random_state=123)
regressor.fit(X=X_train_2, y=y_train, sample_weight=weights)
print(regressor.predict(X=X_test))
```

### Expected Results

array([10.02028706, 10.36184929,  9.45997232,  9.18761327,  9.93495853])
array([10.02028706, 10.36184929,  9.45997232,  9.18761327,  9.93495853])

### Actual Results

array([10.02028706, 10.36184929,  9.45997232,  9.18761327,  9.93495853])
array([10.06163207, 10.17065387,  9.04865117,  9.15543545,  9.92940014])

### Versions

```shell
System:
    python: 3.8.13 (default, Mar 28 2022, 11:38:47)  [GCC 7.5.0]
executable: /home/ubuntu/anaconda3/envs/skforecast/bin/python
   machine: Linux-5.15.0-1022-aws-x86_64-with-glibc2.17

Python dependencies:
      sklearn: 1.1.0
          pip: 22.1.2
   setuptools: 63.4.1
        numpy: 1.23.0
        scipy: 1.9.1
       Cython: None
       pandas: 1.4.0
   matplotlib: 3.5.0
       joblib: 1.1.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /home/ubuntu/anaconda3/envs/skforecast/lib/python3.8/site-packages/numpy.libs/libopenblas64_p-r0-742d56dc.3.20.so
        version: 0.3.20
threading_layer: pthreads
   architecture: SkylakeX
    num_threads: 8

       user_api: openmp
   internal_api: openmp
         prefix: libgomp
       filepath: /home/ubuntu/anaconda3/envs/skforecast/lib/python3.8/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
    num_threads: 8

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /home/ubuntu/anaconda3/envs/skforecast/lib/python3.8/site-packages/scipy.libs/libopenblasp-r0-9f9f5dbc.3.18.so
        version: 0.3.18
threading_layer: pthreads
   architecture: SkylakeX
    num_threads: 8
```

COMMENT:
I close in favor of #27117.

Labels: Bug, module:ensemble
`See also` section for `pairwise_distances` and `paired_distances` does not link to referenced documentation.

### Describe the issue linked to the documentation

The current documentation of [pairwise_distances](https://scikit-learn.org/dev/modules/generated/sklearn.metrics.pairwise_distances.html) and [paired_distances](https://scikit-learn.org/dev/modules/generated/sklearn.metrics.pairwise.paired_distances.html) does not contain hyperlinks working in the `See Also` section. 

Is this the intended behavior? 

### Suggest a potential alternative/fix

I'm not actually sure why this would happen as the references seem to be correct. 
Is it because `pairwise_distances` has hyperlink to `paired_distances` and vice-versa is also done?

COMMENT:
<img width="1060" alt="Screenshot 2022-10-22 at 6 40 25 AM" src="https://user-images.githubusercontent.com/25542291/197309819-f4c3021e-5f0c-4b67-9258-6eb124155b31.png">

<img width="1170" alt="Screenshot 2022-10-22 at 6 40 35 AM" src="https://user-images.githubusercontent.com/25542291/197309826-d646a444-3ade-448c-9566-fc4054ae3e9b.png">

This is what I am seeing

Labels: Documentation
`See also` section for `pairwise_distances` and `paired_distances` does not link to referenced documentation.

### Describe the issue linked to the documentation

The current documentation of [pairwise_distances](https://scikit-learn.org/dev/modules/generated/sklearn.metrics.pairwise_distances.html) and [paired_distances](https://scikit-learn.org/dev/modules/generated/sklearn.metrics.pairwise.paired_distances.html) does not contain hyperlinks working in the `See Also` section. 

Is this the intended behavior? 

### Suggest a potential alternative/fix

I'm not actually sure why this would happen as the references seem to be correct. 
Is it because `pairwise_distances` has hyperlink to `paired_distances` and vice-versa is also done?

COMMENT:
For the `paired_distances`, I assume that this is because it leaves in the module `sklearn.metrics.pairwise` and is not exposed in `sklearn.metrics`.

I assume that this is a similar reason in the "See Also" of `paired_distances`. We try to link to `pairwise_distances` that live in `sklearn.metrics` and not `sklearn.metrics.pairwise` as `paired_distances`, thus the back reference is failing.

@jjerphan is there a reason that we don't expose `paired_distances` directly in the module `metrics`. I know that we should most probably make it private but until we deprecate its access, we should also expose it at the upper level as well to be consistent.

Labels: Documentation
`See also` section for `pairwise_distances` and `paired_distances` does not link to referenced documentation.

### Describe the issue linked to the documentation

The current documentation of [pairwise_distances](https://scikit-learn.org/dev/modules/generated/sklearn.metrics.pairwise_distances.html) and [paired_distances](https://scikit-learn.org/dev/modules/generated/sklearn.metrics.pairwise.paired_distances.html) does not contain hyperlinks working in the `See Also` section. 

Is this the intended behavior? 

### Suggest a potential alternative/fix

I'm not actually sure why this would happen as the references seem to be correct. 
Is it because `pairwise_distances` has hyperlink to `paired_distances` and vice-versa is also done?

COMMENT:
> @jjerphan is there a reason that we don't expose paired_distances directly in the module metrics [?]

I do not know. I would suspect that historically [all the interfaces related to pairwise distances computations were chosen to be exposed via `sklearn.metrics.pairwise`](https://github.com/scikit-learn/scikit-learn/blob/3e6a39a73c2ca39e073e4b58117f59e92b3b2313/doc/modules/classes.rst#pairwise-metrics) since `sklearn.metrics` also exposed a lot of semantically different interfaces. I would also suspect `sklearn.metrics.pairwise_distances` was preferred over `sklearn.metrics.pairwise.pairwise_distances` as an interface because this last one is a bit to cumbersome.

Not that `metrics.pairwise.paired_{euclidean,manhattan,cosine}_distances` might also be subject to this issue.

> I know that we should most probably make it private but until we deprecate its access, we should also expose it at the upper level as well to be consistent.

Do you mean we should deprecated `paired_distances` entirely or just deprecated `sklearn.metrics.pairwise.paired_distance` as an API in preference for `sklearn.metrics.paired_distance`?

I am not sure if `paired_*_distances` API are than much used by users. If they aren't, I think it would make sense deprecating them. What do you think?

Labels: Documentation
`See also` section for `pairwise_distances` and `paired_distances` does not link to referenced documentation.

### Describe the issue linked to the documentation

The current documentation of [pairwise_distances](https://scikit-learn.org/dev/modules/generated/sklearn.metrics.pairwise_distances.html) and [paired_distances](https://scikit-learn.org/dev/modules/generated/sklearn.metrics.pairwise.paired_distances.html) does not contain hyperlinks working in the `See Also` section. 

Is this the intended behavior? 

### Suggest a potential alternative/fix

I'm not actually sure why this would happen as the references seem to be correct. 
Is it because `pairwise_distances` has hyperlink to `paired_distances` and vice-versa is also done?

COMMENT:
> Do you mean we should deprecated paired_distances entirely or just deprecated sklearn.metrics.pairwise.paired_distance as an API in preference for sklearn.metrics.paired_distance?

From previous discussions, I was thinking that you would be inclined to completely deprecate it, isn't it?

For the import path, exposes it in `__init__.py` will not make it impossible to import it via the `pairwise` module so it would be backward compatible as well.

> I am not sure if paired_*_distances API are than much used by users. If they aren't, I think it would make sense deprecating them. What do you think?

It was actually my question :). I don't know enough about the usage of the function (users and developers) to have the necessary perspective to know if a deprecation and removal will have a negative impact.

Labels: Documentation
`See also` section for `pairwise_distances` and `paired_distances` does not link to referenced documentation.

### Describe the issue linked to the documentation

The current documentation of [pairwise_distances](https://scikit-learn.org/dev/modules/generated/sklearn.metrics.pairwise_distances.html) and [paired_distances](https://scikit-learn.org/dev/modules/generated/sklearn.metrics.pairwise.paired_distances.html) does not contain hyperlinks working in the `See Also` section. 

Is this the intended behavior? 

### Suggest a potential alternative/fix

I'm not actually sure why this would happen as the references seem to be correct. 
Is it because `pairwise_distances` has hyperlink to `paired_distances` and vice-versa is also done?

COMMENT:
I would deprecate this function entirely if they aren't used by many, but I think we should discuss it separately (to me, the scope of this issue is different).

I think changing API fully qualified names are costly for everyone. I also think having two import paths for API is confusing and potentially error prone. Hence I would prefer to only correct the references in doc-strings if there is not strong inconsistency. What do you think?

Labels: Documentation
verbose_feature_names_out default value

### Describe the workflow you want to enable

So I obviously love the example in https://scikit-learn.org/dev/auto_examples/miscellaneous/plot_set_output.html#sphx-glr-auto-examples-miscellaneous-plot-set-output-py

But it's a bit of a bummer that the pipeline needs so many parameters.
I wonder if we can revisit the default for ``verbose_feature_names_out``. I think we had a lengthy discussion about it, which I remember as follows

- if we do False by default, pipelines might start to fail that haven't failed before
- if we do "auto" by default (only prefix if required to disambiguate), feature names might start changing unexpectedly in a magic way
- if we do True by default, the names might be awkwardly long unnecessarily

So doing "true" seems the best trade-off for production-grade use....

### Describe your proposed solution

maybe this is a good reason to re-consider adding an "interactive" or "unsafe" mode with interactivity-friendly defaults? 

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I like the "interactive" for production-unsafe-but-friendly defaults more than other options.

Labels: Needs Decision, RFC
verbose_feature_names_out default value

### Describe the workflow you want to enable

So I obviously love the example in https://scikit-learn.org/dev/auto_examples/miscellaneous/plot_set_output.html#sphx-glr-auto-examples-miscellaneous-plot-set-output-py

But it's a bit of a bummer that the pipeline needs so many parameters.
I wonder if we can revisit the default for ``verbose_feature_names_out``. I think we had a lengthy discussion about it, which I remember as follows

- if we do False by default, pipelines might start to fail that haven't failed before
- if we do "auto" by default (only prefix if required to disambiguate), feature names might start changing unexpectedly in a magic way
- if we do True by default, the names might be awkwardly long unnecessarily

So doing "true" seems the best trade-off for production-grade use....

### Describe your proposed solution

maybe this is a good reason to re-consider adding an "interactive" or "unsafe" mode with interactivity-friendly defaults? 

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
> maybe this is a good reason to re-consider adding an "interactive" or "unsafe" mode with interactivity-friendly defaults?

I like having an interactive mode. To clarify, are you proposing a global configuration option for "interactive mode"? If so, what other defaults would fall under this mode?

The other configuration I see is parallelism, where "interactive mode" == "use all cores" and the default is single core. This means all OpenMP and BLAS routines will be configured to be single threaded by default. I think single threaded is a better default for production, but interactive mode is better with all cores..

Labels: Needs Decision, RFC