Talip7/scikit-learn-issues-multilabel huggingface dataset:
Add `n_jobs` and `random_state` to global config

### Describe the workflow you want to enable

I would like to be able to set `n_jobs=-1` in one place and have this take effect in any function with an `n_jobs` parameter. Same for `random_state`.

Perhaps there are other parameters that fit the theme: "if a user sets this in one instance, they _probably_ want to set it in all instances".

### Describe your proposed solution

Expand the accepted parameters of [sklearn.set_config](https://scikit-learn.org/stable/modules/generated/sklearn.set_config.html#sklearn.set_config), update all functions to fall back to the config value if the parameter isn't passed.

This would require changing a default of `None` to a sentinel (ala `_NoValue` in NumPy), to allow a user to override the global config with the value `None`, while still allowing the code to detect if the argument was passed.

Rudimentary mockup:
```py
_sentinel = object()

config = {}


def resolve_arg_value(arg_name, passed_value, default_value):
    if passed_value is not _sentinel:
        return passed_value
    
    if arg_name in config:
        return config[arg_name]
    
    return default_value


def do_something(random_state=_sentinel):
    random_state = resolve_arg_value("random_state", random_state, None)
    print(f"{random_state!s}")


do_something('Hi')  # 'Hi'
do_something()  # None
do_something(None)  # None

config['random_state'] = 77

do_something('Hi')  # 'Hi'
do_something()  # 77
do_something(None)  # None

config.pop('random_state')

do_something('Hi')  # 'Hi'
do_something()  # None
do_something(None)  # None
```

I'll admit, having to add something like `resolve_arg_value("random_state", random_state, None)` to tons of functions sounds painful, but I think for the user, being able to set and forget a random state/other params, potentially based on an environment variable would be nice.
```py
sklearn.set_config(
    random_state=None if os.environ["PROD"] else 0,
    n_jobs=-1,
)
```


### Describe alternatives you've considered, if relevant

_No response_

### Additional context

I see that the tests get a global random seed. https://github.com/scikit-learn/scikit-learn/pull/22749

COMMENT:
Update: I've just noticed that setting `np.random.seed(77)` doesn't give me reproducible results when using `GradientBoostingRegressor`, but `GradientBoostingRegressor(random_state=77)` does.

I'm also using `HalvingRandomSearchCV` with `scipy.stats.randint`, maybe that complicates things (I'm a newbie, the landscape is still a little foggy). If this isn't a known issue I'll try and create a minimal repro.

Labels: New Feature, API, Needs Decision
[Meta] consistently break ties randomly in scikit-learn estimators (with random_state) in an unbiased way

Some estimators have arbitrary ways to break ties:

- k-nearest neighbors with data points lying on a uniform 1d grid (see #23667 for the BallTree for instance);
- tied splits on different features in histogram gradient boosted trees with redundant features.
- decision tree splits on the same feature with equivalent threshods: `X = [[0], [1], [2], [3]]` and `y = [0, 1, 1, 0]`: `X > 0.5` and `X > 2.5` are tied splits but only `X > 0.5` is considered;
- possibly other models (please feel free to edit this list or suggest missing estimators in the comments).

If the tie breaking logic is deterministic, then it might introduce a non-controllable bias in a datascience pipeline. For instance when analyzing the feature importance of an histogram gradient boosting model (via permutations or SHAP values), the first feature of a group of redundant features would always deterministically be picked up by the model and could lead a naive datascientist to believe that the other features of the group are not as predictive.

Note that this is not the case for our traditional `DecisionTreeClassifier/Regressor` / `RandomForestClassifier/Regressor` and extra trees because they all do [feature shuffling](https://github.com/scikit-learn/scikit-learn/blob/c217527af5744b9d0db8761c1e3667552312e5e7/sklearn/tree/_splitter.pyx#L322) (controllable by `random_state`) by default even when `max_features == 1.0`. This makes it easy to conduct the same study many times with different seeds to see if the results are an artifact of an arbitrary tie breaking or not.

COMMENT:
Note that feature shuffling _before the call_ does not help with the k-NN bias: Still all the distance ties would not be resolved independently.

Labels: Needs Decision
[Meta] consistently break ties randomly in scikit-learn estimators (with random_state) in an unbiased way

Some estimators have arbitrary ways to break ties:

- k-nearest neighbors with data points lying on a uniform 1d grid (see #23667 for the BallTree for instance);
- tied splits on different features in histogram gradient boosted trees with redundant features.
- decision tree splits on the same feature with equivalent threshods: `X = [[0], [1], [2], [3]]` and `y = [0, 1, 1, 0]`: `X > 0.5` and `X > 2.5` are tied splits but only `X > 0.5` is considered;
- possibly other models (please feel free to edit this list or suggest missing estimators in the comments).

If the tie breaking logic is deterministic, then it might introduce a non-controllable bias in a datascience pipeline. For instance when analyzing the feature importance of an histogram gradient boosting model (via permutations or SHAP values), the first feature of a group of redundant features would always deterministically be picked up by the model and could lead a naive datascientist to believe that the other features of the group are not as predictive.

Note that this is not the case for our traditional `DecisionTreeClassifier/Regressor` / `RandomForestClassifier/Regressor` and extra trees because they all do [feature shuffling](https://github.com/scikit-learn/scikit-learn/blob/c217527af5744b9d0db8761c1e3667552312e5e7/sklearn/tree/_splitter.pyx#L322) (controllable by `random_state`) by default even when `max_features == 1.0`. This makes it easy to conduct the same study many times with different seeds to see if the results are an artifact of an arbitrary tie breaking or not.

COMMENT:
While investigating the causes of XFAIL `check_sample_weight_equivalence_on_dense_data` for tree-based models (e.g. `GradientBoostingRegressor`), we found another case of such deterministic bias: when there are tied splits on the same feature with different threshold values, only the lowest threshold is ever selected. I updated the description to add a bullet point for this one.

Neither feature-wise shuffling nor sample-wise dataset shuffling can fix this one. Sample-wise shuffling can introduce machine level rounding errors that can break those ties randomly but still in a biased manner (always favoring smaller threshold values), which makes inference on the learned partitions of the feature space particularly difficult to interpret.

Labels: Needs Decision
[Meta] consistently break ties randomly in scikit-learn estimators (with random_state) in an unbiased way

Some estimators have arbitrary ways to break ties:

- k-nearest neighbors with data points lying on a uniform 1d grid (see #23667 for the BallTree for instance);
- tied splits on different features in histogram gradient boosted trees with redundant features.
- decision tree splits on the same feature with equivalent threshods: `X = [[0], [1], [2], [3]]` and `y = [0, 1, 1, 0]`: `X > 0.5` and `X > 2.5` are tied splits but only `X > 0.5` is considered;
- possibly other models (please feel free to edit this list or suggest missing estimators in the comments).

If the tie breaking logic is deterministic, then it might introduce a non-controllable bias in a datascience pipeline. For instance when analyzing the feature importance of an histogram gradient boosting model (via permutations or SHAP values), the first feature of a group of redundant features would always deterministically be picked up by the model and could lead a naive datascientist to believe that the other features of the group are not as predictive.

Note that this is not the case for our traditional `DecisionTreeClassifier/Regressor` / `RandomForestClassifier/Regressor` and extra trees because they all do [feature shuffling](https://github.com/scikit-learn/scikit-learn/blob/c217527af5744b9d0db8761c1e3667552312e5e7/sklearn/tree/_splitter.pyx#L322) (controllable by `random_state`) by default even when `max_features == 1.0`. This makes it easy to conduct the same study many times with different seeds to see if the results are an artifact of an arbitrary tie breaking or not.

COMMENT:
> decision tree splits on the same feature with equivalent threshods: X = [[0], [1], [2], [3]] and y = [0, 1, 1, 0]: X > 0.5 and X > 2.5 are tied splits but only X > 0.5 is considered;

I've gotten familiar with the splitting code lately, and I can definitely see why. It because of the ">" in this `if current_proxy_improvement > best_proxy_improvement:` from [_splitter.pyx#L453 ](https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/tree/_splitter.pyx#L453). If you don't want to abritrarly stop at the first* best split, you would have to consider the case `current_proxy_improvement == best_proxy_improvement` to consider all the equally best splits before choosing one randomply.

I can probably open a PR to do this, would that be welcomed?

*first in term of increasing feature values, as the samples are sorted in the loop that finds the best split.

Labels: Needs Decision
[Meta] consistently break ties randomly in scikit-learn estimators (with random_state) in an unbiased way

Some estimators have arbitrary ways to break ties:

- k-nearest neighbors with data points lying on a uniform 1d grid (see #23667 for the BallTree for instance);
- tied splits on different features in histogram gradient boosted trees with redundant features.
- decision tree splits on the same feature with equivalent threshods: `X = [[0], [1], [2], [3]]` and `y = [0, 1, 1, 0]`: `X > 0.5` and `X > 2.5` are tied splits but only `X > 0.5` is considered;
- possibly other models (please feel free to edit this list or suggest missing estimators in the comments).

If the tie breaking logic is deterministic, then it might introduce a non-controllable bias in a datascience pipeline. For instance when analyzing the feature importance of an histogram gradient boosting model (via permutations or SHAP values), the first feature of a group of redundant features would always deterministically be picked up by the model and could lead a naive datascientist to believe that the other features of the group are not as predictive.

Note that this is not the case for our traditional `DecisionTreeClassifier/Regressor` / `RandomForestClassifier/Regressor` and extra trees because they all do [feature shuffling](https://github.com/scikit-learn/scikit-learn/blob/c217527af5744b9d0db8761c1e3667552312e5e7/sklearn/tree/_splitter.pyx#L322) (controllable by `random_state`) by default even when `max_features == 1.0`. This makes it easy to conduct the same study many times with different seeds to see if the results are an artifact of an arbitrary tie breaking or not.

COMMENT:
Actually, there are two dimensions to this issue with decision trees:

* multiple possible optimal splits along a single feature
* multiple features with equally optimal splits

For example, consider this case combining both dimensions:

```python
X = [[0, 0], [1, 0], [2, 1], [3, 2]]
y = [0, 1, 1, 0]
```

There are three tied splits: `X[:, 0] > 0.5`, `X[:, 0] > 2.5`, and `X[:, 1] > 1.5`.
The current implementation will randomly choose between `X[:, 0] > 0.5` and `X[:, 1] > 1.5`, but it will never consider `X[:, 0] > 2.5`.

As mentioned above, it’s fairly clear how to fix this so that `X[:, 0] > 2.5` is also considered. However, it’s less clear what the new behavior should be. Two options come to mind:

1. Choose uniformly among all three tied splits.
2. First, choose uniformly among the two tied splits for feature `X[:, 0]`; then, choose uniformly between the two equally good features.

I lean towards the second behavior. The first one might heavily favor features with many tied splits, which doesn’t seem desirable.
… Plus, I think option 2 is easier to implement ^^

Labels: Needs Decision
[RFC] WASM / pyodide as a (somewhat) officially supported platform for scikit-learn

We started having bug reports (at least one indirect, in real life report at a conference: #23707) from users of scikit-learn in WASM environment (e.g. pyodide / jupyterlite, pyscript...).

Shall we invest effort in setting CI tooling to properly test and maybe even handle packaging of scikit-learn to target that platform?

It's very likely that not all of scikit-learn will work out of the box, but with proper tooling in place we could maintain a public list of modules that have all their tests that pass and maybe a list of modules that required so patches to handle graceful degradation to target this platform (e.g. number of parallel worker threads with n_jobs).

@rth put some interesting info in the following comment on how to run the tests:

- https://github.com/scikit-learn/scikit-learn/issues/23707#issuecomment-1163053666

Pros:
- WASM is likely to be a very popular target platform, at least for education (can directly teach Python programming and ML concepts without having to teach how to install packages from the command line first).

Cons:
- test execution is probably much slower that on our regular CI targets;
- need to maintain a list of known issues / limitations;
- more packaging, release process will be even more complicated;
- SciPy is quite heavily patched because there is no working Fortran compiler on that platform  (that might change soon with lfotran) so it relies on a semi-hackish Fortran to C transpilation step that introduces additional complexity.

COMMENT:
Maybe we should start by listing the fraction of modules with failing test and scanning through them to estimate how many stem from a common known upstream limitation and how likely it is going to be lifted in the short to medium term or if there is a somewhat maintainable work-around we might want to include in the scikit-learn code base or as an external packaging patch.

Labels: RFC, Needs Investigation
[RFC] WASM / pyodide as a (somewhat) officially supported platform for scikit-learn

We started having bug reports (at least one indirect, in real life report at a conference: #23707) from users of scikit-learn in WASM environment (e.g. pyodide / jupyterlite, pyscript...).

Shall we invest effort in setting CI tooling to properly test and maybe even handle packaging of scikit-learn to target that platform?

It's very likely that not all of scikit-learn will work out of the box, but with proper tooling in place we could maintain a public list of modules that have all their tests that pass and maybe a list of modules that required so patches to handle graceful degradation to target this platform (e.g. number of parallel worker threads with n_jobs).

@rth put some interesting info in the following comment on how to run the tests:

- https://github.com/scikit-learn/scikit-learn/issues/23707#issuecomment-1163053666

Pros:
- WASM is likely to be a very popular target platform, at least for education (can directly teach Python programming and ML concepts without having to teach how to install packages from the command line first).

Cons:
- test execution is probably much slower that on our regular CI targets;
- need to maintain a list of known issues / limitations;
- more packaging, release process will be even more complicated;
- SciPy is quite heavily patched because there is no working Fortran compiler on that platform  (that might change soon with lfotran) so it relies on a semi-hackish Fortran to C transpilation step that introduces additional complexity.

COMMENT:
Thanks for the summary, I agree with your plan.

Labels: RFC, Needs Investigation
[RFC] WASM / pyodide as a (somewhat) officially supported platform for scikit-learn

We started having bug reports (at least one indirect, in real life report at a conference: #23707) from users of scikit-learn in WASM environment (e.g. pyodide / jupyterlite, pyscript...).

Shall we invest effort in setting CI tooling to properly test and maybe even handle packaging of scikit-learn to target that platform?

It's very likely that not all of scikit-learn will work out of the box, but with proper tooling in place we could maintain a public list of modules that have all their tests that pass and maybe a list of modules that required so patches to handle graceful degradation to target this platform (e.g. number of parallel worker threads with n_jobs).

@rth put some interesting info in the following comment on how to run the tests:

- https://github.com/scikit-learn/scikit-learn/issues/23707#issuecomment-1163053666

Pros:
- WASM is likely to be a very popular target platform, at least for education (can directly teach Python programming and ML concepts without having to teach how to install packages from the command line first).

Cons:
- test execution is probably much slower that on our regular CI targets;
- need to maintain a list of known issues / limitations;
- more packaging, release process will be even more complicated;
- SciPy is quite heavily patched because there is no working Fortran compiler on that platform  (that might change soon with lfotran) so it relies on a semi-hackish Fortran to C transpilation step that introduces additional complexity.

COMMENT:
Once the test runner tooling is improved, we could imagine a nightly run that would run the test suite of each top level scikit-learn module and consolidate a report of scikit-learn modules that work without any failure, run with some test failures or finally cause an unrecoverable crash of a fatal error of the WASM runtime environment (it would be great to automatically collect the post-mortem output of the JS console of the browser in such a case).

Labels: RFC, Needs Investigation
[RFC] WASM / pyodide as a (somewhat) officially supported platform for scikit-learn

We started having bug reports (at least one indirect, in real life report at a conference: #23707) from users of scikit-learn in WASM environment (e.g. pyodide / jupyterlite, pyscript...).

Shall we invest effort in setting CI tooling to properly test and maybe even handle packaging of scikit-learn to target that platform?

It's very likely that not all of scikit-learn will work out of the box, but with proper tooling in place we could maintain a public list of modules that have all their tests that pass and maybe a list of modules that required so patches to handle graceful degradation to target this platform (e.g. number of parallel worker threads with n_jobs).

@rth put some interesting info in the following comment on how to run the tests:

- https://github.com/scikit-learn/scikit-learn/issues/23707#issuecomment-1163053666

Pros:
- WASM is likely to be a very popular target platform, at least for education (can directly teach Python programming and ML concepts without having to teach how to install packages from the command line first).

Cons:
- test execution is probably much slower that on our regular CI targets;
- need to maintain a list of known issues / limitations;
- more packaging, release process will be even more complicated;
- SciPy is quite heavily patched because there is no working Fortran compiler on that platform  (that might change soon with lfotran) so it relies on a semi-hackish Fortran to C transpilation step that introduces additional complexity.

COMMENT:
Btw, I think one of the benefits we'd get from WASM support is the ability to have interactive examples in the browser on the docs. I think that'll be a gamechanger for documentation.

Labels: RFC, Needs Investigation
[RFC] WASM / pyodide as a (somewhat) officially supported platform for scikit-learn

We started having bug reports (at least one indirect, in real life report at a conference: #23707) from users of scikit-learn in WASM environment (e.g. pyodide / jupyterlite, pyscript...).

Shall we invest effort in setting CI tooling to properly test and maybe even handle packaging of scikit-learn to target that platform?

It's very likely that not all of scikit-learn will work out of the box, but with proper tooling in place we could maintain a public list of modules that have all their tests that pass and maybe a list of modules that required so patches to handle graceful degradation to target this platform (e.g. number of parallel worker threads with n_jobs).

@rth put some interesting info in the following comment on how to run the tests:

- https://github.com/scikit-learn/scikit-learn/issues/23707#issuecomment-1163053666

Pros:
- WASM is likely to be a very popular target platform, at least for education (can directly teach Python programming and ML concepts without having to teach how to install packages from the command line first).

Cons:
- test execution is probably much slower that on our regular CI targets;
- need to maintain a list of known issues / limitations;
- more packaging, release process will be even more complicated;
- SciPy is quite heavily patched because there is no working Fortran compiler on that platform  (that might change soon with lfotran) so it relies on a semi-hackish Fortran to C transpilation step that introduces additional complexity.

COMMENT:
I put together a repo to run the scikit-learn inside Pyodide. I listed the issues I have spotted now there. This will need more investigation. Any feed-back, let me know!

https://github.com/lesteve/scikit-learn-tests-pyodide

Labels: RFC, Needs Investigation
[RFC] WASM / pyodide as a (somewhat) officially supported platform for scikit-learn

We started having bug reports (at least one indirect, in real life report at a conference: #23707) from users of scikit-learn in WASM environment (e.g. pyodide / jupyterlite, pyscript...).

Shall we invest effort in setting CI tooling to properly test and maybe even handle packaging of scikit-learn to target that platform?

It's very likely that not all of scikit-learn will work out of the box, but with proper tooling in place we could maintain a public list of modules that have all their tests that pass and maybe a list of modules that required so patches to handle graceful degradation to target this platform (e.g. number of parallel worker threads with n_jobs).

@rth put some interesting info in the following comment on how to run the tests:

- https://github.com/scikit-learn/scikit-learn/issues/23707#issuecomment-1163053666

Pros:
- WASM is likely to be a very popular target platform, at least for education (can directly teach Python programming and ML concepts without having to teach how to install packages from the command line first).

Cons:
- test execution is probably much slower that on our regular CI targets;
- need to maintain a list of known issues / limitations;
- more packaging, release process will be even more complicated;
- SciPy is quite heavily patched because there is no working Fortran compiler on that platform  (that might change soon with lfotran) so it relies on a semi-hackish Fortran to C transpilation step that introduces additional complexity.

COMMENT:
We have the build in our CI as a cron job and we pretty much support it at this point. I think we can close this one and have dedicated issues for whatever comes up related to WASM.

Labels: RFC, Needs Investigation