Talip7/scikit-learn-issues-multilabel huggingface dataset:
Allow LogisticRegression with lbfgs solver to control `maxfun` parameter of solver

### Describe the workflow you want to enable

Similarly to what is mentioned on https://github.com/scikit-learn/scikit-learn/issues/9273

> Training an MLP regressor (or classifier) using l-bfgs currently cannot run for more than (approx) 15000 iterations.
This artificial limit is caused by the call site to l-bfgs passing the MLP argument value "max_iters" to the argument for "maxfun" (maximum number of function calls), but not for "maxiter" (maximum number of iterations), so that no matter how large a number you pass as "max_iters" to train for MLP, the iterations are capped by the default value for maxiter (15000).

training a LogisticRegression regressor using l-bfgs currently cannot perform more than (approx) 15000 **function evaluations**.
This artificial limit is caused by the call site to l-bfgs passing the `LogisticRegression` argument value `max_iters` to the argument for `maxiter` (maximum number of iterations), but not for `maxfun` (maximum number of function evaluations), so that no matter how large a number you pass as "max_iters" to train for LogisticRegression, the function evaluations are capped by the default value for maxiter (15000, defined on https://github.com/scipy/scipy/blob/bf776169c753fff655200dc15ae26db95a083b02/scipy/optimize/_lbfgsb_py.py#L212).



### Describe your proposed solution

When calling the l-bfgs solver, set `maxfun` to be the same as `maxiter`, allow the user to control it.

As stated on [#9274](https://github.com/scikit-learn/scikit-learn/pull/9274) by @daniel-perry,
> Ideally you would want to pass in both a 'max_iter' and 'max_fun' argument to MLP, however 'max_fun' doesn't make sense for anything but l-bfgs, so using 'max_iter' to control both seems a reasonable compromise.

The same rational could be used here.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Just started fixing this myself.

Design-wise, I would opt for passing an optimizer to `fit` and/or `__init__`. The idea is that the user has complete freedom to set up an optimizer of their choice and then they combine it with a model in order to do the fitting. However, this is not how scikit-learn has been put together.

The option that aligns most with the code that already exists is to just add a `max_f_evals` argument (or to just set `maxfun` to `max_iter`). Whether this or a general `solver_options` argument would be better seems (to me) to depend on a survey of the available optimizers to find out what other parameters users might want to control. However, reducing all the solver-related arguments to the arguments `solver` and `solver_options` seems like a good general solution short of passing an optimizer directly. This approach seems like large-scale design work, though, in that it would affect the constructor of every model that uses an optimizer for fitting. So, in that sense, are there short- and long-term solutions (small and big solutions) to be considered here? (Is there someone designated to make these calls?)

Labels: Enhancement, API, Needs Decision
Allow LogisticRegression with lbfgs solver to control `maxfun` parameter of solver

### Describe the workflow you want to enable

Similarly to what is mentioned on https://github.com/scikit-learn/scikit-learn/issues/9273

> Training an MLP regressor (or classifier) using l-bfgs currently cannot run for more than (approx) 15000 iterations.
This artificial limit is caused by the call site to l-bfgs passing the MLP argument value "max_iters" to the argument for "maxfun" (maximum number of function calls), but not for "maxiter" (maximum number of iterations), so that no matter how large a number you pass as "max_iters" to train for MLP, the iterations are capped by the default value for maxiter (15000).

training a LogisticRegression regressor using l-bfgs currently cannot perform more than (approx) 15000 **function evaluations**.
This artificial limit is caused by the call site to l-bfgs passing the `LogisticRegression` argument value `max_iters` to the argument for `maxiter` (maximum number of iterations), but not for `maxfun` (maximum number of function evaluations), so that no matter how large a number you pass as "max_iters" to train for LogisticRegression, the function evaluations are capped by the default value for maxiter (15000, defined on https://github.com/scipy/scipy/blob/bf776169c753fff655200dc15ae26db95a083b02/scipy/optimize/_lbfgsb_py.py#L212).



### Describe your proposed solution

When calling the l-bfgs solver, set `maxfun` to be the same as `maxiter`, allow the user to control it.

As stated on [#9274](https://github.com/scikit-learn/scikit-learn/pull/9274) by @daniel-perry,
> Ideally you would want to pass in both a 'max_iter' and 'max_fun' argument to MLP, however 'max_fun' doesn't make sense for anything but l-bfgs, so using 'max_iter' to control both seems a reasonable compromise.

The same rational could be used here.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Taking the design work done by the `Optim.jl` people as a proxy for surveying the available SciPy optimizers myself, they have the following standard options: x_tol, f_tol, g_tol, f_calls_limit, g_calls_limit, h_calls_limit, allow_f_increases, iterations, store_trace, show_trace, extended_trace, trace_simplex, show_every, callback, time_limit.  See https://julianlsolvers.github.io/Optim.jl/stable/#user/config/#general-options.

Labels: Enhancement, API, Needs Decision
Allow LogisticRegression with lbfgs solver to control `maxfun` parameter of solver

### Describe the workflow you want to enable

Similarly to what is mentioned on https://github.com/scikit-learn/scikit-learn/issues/9273

> Training an MLP regressor (or classifier) using l-bfgs currently cannot run for more than (approx) 15000 iterations.
This artificial limit is caused by the call site to l-bfgs passing the MLP argument value "max_iters" to the argument for "maxfun" (maximum number of function calls), but not for "maxiter" (maximum number of iterations), so that no matter how large a number you pass as "max_iters" to train for MLP, the iterations are capped by the default value for maxiter (15000).

training a LogisticRegression regressor using l-bfgs currently cannot perform more than (approx) 15000 **function evaluations**.
This artificial limit is caused by the call site to l-bfgs passing the `LogisticRegression` argument value `max_iters` to the argument for `maxiter` (maximum number of iterations), but not for `maxfun` (maximum number of function evaluations), so that no matter how large a number you pass as "max_iters" to train for LogisticRegression, the function evaluations are capped by the default value for maxiter (15000, defined on https://github.com/scipy/scipy/blob/bf776169c753fff655200dc15ae26db95a083b02/scipy/optimize/_lbfgsb_py.py#L212).



### Describe your proposed solution

When calling the l-bfgs solver, set `maxfun` to be the same as `maxiter`, allow the user to control it.

As stated on [#9274](https://github.com/scikit-learn/scikit-learn/pull/9274) by @daniel-perry,
> Ideally you would want to pass in both a 'max_iter' and 'max_fun' argument to MLP, however 'max_fun' doesn't make sense for anything but l-bfgs, so using 'max_iter' to control both seems a reasonable compromise.

The same rational could be used here.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Example fixes:
* set `maxfun` from `max_iter`: https://github.com/scikit-learn/scikit-learn/compare/main...afbarnard:scikit-learn:issue-27484-fix-copy-max_iter?expand=1
* add a new `max_f_evals` argument: https://github.com/scikit-learn/scikit-learn/compare/main...afbarnard:scikit-learn:issue-27484-fix-plumb-arg?expand=1

(I've ignored making the same changes to `LogisticRegressionCV` for now.)

Labels: Enhancement, API, Needs Decision
Homogeneity Score is Not Consistently Correct For Trivial Clustering

### Describe the bug

The homogeneity_score is not being computed consistently when you have a single truth label for different array sizes. It seems not to matter how many unique labels are in the predicted labels, just so long as there is only one truth label.  



### Steps/Code to Reproduce

```python
from sklearn.metrics import homogeneity_score
for i in range(1000):
    print(f'For trivial array of length {i}, {homogeneity_score([0]*i,[1]*i)}') 
```

### Expected Results

I would expect this to always be 1.0 for a single truth label, regardless of array size. 


### Actual Results

You will see that the homogeneity score bounces around among -1.0, 0.0, and 1.0 depending on array size. 

### Versions

```shell
System:
    python: 3.7.15 (default, Nov 7 2022, 22:00:21) [GCC 11.2.0]
executable: /home/ray/anaconda3/bin/python
    machine: Linux-5.4.0-1071-aws-x86_64-with_debian-bullseye_sid

Python dependencies:
pip: 23.1
setuptools: 65.6.3
sklearn: 1.0.2
numpy: 1.21.6
scipy: 1.7.3
Cython: 0.29.32
pandas: 1.3.5
matplotlib: 3.5.3
joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

COMMENT:
Bro, try to update your Python version(Python 3.7 is old now). Preferably update the Python version to 3.10.12. Its working for me in Python 3.10.12, I am getting all the printed outputs as '1.0'. I am attaching the Colab notebook for your reference, the link is below:
https://colab.research.google.com/drive/1DIY8oy6JCi_FJskbAAI_JIqNdkRIqbN3#scrollTo=o7G0w2nMgWPf

Labels: Bug, Needs Triage
Homogeneity Score is Not Consistently Correct For Trivial Clustering

### Describe the bug

The homogeneity_score is not being computed consistently when you have a single truth label for different array sizes. It seems not to matter how many unique labels are in the predicted labels, just so long as there is only one truth label.  



### Steps/Code to Reproduce

```python
from sklearn.metrics import homogeneity_score
for i in range(1000):
    print(f'For trivial array of length {i}, {homogeneity_score([0]*i,[1]*i)}') 
```

### Expected Results

I would expect this to always be 1.0 for a single truth label, regardless of array size. 


### Actual Results

You will see that the homogeneity score bounces around among -1.0, 0.0, and 1.0 depending on array size. 

### Versions

```shell
System:
    python: 3.7.15 (default, Nov 7 2022, 22:00:21) [GCC 11.2.0]
executable: /home/ray/anaconda3/bin/python
    machine: Linux-5.4.0-1071-aws-x86_64-with_debian-bullseye_sid

Python dependencies:
pip: 23.1
setuptools: 65.6.3
sklearn: 1.0.2
numpy: 1.21.6
scipy: 1.7.3
Cython: 0.29.32
pandas: 1.3.5
matplotlib: 3.5.3
joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

COMMENT:
I cannot reproduce as well in `main`. It was probably a bug in older version of scikit-learn.
Closing.

Labels: Bug, Needs Triage
Homogeneity Score is Not Consistently Correct For Trivial Clustering

### Describe the bug

The homogeneity_score is not being computed consistently when you have a single truth label for different array sizes. It seems not to matter how many unique labels are in the predicted labels, just so long as there is only one truth label.  



### Steps/Code to Reproduce

```python
from sklearn.metrics import homogeneity_score
for i in range(1000):
    print(f'For trivial array of length {i}, {homogeneity_score([0]*i,[1]*i)}') 
```

### Expected Results

I would expect this to always be 1.0 for a single truth label, regardless of array size. 


### Actual Results

You will see that the homogeneity score bounces around among -1.0, 0.0, and 1.0 depending on array size. 

### Versions

```shell
System:
    python: 3.7.15 (default, Nov 7 2022, 22:00:21) [GCC 11.2.0]
executable: /home/ray/anaconda3/bin/python
    machine: Linux-5.4.0-1071-aws-x86_64-with_debian-bullseye_sid

Python dependencies:
pip: 23.1
setuptools: 65.6.3
sklearn: 1.0.2
numpy: 1.21.6
scipy: 1.7.3
Cython: 0.29.32
pandas: 1.3.5
matplotlib: 3.5.3
joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

COMMENT:
The Python version is irrelevant here. A specific version of scikit-learn will work for a given range of Python version. Scikit-learn 1.0.2 was supporting Python 3.7.

The important thing here is to update to scikit-learn 1.3.1 that requires Python >= 3.8

Labels: Bug, Needs Triage
Homogeneity Score is Not Consistently Correct For Trivial Clustering

### Describe the bug

The homogeneity_score is not being computed consistently when you have a single truth label for different array sizes. It seems not to matter how many unique labels are in the predicted labels, just so long as there is only one truth label.  



### Steps/Code to Reproduce

```python
from sklearn.metrics import homogeneity_score
for i in range(1000):
    print(f'For trivial array of length {i}, {homogeneity_score([0]*i,[1]*i)}') 
```

### Expected Results

I would expect this to always be 1.0 for a single truth label, regardless of array size. 


### Actual Results

You will see that the homogeneity score bounces around among -1.0, 0.0, and 1.0 depending on array size. 

### Versions

```shell
System:
    python: 3.7.15 (default, Nov 7 2022, 22:00:21) [GCC 11.2.0]
executable: /home/ray/anaconda3/bin/python
    machine: Linux-5.4.0-1071-aws-x86_64-with_debian-bullseye_sid

Python dependencies:
pip: 23.1
setuptools: 65.6.3
sklearn: 1.0.2
numpy: 1.21.6
scipy: 1.7.3
Cython: 0.29.32
pandas: 1.3.5
matplotlib: 3.5.3
joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

COMMENT:
@Stitaprajna thanks for trying to help. However, we rather avoid super masculine tones in communications happening in our forums. Please defer from phrases such as "Bro, try to update ..." or "Yeah man you are right...". Also, avoid using a commanding language, you can only give people suggestions.

Labels: Bug, Needs Triage
Homogeneity Score is Not Consistently Correct For Trivial Clustering

### Describe the bug

The homogeneity_score is not being computed consistently when you have a single truth label for different array sizes. It seems not to matter how many unique labels are in the predicted labels, just so long as there is only one truth label.  



### Steps/Code to Reproduce

```python
from sklearn.metrics import homogeneity_score
for i in range(1000):
    print(f'For trivial array of length {i}, {homogeneity_score([0]*i,[1]*i)}') 
```

### Expected Results

I would expect this to always be 1.0 for a single truth label, regardless of array size. 


### Actual Results

You will see that the homogeneity score bounces around among -1.0, 0.0, and 1.0 depending on array size. 

### Versions

```shell
System:
    python: 3.7.15 (default, Nov 7 2022, 22:00:21) [GCC 11.2.0]
executable: /home/ray/anaconda3/bin/python
    machine: Linux-5.4.0-1071-aws-x86_64-with_debian-bullseye_sid

Python dependencies:
pip: 23.1
setuptools: 65.6.3
sklearn: 1.0.2
numpy: 1.21.6
scipy: 1.7.3
Cython: 0.29.32
pandas: 1.3.5
matplotlib: 3.5.3
joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

COMMENT:
> how people communicate now-a-days in forums worldwide.

This is incorrect. The Python community, and the scikit-learn community through its code of conduct, strives for inclusive language (https://www.python.org/psf/conduct/). Using gender specific language is not inclusive. To relate to such topic, a good and comprehensive guideline is the United Nations (UN) "gender-inclusive" guideline: https://www.un.org/en/gender-inclusive-language/guidelines.shtml. I could certainly find equivalent recommendation within the PyData ecosystem since this is something that, as a community, we strive for.

> I don't think anyone would have problem with this except you

I therefore disagree and I second @adrinjalali's argument.

Labels: Bug, Needs Triage
⚠️ CI failed on Linux.pylatest_pip_openblas_pandas ⚠️

**CI is still failing on [Linux.pylatest_pip_openblas_pandas](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=59597&view=logs&j=78a0bf4f-79e5-5387-94ec-13e67d216d6e)** (Sep 28, 2023)
- test_kneighbors_brute_backend[float32-manhattan]

COMMENT:
## CI is no longer failing! ✅

[Successful run](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=59802&view=logs&j=78a0bf4f-79e5-5387-94ec-13e67d216d6e) on Oct 03, 2023

Labels: Needs Triage
CI Issues regarding conda lock files

Opening a new issue regarding some of the discussions around https://github.com/scikit-learn/scikit-learn/pull/27448#issuecomment-1733374337

@lesteve these are maybe what I have in mind:

- they're generated files and usually it's a good idea not to have generated files in the repo
- the files are generated, and their generation depends on the exact time when the conda repo data was downloaded in the user's machine. So there's basically no way to review those changes. Which means we have a ton of changed lines which we cannot review.
- since they're generated and we don't review them, we can't really make sure somebody hasn't manually changed anything in the changed file and that the files are indeed generated and not touched afterwards.
- a significant amount of changed lines has been coming from these files
- we are not even consistent on the use of the same snapshot for different CI. We might be using a generated file from last week for circleCI, and one from a month ago for some of the Azure stuff, and one from two weeks ago for other Azure configs.
- we don't really have good docs on how contributors could use the lock files or if they should. Those are only used in the CI, and for that they don't need to be in the same repo.

A few things we could do:
- only allow changes to those files via PRs done by a bot that we write, and generate them periodically, like weekly maybe, and daily for some which change often (like scipy-dev)
- move them out of the main repo.
- we could also move to a docker based system where our CI doesn't have to download and install for each job for each commit. This would also save us a bunch of time, and it would make it even easer to reproduce CI locally.

COMMENT:
> There is nothing to review there: if the lock update is green, then its fine. If it's not green, it means that one of our direct or indirect dependency has introduced a regression, either in scikit-learn or in another dependency of scikit-learn.

You can run arbitrary code by simply installing a malicious package. Not reviewing those changes means a random package can be added to the CI.

> The point is that all of our PRs do not have to be red for a week because something completely unrelated in setuptools or anywhere is causing our CI builds to fail.

I'm not saying we shouldn't be using lock files, or any other means of making CI reproducible. The question is here how to manage those artifacts and where to keep them.

Labels: Build / CI, Needs Decision
CI Issues regarding conda lock files

Opening a new issue regarding some of the discussions around https://github.com/scikit-learn/scikit-learn/pull/27448#issuecomment-1733374337

@lesteve these are maybe what I have in mind:

- they're generated files and usually it's a good idea not to have generated files in the repo
- the files are generated, and their generation depends on the exact time when the conda repo data was downloaded in the user's machine. So there's basically no way to review those changes. Which means we have a ton of changed lines which we cannot review.
- since they're generated and we don't review them, we can't really make sure somebody hasn't manually changed anything in the changed file and that the files are indeed generated and not touched afterwards.
- a significant amount of changed lines has been coming from these files
- we are not even consistent on the use of the same snapshot for different CI. We might be using a generated file from last week for circleCI, and one from a month ago for some of the Azure stuff, and one from two weeks ago for other Azure configs.
- we don't really have good docs on how contributors could use the lock files or if they should. Those are only used in the CI, and for that they don't need to be in the same repo.

A few things we could do:
- only allow changes to those files via PRs done by a bot that we write, and generate them periodically, like weekly maybe, and daily for some which change often (like scipy-dev)
- move them out of the main repo.
- we could also move to a docker based system where our CI doesn't have to download and install for each job for each commit. This would also save us a bunch of time, and it would make it even easer to reproduce CI locally.

COMMENT:
> You can run arbitrary code by simply installing a malicious package. Not reviewing those changes means a random package can be added to the CI.

Re-running the `build_tools/update_environments_and_lock_files.py` script never adds any new direct dependency: it just bumps the version number of existing direct dependencies. And prior to putting the lock files under version control, the indirect dependencies would not even be visible (besides looking at the
pip/mamba install text output in the CI logs which nobody does systematically). Hiding them in another repo or in a Docker container would not help either.

Editing the `build_tools/update_environments_and_lock_files.py` script in PRs such as #27448, we just have to review the changes made to `build_tools/update_environments_and_lock_files.py` itself. The rest is generated by that script. It's good to have a quick look at the output to check that the script works as expected, but there is no point in reviewing each and every line with a change in a package hash value.

Labels: Build / CI, Needs Decision
CI Issues regarding conda lock files

Opening a new issue regarding some of the discussions around https://github.com/scikit-learn/scikit-learn/pull/27448#issuecomment-1733374337

@lesteve these are maybe what I have in mind:

- they're generated files and usually it's a good idea not to have generated files in the repo
- the files are generated, and their generation depends on the exact time when the conda repo data was downloaded in the user's machine. So there's basically no way to review those changes. Which means we have a ton of changed lines which we cannot review.
- since they're generated and we don't review them, we can't really make sure somebody hasn't manually changed anything in the changed file and that the files are indeed generated and not touched afterwards.
- a significant amount of changed lines has been coming from these files
- we are not even consistent on the use of the same snapshot for different CI. We might be using a generated file from last week for circleCI, and one from a month ago for some of the Azure stuff, and one from two weeks ago for other Azure configs.
- we don't really have good docs on how contributors could use the lock files or if they should. Those are only used in the CI, and for that they don't need to be in the same repo.

A few things we could do:
- only allow changes to those files via PRs done by a bot that we write, and generate them periodically, like weekly maybe, and daily for some which change often (like scipy-dev)
- move them out of the main repo.
- we could also move to a docker based system where our CI doesn't have to download and install for each job for each commit. This would also save us a bunch of time, and it would make it even easer to reproduce CI locally.

COMMENT:
We don't have an automated way to make sure the lock file in the PR is actually generated by the script.

Labels: Build / CI, Needs Decision