Talip7/scikit-learn-issues-multilabel huggingface dataset:
log_loss giving nan when input is np.float32 and eps is default

### Describe the bug

When input has values that are numpy array of np.float32, 1-eps (with default eps=1e-15) results in 1.0, and log_loss() when calculating log(1-p) with p=1.0 results in nan.

### Steps/Code to Reproduce

```py
from sklearn.metrics import log_loss
import numpy as np
input = np.array([1],dtype=np.float32)

# when the input is array of np.float32, using the proper eps=eps=np.finfo(np.float32).eps, log_loss is fine
result = log_loss([[0,1]],input,eps=np.finfo(np.float32).eps)
print('with eps=np.finfo(np.float32).eps:',result)

# with input cast as np.float64, log_loss is also fine with the default eps=1e-15
result = log_loss([[0,1]],input.astype(np.float64))
print('with input as np.float64:',result)

# However, the following input as array of np.float32 using the default eps=1e-15 will give nan
result = log_loss([[0,1]],input)
print('with eps=1e-15 (default):',result)
```

### Expected Results

not nan

### Actual Results

with eps=1e-15 (default): nan

```
/Users/gso/anaconda3/lib/python3.9/site-packages/sklearn/metrics/_classification.py:2442: RuntimeWarning: divide by zero encountered in log
  loss = -(transformed_labels * np.log(y_pred)).sum(axis=1)
/Users/gso/anaconda3/lib/python3.9/site-packages/sklearn/metrics/_classification.py:2442: RuntimeWarning: invalid value encountered in multiply
  loss = -(transformed_labels * np.log(y_pred)).sum(axis=1)
```

### Versions

```shell
System:
    python: 3.9.12 (main, Apr  5 2022, 01:53:17)  [Clang 12.0.0 ]
executable: /Users/gso/anaconda3/bin/python
   machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
          pip: 21.2.4
   setuptools: 61.2.0
      sklearn: 1.0.2
        numpy: 1.21.5
        scipy: 1.7.3
       Cython: 0.29.28
       pandas: 1.4.2
   matplotlib: 3.5.1
       joblib: 1.1.0
threadpoolctl: 2.2.0

Built with OpenMP: True
```

COMMENT:
Seems reasonable enough. Would you like to open a PR updating the value to the float32 epsilon (i.e. `np.finfo(np.float32).eps`)?

Labels: Bug, help wanted
log_loss giving nan when input is np.float32 and eps is default

### Describe the bug

When input has values that are numpy array of np.float32, 1-eps (with default eps=1e-15) results in 1.0, and log_loss() when calculating log(1-p) with p=1.0 results in nan.

### Steps/Code to Reproduce

```py
from sklearn.metrics import log_loss
import numpy as np
input = np.array([1],dtype=np.float32)

# when the input is array of np.float32, using the proper eps=eps=np.finfo(np.float32).eps, log_loss is fine
result = log_loss([[0,1]],input,eps=np.finfo(np.float32).eps)
print('with eps=np.finfo(np.float32).eps:',result)

# with input cast as np.float64, log_loss is also fine with the default eps=1e-15
result = log_loss([[0,1]],input.astype(np.float64))
print('with input as np.float64:',result)

# However, the following input as array of np.float32 using the default eps=1e-15 will give nan
result = log_loss([[0,1]],input)
print('with eps=1e-15 (default):',result)
```

### Expected Results

not nan

### Actual Results

with eps=1e-15 (default): nan

```
/Users/gso/anaconda3/lib/python3.9/site-packages/sklearn/metrics/_classification.py:2442: RuntimeWarning: divide by zero encountered in log
  loss = -(transformed_labels * np.log(y_pred)).sum(axis=1)
/Users/gso/anaconda3/lib/python3.9/site-packages/sklearn/metrics/_classification.py:2442: RuntimeWarning: invalid value encountered in multiply
  loss = -(transformed_labels * np.log(y_pred)).sum(axis=1)
```

### Versions

```shell
System:
    python: 3.9.12 (main, Apr  5 2022, 01:53:17)  [Clang 12.0.0 ]
executable: /Users/gso/anaconda3/bin/python
   machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
          pip: 21.2.4
   setuptools: 61.2.0
      sklearn: 1.0.2
        numpy: 1.21.5
        scipy: 1.7.3
       Cython: 0.29.28
       pandas: 1.4.2
   matplotlib: 3.5.1
       joblib: 1.1.0
threadpoolctl: 2.2.0

Built with OpenMP: True
```

COMMENT:
Hi, Can I take this?

Labels: Bug, help wanted
log_loss giving nan when input is np.float32 and eps is default

### Describe the bug

When input has values that are numpy array of np.float32, 1-eps (with default eps=1e-15) results in 1.0, and log_loss() when calculating log(1-p) with p=1.0 results in nan.

### Steps/Code to Reproduce

```py
from sklearn.metrics import log_loss
import numpy as np
input = np.array([1],dtype=np.float32)

# when the input is array of np.float32, using the proper eps=eps=np.finfo(np.float32).eps, log_loss is fine
result = log_loss([[0,1]],input,eps=np.finfo(np.float32).eps)
print('with eps=np.finfo(np.float32).eps:',result)

# with input cast as np.float64, log_loss is also fine with the default eps=1e-15
result = log_loss([[0,1]],input.astype(np.float64))
print('with input as np.float64:',result)

# However, the following input as array of np.float32 using the default eps=1e-15 will give nan
result = log_loss([[0,1]],input)
print('with eps=1e-15 (default):',result)
```

### Expected Results

not nan

### Actual Results

with eps=1e-15 (default): nan

```
/Users/gso/anaconda3/lib/python3.9/site-packages/sklearn/metrics/_classification.py:2442: RuntimeWarning: divide by zero encountered in log
  loss = -(transformed_labels * np.log(y_pred)).sum(axis=1)
/Users/gso/anaconda3/lib/python3.9/site-packages/sklearn/metrics/_classification.py:2442: RuntimeWarning: invalid value encountered in multiply
  loss = -(transformed_labels * np.log(y_pred)).sum(axis=1)
```

### Versions

```shell
System:
    python: 3.9.12 (main, Apr  5 2022, 01:53:17)  [Clang 12.0.0 ]
executable: /Users/gso/anaconda3/bin/python
   machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
          pip: 21.2.4
   setuptools: 61.2.0
      sklearn: 1.0.2
        numpy: 1.21.5
        scipy: 1.7.3
       Cython: 0.29.28
       pandas: 1.4.2
   matplotlib: 3.5.1
       joblib: 1.1.0
threadpoolctl: 2.2.0

Built with OpenMP: True
```

COMMENT:
> Hi, Can I take this?

Yes, go ahead :)

Labels: Bug, help wanted
log_loss giving nan when input is np.float32 and eps is default

### Describe the bug

When input has values that are numpy array of np.float32, 1-eps (with default eps=1e-15) results in 1.0, and log_loss() when calculating log(1-p) with p=1.0 results in nan.

### Steps/Code to Reproduce

```py
from sklearn.metrics import log_loss
import numpy as np
input = np.array([1],dtype=np.float32)

# when the input is array of np.float32, using the proper eps=eps=np.finfo(np.float32).eps, log_loss is fine
result = log_loss([[0,1]],input,eps=np.finfo(np.float32).eps)
print('with eps=np.finfo(np.float32).eps:',result)

# with input cast as np.float64, log_loss is also fine with the default eps=1e-15
result = log_loss([[0,1]],input.astype(np.float64))
print('with input as np.float64:',result)

# However, the following input as array of np.float32 using the default eps=1e-15 will give nan
result = log_loss([[0,1]],input)
print('with eps=1e-15 (default):',result)
```

### Expected Results

not nan

### Actual Results

with eps=1e-15 (default): nan

```
/Users/gso/anaconda3/lib/python3.9/site-packages/sklearn/metrics/_classification.py:2442: RuntimeWarning: divide by zero encountered in log
  loss = -(transformed_labels * np.log(y_pred)).sum(axis=1)
/Users/gso/anaconda3/lib/python3.9/site-packages/sklearn/metrics/_classification.py:2442: RuntimeWarning: invalid value encountered in multiply
  loss = -(transformed_labels * np.log(y_pred)).sum(axis=1)
```

### Versions

```shell
System:
    python: 3.9.12 (main, Apr  5 2022, 01:53:17)  [Clang 12.0.0 ]
executable: /Users/gso/anaconda3/bin/python
   machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
          pip: 21.2.4
   setuptools: 61.2.0
      sklearn: 1.0.2
        numpy: 1.21.5
        scipy: 1.7.3
       Cython: 0.29.28
       pandas: 1.4.2
   matplotlib: 3.5.1
       joblib: 1.1.0
threadpoolctl: 2.2.0

Built with OpenMP: True
```

COMMENT:
@Micky774  We would face the same issue, if we give a default epsilon of np.finfo(np.float32).eps and the user provides a float16 input. 
So, should that be handled as well by having a different eps value for every type? Or going for float16 epsilon by default as that would be the largest epsilon?

Labels: Bug, help wanted
log_loss giving nan when input is np.float32 and eps is default

### Describe the bug

When input has values that are numpy array of np.float32, 1-eps (with default eps=1e-15) results in 1.0, and log_loss() when calculating log(1-p) with p=1.0 results in nan.

### Steps/Code to Reproduce

```py
from sklearn.metrics import log_loss
import numpy as np
input = np.array([1],dtype=np.float32)

# when the input is array of np.float32, using the proper eps=eps=np.finfo(np.float32).eps, log_loss is fine
result = log_loss([[0,1]],input,eps=np.finfo(np.float32).eps)
print('with eps=np.finfo(np.float32).eps:',result)

# with input cast as np.float64, log_loss is also fine with the default eps=1e-15
result = log_loss([[0,1]],input.astype(np.float64))
print('with input as np.float64:',result)

# However, the following input as array of np.float32 using the default eps=1e-15 will give nan
result = log_loss([[0,1]],input)
print('with eps=1e-15 (default):',result)
```

### Expected Results

not nan

### Actual Results

with eps=1e-15 (default): nan

```
/Users/gso/anaconda3/lib/python3.9/site-packages/sklearn/metrics/_classification.py:2442: RuntimeWarning: divide by zero encountered in log
  loss = -(transformed_labels * np.log(y_pred)).sum(axis=1)
/Users/gso/anaconda3/lib/python3.9/site-packages/sklearn/metrics/_classification.py:2442: RuntimeWarning: invalid value encountered in multiply
  loss = -(transformed_labels * np.log(y_pred)).sum(axis=1)
```

### Versions

```shell
System:
    python: 3.9.12 (main, Apr  5 2022, 01:53:17)  [Clang 12.0.0 ]
executable: /Users/gso/anaconda3/bin/python
   machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
          pip: 21.2.4
   setuptools: 61.2.0
      sklearn: 1.0.2
        numpy: 1.21.5
        scipy: 1.7.3
       Cython: 0.29.28
       pandas: 1.4.2
   matplotlib: 3.5.1
       joblib: 1.1.0
threadpoolctl: 2.2.0

Built with OpenMP: True
```

COMMENT:
> @Micky774  We would face the same issue, if we give a default epsilon of np.finfo(np.float32).eps and the user provides a float16 input. 
> So, should that be handled as well by having a different eps value for every type? Or going for float16 epsilon by default as that would be the largest epsilon?

~Afaik we don't generally support FP16 and hence it would be preferable to keep with FP32 epsilon.~

Edit: The `'auto'` option mentioned below handles this well.

Labels: Bug, help wanted
log_loss giving nan when input is np.float32 and eps is default

### Describe the bug

When input has values that are numpy array of np.float32, 1-eps (with default eps=1e-15) results in 1.0, and log_loss() when calculating log(1-p) with p=1.0 results in nan.

### Steps/Code to Reproduce

```py
from sklearn.metrics import log_loss
import numpy as np
input = np.array([1],dtype=np.float32)

# when the input is array of np.float32, using the proper eps=eps=np.finfo(np.float32).eps, log_loss is fine
result = log_loss([[0,1]],input,eps=np.finfo(np.float32).eps)
print('with eps=np.finfo(np.float32).eps:',result)

# with input cast as np.float64, log_loss is also fine with the default eps=1e-15
result = log_loss([[0,1]],input.astype(np.float64))
print('with input as np.float64:',result)

# However, the following input as array of np.float32 using the default eps=1e-15 will give nan
result = log_loss([[0,1]],input)
print('with eps=1e-15 (default):',result)
```

### Expected Results

not nan

### Actual Results

with eps=1e-15 (default): nan

```
/Users/gso/anaconda3/lib/python3.9/site-packages/sklearn/metrics/_classification.py:2442: RuntimeWarning: divide by zero encountered in log
  loss = -(transformed_labels * np.log(y_pred)).sum(axis=1)
/Users/gso/anaconda3/lib/python3.9/site-packages/sklearn/metrics/_classification.py:2442: RuntimeWarning: invalid value encountered in multiply
  loss = -(transformed_labels * np.log(y_pred)).sum(axis=1)
```

### Versions

```shell
System:
    python: 3.9.12 (main, Apr  5 2022, 01:53:17)  [Clang 12.0.0 ]
executable: /Users/gso/anaconda3/bin/python
   machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
          pip: 21.2.4
   setuptools: 61.2.0
      sklearn: 1.0.2
        numpy: 1.21.5
        scipy: 1.7.3
       Cython: 0.29.28
       pandas: 1.4.2
   matplotlib: 3.5.1
       joblib: 1.1.0
threadpoolctl: 2.2.0

Built with OpenMP: True
```

COMMENT:
pull request: https://github.com/scikit-learn/scikit-learn/pull/24357

- added check to see if input is np.float32 or np.float16
- if detected, change eps to satisfy input precision
- added documentation describing the change
- added warning message to user when this happens

The change @Safikh  proposed would fix my immediate problem (Thank you!), but...
- As Safikh alluded to, setting a fixed new default does not account for np.float16 (may not be an issue if everyone knows sklearn doesn't support float16, but I doubt it's common knowledge)
- would decrease the accuracy drastically if input is just a list of numbers, because internally they are treated as float64 by python
- I would leave the default as is, and just let the user know when we dynamically change eps due to input lacking 64bit precision

Labels: Bug, help wanted
log_loss giving nan when input is np.float32 and eps is default

### Describe the bug

When input has values that are numpy array of np.float32, 1-eps (with default eps=1e-15) results in 1.0, and log_loss() when calculating log(1-p) with p=1.0 results in nan.

### Steps/Code to Reproduce

```py
from sklearn.metrics import log_loss
import numpy as np
input = np.array([1],dtype=np.float32)

# when the input is array of np.float32, using the proper eps=eps=np.finfo(np.float32).eps, log_loss is fine
result = log_loss([[0,1]],input,eps=np.finfo(np.float32).eps)
print('with eps=np.finfo(np.float32).eps:',result)

# with input cast as np.float64, log_loss is also fine with the default eps=1e-15
result = log_loss([[0,1]],input.astype(np.float64))
print('with input as np.float64:',result)

# However, the following input as array of np.float32 using the default eps=1e-15 will give nan
result = log_loss([[0,1]],input)
print('with eps=1e-15 (default):',result)
```

### Expected Results

not nan

### Actual Results

with eps=1e-15 (default): nan

```
/Users/gso/anaconda3/lib/python3.9/site-packages/sklearn/metrics/_classification.py:2442: RuntimeWarning: divide by zero encountered in log
  loss = -(transformed_labels * np.log(y_pred)).sum(axis=1)
/Users/gso/anaconda3/lib/python3.9/site-packages/sklearn/metrics/_classification.py:2442: RuntimeWarning: invalid value encountered in multiply
  loss = -(transformed_labels * np.log(y_pred)).sum(axis=1)
```

### Versions

```shell
System:
    python: 3.9.12 (main, Apr  5 2022, 01:53:17)  [Clang 12.0.0 ]
executable: /Users/gso/anaconda3/bin/python
   machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
          pip: 21.2.4
   setuptools: 61.2.0
      sklearn: 1.0.2
        numpy: 1.21.5
        scipy: 1.7.3
       Cython: 0.29.28
       pandas: 1.4.2
   matplotlib: 3.5.1
       joblib: 1.1.0
threadpoolctl: 2.2.0

Built with OpenMP: True
```

COMMENT:
Not sure how to fix this in the pull request
<img width="451" alt="Screen Shot 2022-09-04 at 5 16 02 PM" src="https://user-images.githubusercontent.com/19766295/188339179-4a38df02-a854-47d3-beeb-8dc03e6d75b1.png">

Labels: Bug, help wanted
log_loss giving nan when input is np.float32 and eps is default

### Describe the bug

When input has values that are numpy array of np.float32, 1-eps (with default eps=1e-15) results in 1.0, and log_loss() when calculating log(1-p) with p=1.0 results in nan.

### Steps/Code to Reproduce

```py
from sklearn.metrics import log_loss
import numpy as np
input = np.array([1],dtype=np.float32)

# when the input is array of np.float32, using the proper eps=eps=np.finfo(np.float32).eps, log_loss is fine
result = log_loss([[0,1]],input,eps=np.finfo(np.float32).eps)
print('with eps=np.finfo(np.float32).eps:',result)

# with input cast as np.float64, log_loss is also fine with the default eps=1e-15
result = log_loss([[0,1]],input.astype(np.float64))
print('with input as np.float64:',result)

# However, the following input as array of np.float32 using the default eps=1e-15 will give nan
result = log_loss([[0,1]],input)
print('with eps=1e-15 (default):',result)
```

### Expected Results

not nan

### Actual Results

with eps=1e-15 (default): nan

```
/Users/gso/anaconda3/lib/python3.9/site-packages/sklearn/metrics/_classification.py:2442: RuntimeWarning: divide by zero encountered in log
  loss = -(transformed_labels * np.log(y_pred)).sum(axis=1)
/Users/gso/anaconda3/lib/python3.9/site-packages/sklearn/metrics/_classification.py:2442: RuntimeWarning: invalid value encountered in multiply
  loss = -(transformed_labels * np.log(y_pred)).sum(axis=1)
```

### Versions

```shell
System:
    python: 3.9.12 (main, Apr  5 2022, 01:53:17)  [Clang 12.0.0 ]
executable: /Users/gso/anaconda3/bin/python
   machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
          pip: 21.2.4
   setuptools: 61.2.0
      sklearn: 1.0.2
        numpy: 1.21.5
        scipy: 1.7.3
       Cython: 0.29.28
       pandas: 1.4.2
   matplotlib: 3.5.1
       joblib: 1.1.0
threadpoolctl: 2.2.0

Built with OpenMP: True
```

COMMENT:
Just mentioning what @ogrisel proposed in one of the PR.

It would be better to introduce a `"auto"` solution that will switch depending on the dtype using: `np.finfo(y_pred.dtype).eps`.

Labels: Bug, help wanted
Coordinate with other scientific python projects to use shared tools and policy to upload nightly build wheels

See the motivation in this discussion: https://discuss.scientific-python.org/t/interest-in-github-action-for-scipy-wheels-nightly-uploads-and-removals/397

- Draft shared Github Actions config: https://github.com/matthewfeickert/example-nightly-wheels
- New github dedicated org/repo: https://github.com/scientific-python/nightly-wheels/

COMMENT:
For reference, at the moment, we configure the upload of the nightly builds in this config file:

- https://github.com/scikit-learn/scikit-learn/blob/main/.github/workflows/wheels.yml#L202-L228

that calls this script:

- https://github.com/scikit-learn/scikit-learn/blob/main/build_tools/github/upload_anaconda.sh

Labels: Build / CI, help wanted
Coordinate with other scientific python projects to use shared tools and policy to upload nightly build wheels

See the motivation in this discussion: https://discuss.scientific-python.org/t/interest-in-github-action-for-scipy-wheels-nightly-uploads-and-removals/397

- Draft shared Github Actions config: https://github.com/matthewfeickert/example-nightly-wheels
- New github dedicated org/repo: https://github.com/scientific-python/nightly-wheels/

COMMENT:
@ogrisel, may I ask what "help wanted" means in this specific case?
If I understand correctly https://github.com/scientific-python/upload-nightly-action/issues/6 is now the issue looking for contributors. This issue could stay open as reference to that one.... is that right?

Labels: Build / CI, help wanted
Coordinate with other scientific python projects to use shared tools and policy to upload nightly build wheels

See the motivation in this discussion: https://discuss.scientific-python.org/t/interest-in-github-action-for-scipy-wheels-nightly-uploads-and-removals/397

- Draft shared Github Actions config: https://github.com/matthewfeickert/example-nightly-wheels
- New github dedicated org/repo: https://github.com/scientific-python/nightly-wheels/

COMMENT:
> @ogrisel, may I ask what "help wanted" means in this specific case?

To attract potential contributors who would be interested in contributing to the underlying tools or to update our CI setup to use the shared tools.

Labels: Build / CI, help wanted
Coordinate with other scientific python projects to use shared tools and policy to upload nightly build wheels

See the motivation in this discussion: https://discuss.scientific-python.org/t/interest-in-github-action-for-scipy-wheels-nightly-uploads-and-removals/397

- Draft shared Github Actions config: https://github.com/matthewfeickert/example-nightly-wheels
- New github dedicated org/repo: https://github.com/scientific-python/nightly-wheels/

COMMENT:
> * New github dedicated org/repo: https://github.com/scientific-python/nightly-wheels/

This repo got deleted (c.f. https://github.com/scientific-python/upload-nightly-action/issues/5) so please use https://github.com/scientific-python/upload-nightly-action now.

Labels: Build / CI, help wanted
Coordinate with other scientific python projects to use shared tools and policy to upload nightly build wheels

See the motivation in this discussion: https://discuss.scientific-python.org/t/interest-in-github-action-for-scipy-wheels-nightly-uploads-and-removals/397

- Draft shared Github Actions config: https://github.com/matthewfeickert/example-nightly-wheels
- New github dedicated org/repo: https://github.com/scientific-python/nightly-wheels/

COMMENT:
I guess we have been doing it for some time and this can be closed.

Labels: Build / CI, help wanted
AttributeError: 'NoneType' object has no attribute 'copy'

I am using natureInspiredSearchcv for tuning randomForest classifier, while tuning min_impurity_decrease I am getting following error, May I have some help please
```
grid_result = grid.fit(X_trainf32, y_train)
  File "C:\Users\pg\anaconda3\envs\autoMLpy395TestEnv\lib\site-packages\sklearn\utils\validation.py", line 63, in inner_f
    return f(*args, **kwargs)
  File "C:\Users\pg\anaconda3\envs\autoMLpy395TestEnv\lib\site-packages\sklearn\model_selection\_search.py", line 841, in fit
    self._run_search(evaluate_candidates)
  File "C:\Users\pg\anaconda3\envs\autoMLpy395TestEnv\lib\site-packages\sklearn_nature_inspired_algorithms\model_selection\nature_inspired_search_cv.py", line 46, in _run_search
    raise self.__algorithm.exception
  File "C:\Users\pg\anaconda3\envs\autoMLpy395TestEnv\lib\site-packages\niapy\algorithms\algorithm.py", line 353, in run
    r = self.run_task(task)
  File "C:\Users\pg\anaconda3\envs\autoMLpy395TestEnv\lib\site-packages\niapy\algorithms\algorithm.py", line 333, in run_task
    xb, fxb = next(algo)
  File "C:\Users\pg\anaconda3\envs\autoMLpy395TestEnv\lib\site-packages\niapy\algorithms\algorithm.py", line 309, in iteration_generator
    xb, fxb = self.get_best(pop, fpop)
  File "C:\Users\pg\anaconda3\envs\autoMLpy395TestEnv\lib\site-packages\niapy\algorithms\algorithm.py", line 240, in get_best
    return (best_x.x.copy() if isinstance(best_x, Individual) else best_x.copy()), best_fitness
AttributeError: 'NoneType' object has no attribute 'copy'
```
Is this due to randomForest classifier parameters value issue, like min_impurity_decrease not getting proper values

COMMENT:
Yes, It is due to randomForest classifier parameters value issue, 
min_impurity_decrease was tuned separately
Thanks

Labels: Needs Triage