Talip7/scikit-learn-issues-multilabel huggingface dataset:
Wrong behaviour when calculating `f1_score` with `zero_division=1`

### Describe the bug

It seems that in the new version 1.3 `f1_score` with `zero_division=1` incorrectly calculates the results. It treats the prediction with 0 true positives as a division by zero and assigned 1. This was not the case in 1.2.X version and below, which were triggering division by zero only when true positives, false negatives, and false positives were 0. The example below demonstrates the issue. This bug may result in over-positive estimates, especially when using `macro` averaging in multi-label classification.

### Steps/Code to Reproduce

```
y_true = np.array([0, 1])
y_pred = np.array([1, 0])
print(f1_score(y_true, y_pred, zero_division=1)) # Should be 0.0

y_true = np.array([0, 1])
y_pred = np.array([0, 1])
print(f1_score(y_true, y_pred, zero_division=1)) # Should be 1.0

y_true = np.array([0, 1])
y_pred = np.array([0, 0])
print(f1_score(y_true, y_pred, zero_division=1)) # Should be 0.0

y_true = np.array([0, 0])
y_pred = np.array([0, 0])
print(f1_score(y_true, y_pred, zero_division=1)) # Here division by zero should be triggered resulting in 1.0
```

### Expected Results

In versions 1.2.X and below, the correct results are printed, that is `0.0, 1.0, 0.0, 1.0`,

### Actual Results

The new version 1.3 prints `1.0, 1.0, 0.0, 1.0` - the first value is incorrect.

### Versions

```shell
System:
    python: 3.10.6 (main, May 29 2023, 11:10:38) [GCC 11.3.0]
executable: /usr/bin/python
   machine: Linux-5.19.0-50-generic-x86_64-with-glibc2.35

Python dependencies:
      sklearn: 1.3.0
          pip: 22.0.2
   setuptools: 59.6.0
        numpy: 1.25.1
        scipy: 1.10.0
       Cython: None
       pandas: 1.5.3
   matplotlib: 3.7.1
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
         prefix: libgomp
       filepath: /home/marek/.local/lib/python3.10/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
    num_threads: 16

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /usr/local/lib/python3.10/dist-packages/numpy.libs/libopenblas64_p-r0-7a851222.3.23.so
        version: 0.3.23
threading_layer: pthreads
   architecture: Zen
    num_threads: 16

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /home/marek/.local/lib/python3.10/site-packages/scipy.libs/libopenblasp-r0-41284840.3.18.so
        version: 0.3.18
threading_layer: pthreads
   architecture: Zen
    num_threads: 16
```

COMMENT:
/take

Labels: Bug, help wanted
Path to adopt 32-bit implementations for `{KD, Ball}Tree`

## Motivation
Having and using 32-bit implementations of `{KD, Ball}Tree` allows for better preservation of dtype, lower memory footprint, and more consistent Cython code.

## Strategy
Work has already been started in #25914 to add the code for the new 32-bit implementations. These will not be directly used yet, and the PR instead focuses on the actual creation of the new classes. Consequently, `{KD, Ball}Tree` are bound to `{KD, Ball}Tree64` for consistency and backwards compatibility.

Following this PR, we will need to begin an API deprecation to move users away from constructing trees directly, and instead using a factory method (similar to `DistanceMetric.get_metric`). Then, later, we can separate `{KD, Ball}Tree` from `{KD, Ball}Tree64` so that we have a singular dispatcher `{KD, Ball}Tree` and two type-specialized implementations. 

The deprecation process involves:
1. Introducing `get_tree(...)` to `{KD, Ball}Tree64` with the same signature of `BinaryTree.__init__`, which is in charge of constructing the type-specialized trees directly.
2. Adding a `FutureWarning` to `BinaryTree.__init__` to begin deprecation, and suppressing the warning using a context manager in `{KD, Ball}Tree64.get_tree` to enforce it as the "correct" way to construct the trees.
3. **Complete the deprecation** by introducing a separate `{KD, Ball}Tree` into the hierarchy which comes with the same `get_tree` method and fully remove the `FutureWarning` from `{KD, Ball}Tree{32, 64}.__init__` since they should no longer be directly constructed, as well as getting rid of the `{KD, Ball}Tree{32, 64}.get_tree` method since it should only exist in `{KD, Ball}Tree`.

At the end, the expected API is:
```python
from sklearn.neighbors import KDTree

# Data
X = ...

# tree is thus an instance of KDTree{32, 64} based on X.dtype
tree = KDTree.get_tree(X, ...)
```

cc: @thomasjpfan @jjerphan @OmarManzoor

COMMENT:
@Micky774 Thanks a lot for creating this issue.

Labels: API, Needs Decision, RFC
Path to adopt 32-bit implementations for `{KD, Ball}Tree`

## Motivation
Having and using 32-bit implementations of `{KD, Ball}Tree` allows for better preservation of dtype, lower memory footprint, and more consistent Cython code.

## Strategy
Work has already been started in #25914 to add the code for the new 32-bit implementations. These will not be directly used yet, and the PR instead focuses on the actual creation of the new classes. Consequently, `{KD, Ball}Tree` are bound to `{KD, Ball}Tree64` for consistency and backwards compatibility.

Following this PR, we will need to begin an API deprecation to move users away from constructing trees directly, and instead using a factory method (similar to `DistanceMetric.get_metric`). Then, later, we can separate `{KD, Ball}Tree` from `{KD, Ball}Tree64` so that we have a singular dispatcher `{KD, Ball}Tree` and two type-specialized implementations. 

The deprecation process involves:
1. Introducing `get_tree(...)` to `{KD, Ball}Tree64` with the same signature of `BinaryTree.__init__`, which is in charge of constructing the type-specialized trees directly.
2. Adding a `FutureWarning` to `BinaryTree.__init__` to begin deprecation, and suppressing the warning using a context manager in `{KD, Ball}Tree64.get_tree` to enforce it as the "correct" way to construct the trees.
3. **Complete the deprecation** by introducing a separate `{KD, Ball}Tree` into the hierarchy which comes with the same `get_tree` method and fully remove the `FutureWarning` from `{KD, Ball}Tree{32, 64}.__init__` since they should no longer be directly constructed, as well as getting rid of the `{KD, Ball}Tree{32, 64}.get_tree` method since it should only exist in `{KD, Ball}Tree`.

At the end, the expected API is:
```python
from sklearn.neighbors import KDTree

# Data
X = ...

# tree is thus an instance of KDTree{32, 64} based on X.dtype
tree = KDTree.get_tree(X, ...)
```

cc: @thomasjpfan @jjerphan @OmarManzoor

COMMENT:
Thank you for creating the PR, @Micky774.

I think there are alternatives which would allow not introducing any new API while supporting both implementations independently from the existing API.

I do not have time to write them now, but I should be able to soon.

Labels: API, Needs Decision, RFC
Path to adopt 32-bit implementations for `{KD, Ball}Tree`

## Motivation
Having and using 32-bit implementations of `{KD, Ball}Tree` allows for better preservation of dtype, lower memory footprint, and more consistent Cython code.

## Strategy
Work has already been started in #25914 to add the code for the new 32-bit implementations. These will not be directly used yet, and the PR instead focuses on the actual creation of the new classes. Consequently, `{KD, Ball}Tree` are bound to `{KD, Ball}Tree64` for consistency and backwards compatibility.

Following this PR, we will need to begin an API deprecation to move users away from constructing trees directly, and instead using a factory method (similar to `DistanceMetric.get_metric`). Then, later, we can separate `{KD, Ball}Tree` from `{KD, Ball}Tree64` so that we have a singular dispatcher `{KD, Ball}Tree` and two type-specialized implementations. 

The deprecation process involves:
1. Introducing `get_tree(...)` to `{KD, Ball}Tree64` with the same signature of `BinaryTree.__init__`, which is in charge of constructing the type-specialized trees directly.
2. Adding a `FutureWarning` to `BinaryTree.__init__` to begin deprecation, and suppressing the warning using a context manager in `{KD, Ball}Tree64.get_tree` to enforce it as the "correct" way to construct the trees.
3. **Complete the deprecation** by introducing a separate `{KD, Ball}Tree` into the hierarchy which comes with the same `get_tree` method and fully remove the `FutureWarning` from `{KD, Ball}Tree{32, 64}.__init__` since they should no longer be directly constructed, as well as getting rid of the `{KD, Ball}Tree{32, 64}.get_tree` method since it should only exist in `{KD, Ball}Tree`.

At the end, the expected API is:
```python
from sklearn.neighbors import KDTree

# Data
X = ...

# tree is thus an instance of KDTree{32, 64} based on X.dtype
tree = KDTree.get_tree(X, ...)
```

cc: @thomasjpfan @jjerphan @OmarManzoor

COMMENT:
I think that introducing a public `{KD,Ball}Tree.get_tree` class method would be inconsistent with the current and overall design estimator creation. To me, we must stick to only one interface to create a `{KD,Ball}Tree`: their constructor.

Would it be possible to have something similar to what exists for `DecisionTree`? That is:
 - keep the current `{KD,Ball}Tree` public API unchanged
 - have a `{KD,Ball}Tree{32,64}` implementation be a private `_tree` attribute for `{KD,Ball}Tree` when it is being constructed based on the type of `X`
 - forward calls on methods of `{KD,Ball}Tree` to their `_tree` attribute's

I think this would allow keeping the UX of `{KD,Ball}Tree` unchanged while supporting float32 data.

Yet, I am not against having a private class method if we need to have typing for `{KD,Ball}Tree{32,64}` in Cython code without instantiating instances of the public `{KD,Ball}Tree` classes.

What do you think?

Labels: API, Needs Decision, RFC
Path to adopt 32-bit implementations for `{KD, Ball}Tree`

## Motivation
Having and using 32-bit implementations of `{KD, Ball}Tree` allows for better preservation of dtype, lower memory footprint, and more consistent Cython code.

## Strategy
Work has already been started in #25914 to add the code for the new 32-bit implementations. These will not be directly used yet, and the PR instead focuses on the actual creation of the new classes. Consequently, `{KD, Ball}Tree` are bound to `{KD, Ball}Tree64` for consistency and backwards compatibility.

Following this PR, we will need to begin an API deprecation to move users away from constructing trees directly, and instead using a factory method (similar to `DistanceMetric.get_metric`). Then, later, we can separate `{KD, Ball}Tree` from `{KD, Ball}Tree64` so that we have a singular dispatcher `{KD, Ball}Tree` and two type-specialized implementations. 

The deprecation process involves:
1. Introducing `get_tree(...)` to `{KD, Ball}Tree64` with the same signature of `BinaryTree.__init__`, which is in charge of constructing the type-specialized trees directly.
2. Adding a `FutureWarning` to `BinaryTree.__init__` to begin deprecation, and suppressing the warning using a context manager in `{KD, Ball}Tree64.get_tree` to enforce it as the "correct" way to construct the trees.
3. **Complete the deprecation** by introducing a separate `{KD, Ball}Tree` into the hierarchy which comes with the same `get_tree` method and fully remove the `FutureWarning` from `{KD, Ball}Tree{32, 64}.__init__` since they should no longer be directly constructed, as well as getting rid of the `{KD, Ball}Tree{32, 64}.get_tree` method since it should only exist in `{KD, Ball}Tree`.

At the end, the expected API is:
```python
from sklearn.neighbors import KDTree

# Data
X = ...

# tree is thus an instance of KDTree{32, 64} based on X.dtype
tree = KDTree.get_tree(X, ...)
```

cc: @thomasjpfan @jjerphan @OmarManzoor

COMMENT:
> I like the use of __new__, but it returns instances that leak implementations details; as you have shown: ...
> I think we must prevent public API from leaking such implementation details to users.

I don't personally consider this a negative aspect of the solution. What situations does this risk leading to that we would prefer avoiding? I'm open to revising my opinion on this, but as a developer and user, I personally enjoy having this information available as it makes debugging easier and more direct.

> I would wait for a few other maintainers to give their points of view before starting an implementation. What do you think?

Agreed. I think this is opinionated enough that I don't want to haphazardly move forwards without considering more perspectives.

Labels: API, Needs Decision, RFC
Path to adopt 32-bit implementations for `{KD, Ball}Tree`

## Motivation
Having and using 32-bit implementations of `{KD, Ball}Tree` allows for better preservation of dtype, lower memory footprint, and more consistent Cython code.

## Strategy
Work has already been started in #25914 to add the code for the new 32-bit implementations. These will not be directly used yet, and the PR instead focuses on the actual creation of the new classes. Consequently, `{KD, Ball}Tree` are bound to `{KD, Ball}Tree64` for consistency and backwards compatibility.

Following this PR, we will need to begin an API deprecation to move users away from constructing trees directly, and instead using a factory method (similar to `DistanceMetric.get_metric`). Then, later, we can separate `{KD, Ball}Tree` from `{KD, Ball}Tree64` so that we have a singular dispatcher `{KD, Ball}Tree` and two type-specialized implementations. 

The deprecation process involves:
1. Introducing `get_tree(...)` to `{KD, Ball}Tree64` with the same signature of `BinaryTree.__init__`, which is in charge of constructing the type-specialized trees directly.
2. Adding a `FutureWarning` to `BinaryTree.__init__` to begin deprecation, and suppressing the warning using a context manager in `{KD, Ball}Tree64.get_tree` to enforce it as the "correct" way to construct the trees.
3. **Complete the deprecation** by introducing a separate `{KD, Ball}Tree` into the hierarchy which comes with the same `get_tree` method and fully remove the `FutureWarning` from `{KD, Ball}Tree{32, 64}.__init__` since they should no longer be directly constructed, as well as getting rid of the `{KD, Ball}Tree{32, 64}.get_tree` method since it should only exist in `{KD, Ball}Tree`.

At the end, the expected API is:
```python
from sklearn.neighbors import KDTree

# Data
X = ...

# tree is thus an instance of KDTree{32, 64} based on X.dtype
tree = KDTree.get_tree(X, ...)
```

cc: @thomasjpfan @jjerphan @OmarManzoor

COMMENT:
@scikit-learn/core-devs We need a few additional perspectives here before we can continue.

Labels: API, Needs Decision, RFC
Path to adopt 32-bit implementations for `{KD, Ball}Tree`

## Motivation
Having and using 32-bit implementations of `{KD, Ball}Tree` allows for better preservation of dtype, lower memory footprint, and more consistent Cython code.

## Strategy
Work has already been started in #25914 to add the code for the new 32-bit implementations. These will not be directly used yet, and the PR instead focuses on the actual creation of the new classes. Consequently, `{KD, Ball}Tree` are bound to `{KD, Ball}Tree64` for consistency and backwards compatibility.

Following this PR, we will need to begin an API deprecation to move users away from constructing trees directly, and instead using a factory method (similar to `DistanceMetric.get_metric`). Then, later, we can separate `{KD, Ball}Tree` from `{KD, Ball}Tree64` so that we have a singular dispatcher `{KD, Ball}Tree` and two type-specialized implementations. 

The deprecation process involves:
1. Introducing `get_tree(...)` to `{KD, Ball}Tree64` with the same signature of `BinaryTree.__init__`, which is in charge of constructing the type-specialized trees directly.
2. Adding a `FutureWarning` to `BinaryTree.__init__` to begin deprecation, and suppressing the warning using a context manager in `{KD, Ball}Tree64.get_tree` to enforce it as the "correct" way to construct the trees.
3. **Complete the deprecation** by introducing a separate `{KD, Ball}Tree` into the hierarchy which comes with the same `get_tree` method and fully remove the `FutureWarning` from `{KD, Ball}Tree{32, 64}.__init__` since they should no longer be directly constructed, as well as getting rid of the `{KD, Ball}Tree{32, 64}.get_tree` method since it should only exist in `{KD, Ball}Tree`.

At the end, the expected API is:
```python
from sklearn.neighbors import KDTree

# Data
X = ...

# tree is thus an instance of KDTree{32, 64} based on X.dtype
tree = KDTree.get_tree(X, ...)
```

cc: @thomasjpfan @jjerphan @OmarManzoor

COMMENT:
From the discussion, I see two options:

1. Public `KDTree` and `BallTree` and keep the 32bit and 64bit implementations private.
2. Introduce `KDTree.get_tree` and `BallTree.get_tree` and make the 32bit and 64bit implementations public.

In `DistanceMetric`, we decided to go with option 2 with `get_metric`, because of performance reasons. `{KD, Ball}Tree` does not have the same performance constraints so we can consider option 1. I prefer not to have two patterns for handling 32/64 Cython classes, so I am leaning toward option 2 for `{KD, Ball}Tree` to be consistent with `DistanceMetric{32,64}`.

**Edit**: I meant option 2.

Labels: API, Needs Decision, RFC
Path to adopt 32-bit implementations for `{KD, Ball}Tree`

## Motivation
Having and using 32-bit implementations of `{KD, Ball}Tree` allows for better preservation of dtype, lower memory footprint, and more consistent Cython code.

## Strategy
Work has already been started in #25914 to add the code for the new 32-bit implementations. These will not be directly used yet, and the PR instead focuses on the actual creation of the new classes. Consequently, `{KD, Ball}Tree` are bound to `{KD, Ball}Tree64` for consistency and backwards compatibility.

Following this PR, we will need to begin an API deprecation to move users away from constructing trees directly, and instead using a factory method (similar to `DistanceMetric.get_metric`). Then, later, we can separate `{KD, Ball}Tree` from `{KD, Ball}Tree64` so that we have a singular dispatcher `{KD, Ball}Tree` and two type-specialized implementations. 

The deprecation process involves:
1. Introducing `get_tree(...)` to `{KD, Ball}Tree64` with the same signature of `BinaryTree.__init__`, which is in charge of constructing the type-specialized trees directly.
2. Adding a `FutureWarning` to `BinaryTree.__init__` to begin deprecation, and suppressing the warning using a context manager in `{KD, Ball}Tree64.get_tree` to enforce it as the "correct" way to construct the trees.
3. **Complete the deprecation** by introducing a separate `{KD, Ball}Tree` into the hierarchy which comes with the same `get_tree` method and fully remove the `FutureWarning` from `{KD, Ball}Tree{32, 64}.__init__` since they should no longer be directly constructed, as well as getting rid of the `{KD, Ball}Tree{32, 64}.get_tree` method since it should only exist in `{KD, Ball}Tree`.

At the end, the expected API is:
```python
from sklearn.neighbors import KDTree

# Data
X = ...

# tree is thus an instance of KDTree{32, 64} based on X.dtype
tree = KDTree.get_tree(X, ...)
```

cc: @thomasjpfan @jjerphan @OmarManzoor

COMMENT:
>I prefer not to have two patterns for handling 32/64 Cython classes, so I am leaning toward option 1 for {KD, Ball}Tree to be consistent with DistanceMetric{32,64}.

Do you mean for consistency you prefer option two? Or do you indeed mean option one here?

Labels: API, Needs Decision, RFC
Path to adopt 32-bit implementations for `{KD, Ball}Tree`

## Motivation
Having and using 32-bit implementations of `{KD, Ball}Tree` allows for better preservation of dtype, lower memory footprint, and more consistent Cython code.

## Strategy
Work has already been started in #25914 to add the code for the new 32-bit implementations. These will not be directly used yet, and the PR instead focuses on the actual creation of the new classes. Consequently, `{KD, Ball}Tree` are bound to `{KD, Ball}Tree64` for consistency and backwards compatibility.

Following this PR, we will need to begin an API deprecation to move users away from constructing trees directly, and instead using a factory method (similar to `DistanceMetric.get_metric`). Then, later, we can separate `{KD, Ball}Tree` from `{KD, Ball}Tree64` so that we have a singular dispatcher `{KD, Ball}Tree` and two type-specialized implementations. 

The deprecation process involves:
1. Introducing `get_tree(...)` to `{KD, Ball}Tree64` with the same signature of `BinaryTree.__init__`, which is in charge of constructing the type-specialized trees directly.
2. Adding a `FutureWarning` to `BinaryTree.__init__` to begin deprecation, and suppressing the warning using a context manager in `{KD, Ball}Tree64.get_tree` to enforce it as the "correct" way to construct the trees.
3. **Complete the deprecation** by introducing a separate `{KD, Ball}Tree` into the hierarchy which comes with the same `get_tree` method and fully remove the `FutureWarning` from `{KD, Ball}Tree{32, 64}.__init__` since they should no longer be directly constructed, as well as getting rid of the `{KD, Ball}Tree{32, 64}.get_tree` method since it should only exist in `{KD, Ball}Tree`.

At the end, the expected API is:
```python
from sklearn.neighbors import KDTree

# Data
X = ...

# tree is thus an instance of KDTree{32, 64} based on X.dtype
tree = KDTree.get_tree(X, ...)
```

cc: @thomasjpfan @jjerphan @OmarManzoor

COMMENT:
> Do you mean for consistency you prefer option two? 

Yes, I meant option 2 and I corrected my typo with option 2.

Labels: API, Needs Decision, RFC
Path to adopt 32-bit implementations for `{KD, Ball}Tree`

## Motivation
Having and using 32-bit implementations of `{KD, Ball}Tree` allows for better preservation of dtype, lower memory footprint, and more consistent Cython code.

## Strategy
Work has already been started in #25914 to add the code for the new 32-bit implementations. These will not be directly used yet, and the PR instead focuses on the actual creation of the new classes. Consequently, `{KD, Ball}Tree` are bound to `{KD, Ball}Tree64` for consistency and backwards compatibility.

Following this PR, we will need to begin an API deprecation to move users away from constructing trees directly, and instead using a factory method (similar to `DistanceMetric.get_metric`). Then, later, we can separate `{KD, Ball}Tree` from `{KD, Ball}Tree64` so that we have a singular dispatcher `{KD, Ball}Tree` and two type-specialized implementations. 

The deprecation process involves:
1. Introducing `get_tree(...)` to `{KD, Ball}Tree64` with the same signature of `BinaryTree.__init__`, which is in charge of constructing the type-specialized trees directly.
2. Adding a `FutureWarning` to `BinaryTree.__init__` to begin deprecation, and suppressing the warning using a context manager in `{KD, Ball}Tree64.get_tree` to enforce it as the "correct" way to construct the trees.
3. **Complete the deprecation** by introducing a separate `{KD, Ball}Tree` into the hierarchy which comes with the same `get_tree` method and fully remove the `FutureWarning` from `{KD, Ball}Tree{32, 64}.__init__` since they should no longer be directly constructed, as well as getting rid of the `{KD, Ball}Tree{32, 64}.get_tree` method since it should only exist in `{KD, Ball}Tree`.

At the end, the expected API is:
```python
from sklearn.neighbors import KDTree

# Data
X = ...

# tree is thus an instance of KDTree{32, 64} based on X.dtype
tree = KDTree.get_tree(X, ...)
```

cc: @thomasjpfan @jjerphan @OmarManzoor

COMMENT:
@jjerphan Should we progress with option 2 and start with introducing the get_tree method while deprecating the current way to create trees?

Labels: API, Needs Decision, RFC