Talip7/scikit-learn-issues-multilabel huggingface dataset:
Change how KNeighborsClassifier handles tiebreakers

### Describe the workflow you want to enable

I see that KNeighborsClassifier handles tiebreakers by taking the mode of the classes contained with the tie. What about using the sum of the distances as the tiebreaker instead (of using the mode.)

I feel that taking the least sum of distances is a better educated guess with minimal changes to the code.

### Describe your proposed solution

Create two dictionaries to keep a count of the Class count and the sum of Class distances.
If there is no tie return the Class value that won the vote (as normal).
If there is a tie return the Class that has the minimum sum of distances (instead of what scipy.stats.mode yields which I believe will always be the smallest Class value from the Classes within the tie.)

```
    dict_count = {}
    dict_dist = {}
    
    for i in range(num_neighbors):
        
        # Get the class from the i-th neighbor - here my Class value is at offset 13
        c = neighbors[i][13]
        
        # Keep a dictionary of Class counts
        if c in dict_count:
            dict_count[c] = dict_count[c] + 1
        else:
            dict_count[c] = 1
          
     # Keep a dictionary of total Class distances 
        if c in dict_dist:
            dict_dist[c] = dict_dist[c] + distances[i]
        else:
            dict_dist[c] = distances[i]
    
    # Return the key within the dictionary that has the largest value
    best_keys = [key for key, value in dict_count.items() if value == max(dict_count.values())]

    if len(best_keys) == 1:
        prediction = best_keys[0]

    else:   # There was a tie
        tie_dict = {}
        for b in best_keys:
            tie_dict[b] = dict_dist[b]
            
        prediction = min(tie_dict, key=tie_dict.get)

    return prediction
```

### Describe alternatives you've considered, if relevant

Writing my own _k_-NN algorithm 🙁

### Additional context

_No response_

COMMENT:
Hi @jjerphan  Yes, I am aware of that - but that's not what I am asking. Distances has a callable and a parameter to pass a dictionary to your callable. The weights also have a callable (like you have shown) but NO option to pass parameters to it.

Labels: New Feature, module:neighbors, Needs Investigation
Change how KNeighborsClassifier handles tiebreakers

### Describe the workflow you want to enable

I see that KNeighborsClassifier handles tiebreakers by taking the mode of the classes contained with the tie. What about using the sum of the distances as the tiebreaker instead (of using the mode.)

I feel that taking the least sum of distances is a better educated guess with minimal changes to the code.

### Describe your proposed solution

Create two dictionaries to keep a count of the Class count and the sum of Class distances.
If there is no tie return the Class value that won the vote (as normal).
If there is a tie return the Class that has the minimum sum of distances (instead of what scipy.stats.mode yields which I believe will always be the smallest Class value from the Classes within the tie.)

```
    dict_count = {}
    dict_dist = {}
    
    for i in range(num_neighbors):
        
        # Get the class from the i-th neighbor - here my Class value is at offset 13
        c = neighbors[i][13]
        
        # Keep a dictionary of Class counts
        if c in dict_count:
            dict_count[c] = dict_count[c] + 1
        else:
            dict_count[c] = 1
          
     # Keep a dictionary of total Class distances 
        if c in dict_dist:
            dict_dist[c] = dict_dist[c] + distances[i]
        else:
            dict_dist[c] = distances[i]
    
    # Return the key within the dictionary that has the largest value
    best_keys = [key for key, value in dict_count.items() if value == max(dict_count.values())]

    if len(best_keys) == 1:
        prediction = best_keys[0]

    else:   # There was a tie
        tie_dict = {}
        for b in best_keys:
            tie_dict[b] = dict_dist[b]
            
        prediction = min(tie_dict, key=tie_dict.get)

    return prediction
```

### Describe alternatives you've considered, if relevant

Writing my own _k_-NN algorithm 🙁

### Additional context

_No response_

COMMENT:
I think that flipping a coin is worse than a deterministic modal determination. I don't think running the classifier multiple times should give different responses with the exact same dataset. But the original poster left out one important feature. It is NOT just with even an number of neighbors. It can happen with a 3 way tie with 3 (an odd) number of neighbors, it can be a 3 way tie with 4 (an even) number of neighbors, etc.

Labels: New Feature, module:neighbors, Needs Investigation
Outdated test name in contributing page

### Describe the issue linked to the documentation

On this page https://scikit-learn.org/stable/developers/contributing.html#contributing-code in the "Pull request checklist" section, test command `pytest sklearn/linear_model/logistic.py`.
The logistic.py file does not exist (probably renamed to  [_logistic.py](https://github.com/RazHoshia/scikit-learn/blob/main/sklearn/linear_model/_logistic.py) but I`m not sure).

### Suggest a potential alternative/fix

_No response_

COMMENT:
Yes it is `_logistic.py`. Would you be interested in opening a pull request to fix the documentation?

Labels: Documentation, help wanted
Outdated test name in contributing page

### Describe the issue linked to the documentation

On this page https://scikit-learn.org/stable/developers/contributing.html#contributing-code in the "Pull request checklist" section, test command `pytest sklearn/linear_model/logistic.py`.
The logistic.py file does not exist (probably renamed to  [_logistic.py](https://github.com/RazHoshia/scikit-learn/blob/main/sklearn/linear_model/_logistic.py) but I`m not sure).

### Suggest a potential alternative/fix

_No response_

COMMENT:
Hi, @thomasjpfan @RazHoshia  I have raised a pr for this issue please review that pr and merge. If any more changes require please do let me know I will update. Thank you

Labels: Documentation, help wanted
KNeighborsClassifier: allow p value for Minkowski calculation to be less than 1.0 (and greater than 0)

### Describe the workflow you want to enable

I would like to be able to use the KNeighborsClassifier with something like:

`neigh = KNeighborsClassifier(n_neighbors=2, p=0.1)`

The error you get is:

```
File ~/opt/anaconda3/envs/tensorflow-3/lib/python3.10/site-packages/sklearn/neighbors/_base.py:395, in NeighborsBase._check_algorithm_metric(self)
    392     effective_p = self.p
    394 if self.metric in ["wminkowski", "minkowski"] and effective_p < 1:
--> 395     raise ValueError("p must be greater or equal to one for minkowski metric")

ValueError: p must be greater or equal to one for minkowski metric
```

Change the above line to limit minkowski p value to < 0:

`if self.metric in ["wminkowski", "minkowski"] and effective_p < 0:`

There is nothing wrong with using a p value > 0 and < 1.

### Describe your proposed solution

Don't throw an exception if p is < 1.0.

Calculating Minkowski distance is most definitely valid for p > 0 and p < 1.  I.e.: 0.1, 0.2, 0.3, 0.4, etc

The only requirement is compute for each dimension raised to the p power. In this case **p and after the sum of all dimensions. Take the p-th root of the sum of dimensions:  `distance = distance_sum**p`

There are many cases where it is desirable to compute minkowski > 0 and < 1

### Describe alternatives you've considered, if relevant

Write my own kNN classifier with my own minkowski distance calculator:

```
def minkowski_distance(row1, row2, p=3):
    distance = 0.0
    for i in range(len(row1)-1):
        distance += abs(row1[i] - row2[i])**p
    return distance**(1/p)
```

### Additional context

_No response_

COMMENT:
The issue is that minkowski with p< 1 is not a metric (no triangular inequality). Then the question is do we want to support similarity measures that are not metrics ?

Labels: New Feature, module:neighbors, Needs Info
KNeighborsClassifier: allow p value for Minkowski calculation to be less than 1.0 (and greater than 0)

### Describe the workflow you want to enable

I would like to be able to use the KNeighborsClassifier with something like:

`neigh = KNeighborsClassifier(n_neighbors=2, p=0.1)`

The error you get is:

```
File ~/opt/anaconda3/envs/tensorflow-3/lib/python3.10/site-packages/sklearn/neighbors/_base.py:395, in NeighborsBase._check_algorithm_metric(self)
    392     effective_p = self.p
    394 if self.metric in ["wminkowski", "minkowski"] and effective_p < 1:
--> 395     raise ValueError("p must be greater or equal to one for minkowski metric")

ValueError: p must be greater or equal to one for minkowski metric
```

Change the above line to limit minkowski p value to < 0:

`if self.metric in ["wminkowski", "minkowski"] and effective_p < 0:`

There is nothing wrong with using a p value > 0 and < 1.

### Describe your proposed solution

Don't throw an exception if p is < 1.0.

Calculating Minkowski distance is most definitely valid for p > 0 and p < 1.  I.e.: 0.1, 0.2, 0.3, 0.4, etc

The only requirement is compute for each dimension raised to the p power. In this case **p and after the sum of all dimensions. Take the p-th root of the sum of dimensions:  `distance = distance_sum**p`

There are many cases where it is desirable to compute minkowski > 0 and < 1

### Describe alternatives you've considered, if relevant

Write my own kNN classifier with my own minkowski distance calculator:

```
def minkowski_distance(row1, row2, p=3):
    distance = 0.0
    for i in range(len(row1)-1):
        distance += abs(row1[i] - row2[i])**p
    return distance**(1/p)
```

### Additional context

_No response_

COMMENT:
> The issue is that minkowski with p< 1 is not a metric (no triangular inequality). Then the question is do we want to support similarity measures that are not metrics ?

That seems like a typical example of where it's best to allow that behaviour but raise a warning.

Labels: New Feature, module:neighbors, Needs Info
KNeighborsClassifier: allow p value for Minkowski calculation to be less than 1.0 (and greater than 0)

### Describe the workflow you want to enable

I would like to be able to use the KNeighborsClassifier with something like:

`neigh = KNeighborsClassifier(n_neighbors=2, p=0.1)`

The error you get is:

```
File ~/opt/anaconda3/envs/tensorflow-3/lib/python3.10/site-packages/sklearn/neighbors/_base.py:395, in NeighborsBase._check_algorithm_metric(self)
    392     effective_p = self.p
    394 if self.metric in ["wminkowski", "minkowski"] and effective_p < 1:
--> 395     raise ValueError("p must be greater or equal to one for minkowski metric")

ValueError: p must be greater or equal to one for minkowski metric
```

Change the above line to limit minkowski p value to < 0:

`if self.metric in ["wminkowski", "minkowski"] and effective_p < 0:`

There is nothing wrong with using a p value > 0 and < 1.

### Describe your proposed solution

Don't throw an exception if p is < 1.0.

Calculating Minkowski distance is most definitely valid for p > 0 and p < 1.  I.e.: 0.1, 0.2, 0.3, 0.4, etc

The only requirement is compute for each dimension raised to the p power. In this case **p and after the sum of all dimensions. Take the p-th root of the sum of dimensions:  `distance = distance_sum**p`

There are many cases where it is desirable to compute minkowski > 0 and < 1

### Describe alternatives you've considered, if relevant

Write my own kNN classifier with my own minkowski distance calculator:

```
def minkowski_distance(row1, row2, p=3):
    distance = 0.0
    for i in range(len(row1)-1):
        distance += abs(row1[i] - row2[i])**p
    return distance**(1/p)
```

### Additional context

_No response_

COMMENT:
@raymondj-pace would you mind sharing some references for us to see how this would be useful and how it would be sensible to apply KNN on a non-metric function like this?

Labels: New Feature, module:neighbors, Needs Info
KNeighborsClassifier: allow p value for Minkowski calculation to be less than 1.0 (and greater than 0)

### Describe the workflow you want to enable

I would like to be able to use the KNeighborsClassifier with something like:

`neigh = KNeighborsClassifier(n_neighbors=2, p=0.1)`

The error you get is:

```
File ~/opt/anaconda3/envs/tensorflow-3/lib/python3.10/site-packages/sklearn/neighbors/_base.py:395, in NeighborsBase._check_algorithm_metric(self)
    392     effective_p = self.p
    394 if self.metric in ["wminkowski", "minkowski"] and effective_p < 1:
--> 395     raise ValueError("p must be greater or equal to one for minkowski metric")

ValueError: p must be greater or equal to one for minkowski metric
```

Change the above line to limit minkowski p value to < 0:

`if self.metric in ["wminkowski", "minkowski"] and effective_p < 0:`

There is nothing wrong with using a p value > 0 and < 1.

### Describe your proposed solution

Don't throw an exception if p is < 1.0.

Calculating Minkowski distance is most definitely valid for p > 0 and p < 1.  I.e.: 0.1, 0.2, 0.3, 0.4, etc

The only requirement is compute for each dimension raised to the p power. In this case **p and after the sum of all dimensions. Take the p-th root of the sum of dimensions:  `distance = distance_sum**p`

There are many cases where it is desirable to compute minkowski > 0 and < 1

### Describe alternatives you've considered, if relevant

Write my own kNN classifier with my own minkowski distance calculator:

```
def minkowski_distance(row1, row2, p=3):
    distance = 0.0
    for i in range(len(row1)-1):
        distance += abs(row1[i] - row2[i])**p
    return distance**(1/p)
```

### Additional context

_No response_

COMMENT:
One is here: [Combining Minkowski and Chebyshev - arXivhttps://arxiv.org › pdf]  https://arxiv.org/pdf/2112.12549

Scroll down to page 69 - p = 0.5, 0.75. 

The same diagrams for p < 1 are also on wikipedia as well: https://en.wikipedia.org/wiki/Minkowski_distance

Labels: New Feature, module:neighbors, Needs Info
KNeighborsClassifier: allow p value for Minkowski calculation to be less than 1.0 (and greater than 0)

### Describe the workflow you want to enable

I would like to be able to use the KNeighborsClassifier with something like:

`neigh = KNeighborsClassifier(n_neighbors=2, p=0.1)`

The error you get is:

```
File ~/opt/anaconda3/envs/tensorflow-3/lib/python3.10/site-packages/sklearn/neighbors/_base.py:395, in NeighborsBase._check_algorithm_metric(self)
    392     effective_p = self.p
    394 if self.metric in ["wminkowski", "minkowski"] and effective_p < 1:
--> 395     raise ValueError("p must be greater or equal to one for minkowski metric")

ValueError: p must be greater or equal to one for minkowski metric
```

Change the above line to limit minkowski p value to < 0:

`if self.metric in ["wminkowski", "minkowski"] and effective_p < 0:`

There is nothing wrong with using a p value > 0 and < 1.

### Describe your proposed solution

Don't throw an exception if p is < 1.0.

Calculating Minkowski distance is most definitely valid for p > 0 and p < 1.  I.e.: 0.1, 0.2, 0.3, 0.4, etc

The only requirement is compute for each dimension raised to the p power. In this case **p and after the sum of all dimensions. Take the p-th root of the sum of dimensions:  `distance = distance_sum**p`

There are many cases where it is desirable to compute minkowski > 0 and < 1

### Describe alternatives you've considered, if relevant

Write my own kNN classifier with my own minkowski distance calculator:

```
def minkowski_distance(row1, row2, p=3):
    distance = 0.0
    for i in range(len(row1)-1):
        distance += abs(row1[i] - row2[i])**p
    return distance**(1/p)
```

### Additional context

_No response_

COMMENT:
@scikit-learn/core-devs are we happy with the literature on p<1 here?

Labels: New Feature, module:neighbors, Needs Info
KNeighborsClassifier: allow p value for Minkowski calculation to be less than 1.0 (and greater than 0)

### Describe the workflow you want to enable

I would like to be able to use the KNeighborsClassifier with something like:

`neigh = KNeighborsClassifier(n_neighbors=2, p=0.1)`

The error you get is:

```
File ~/opt/anaconda3/envs/tensorflow-3/lib/python3.10/site-packages/sklearn/neighbors/_base.py:395, in NeighborsBase._check_algorithm_metric(self)
    392     effective_p = self.p
    394 if self.metric in ["wminkowski", "minkowski"] and effective_p < 1:
--> 395     raise ValueError("p must be greater or equal to one for minkowski metric")

ValueError: p must be greater or equal to one for minkowski metric
```

Change the above line to limit minkowski p value to < 0:

`if self.metric in ["wminkowski", "minkowski"] and effective_p < 0:`

There is nothing wrong with using a p value > 0 and < 1.

### Describe your proposed solution

Don't throw an exception if p is < 1.0.

Calculating Minkowski distance is most definitely valid for p > 0 and p < 1.  I.e.: 0.1, 0.2, 0.3, 0.4, etc

The only requirement is compute for each dimension raised to the p power. In this case **p and after the sum of all dimensions. Take the p-th root of the sum of dimensions:  `distance = distance_sum**p`

There are many cases where it is desirable to compute minkowski > 0 and < 1

### Describe alternatives you've considered, if relevant

Write my own kNN classifier with my own minkowski distance calculator:

```
def minkowski_distance(row1, row2, p=3):
    distance = 0.0
    for i in range(len(row1)-1):
        distance += abs(row1[i] - row2[i])**p
    return distance**(1/p)
```

### Additional context

_No response_

COMMENT:
I think that I would rather not support it as a `DistanceMetric` since it is not a distance metric for the reason given by @jeremiedbb above.

Note that you might still define your own distance using the `pyfunc` argument to support those cases potentially, [see the documentation](https://scikit-learn.org/0.24/modules/generated/sklearn.neighbors.DistanceMetric.html).

Labels: New Feature, module:neighbors, Needs Info
KNeighborsClassifier: allow p value for Minkowski calculation to be less than 1.0 (and greater than 0)

### Describe the workflow you want to enable

I would like to be able to use the KNeighborsClassifier with something like:

`neigh = KNeighborsClassifier(n_neighbors=2, p=0.1)`

The error you get is:

```
File ~/opt/anaconda3/envs/tensorflow-3/lib/python3.10/site-packages/sklearn/neighbors/_base.py:395, in NeighborsBase._check_algorithm_metric(self)
    392     effective_p = self.p
    394 if self.metric in ["wminkowski", "minkowski"] and effective_p < 1:
--> 395     raise ValueError("p must be greater or equal to one for minkowski metric")

ValueError: p must be greater or equal to one for minkowski metric
```

Change the above line to limit minkowski p value to < 0:

`if self.metric in ["wminkowski", "minkowski"] and effective_p < 0:`

There is nothing wrong with using a p value > 0 and < 1.

### Describe your proposed solution

Don't throw an exception if p is < 1.0.

Calculating Minkowski distance is most definitely valid for p > 0 and p < 1.  I.e.: 0.1, 0.2, 0.3, 0.4, etc

The only requirement is compute for each dimension raised to the p power. In this case **p and after the sum of all dimensions. Take the p-th root of the sum of dimensions:  `distance = distance_sum**p`

There are many cases where it is desirable to compute minkowski > 0 and < 1

### Describe alternatives you've considered, if relevant

Write my own kNN classifier with my own minkowski distance calculator:

```
def minkowski_distance(row1, row2, p=3):
    distance = 0.0
    for i in range(len(row1)-1):
        distance += abs(row1[i] - row2[i])**p
    return distance**(1/p)
```

### Additional context

_No response_

COMMENT:
This functionality is available via `scipy.spatial.distance.cdist(X, Y, "minkowski", p=0.1)` and `KNeighborsClassifier` accepts a callable as parameter.
I'm fine with either the current behaviour (not allowing p<1) or at least throwing a warning.

Labels: New Feature, module:neighbors, Needs Info
KNeighborsClassifier: allow p value for Minkowski calculation to be less than 1.0 (and greater than 0)

### Describe the workflow you want to enable

I would like to be able to use the KNeighborsClassifier with something like:

`neigh = KNeighborsClassifier(n_neighbors=2, p=0.1)`

The error you get is:

```
File ~/opt/anaconda3/envs/tensorflow-3/lib/python3.10/site-packages/sklearn/neighbors/_base.py:395, in NeighborsBase._check_algorithm_metric(self)
    392     effective_p = self.p
    394 if self.metric in ["wminkowski", "minkowski"] and effective_p < 1:
--> 395     raise ValueError("p must be greater or equal to one for minkowski metric")

ValueError: p must be greater or equal to one for minkowski metric
```

Change the above line to limit minkowski p value to < 0:

`if self.metric in ["wminkowski", "minkowski"] and effective_p < 0:`

There is nothing wrong with using a p value > 0 and < 1.

### Describe your proposed solution

Don't throw an exception if p is < 1.0.

Calculating Minkowski distance is most definitely valid for p > 0 and p < 1.  I.e.: 0.1, 0.2, 0.3, 0.4, etc

The only requirement is compute for each dimension raised to the p power. In this case **p and after the sum of all dimensions. Take the p-th root of the sum of dimensions:  `distance = distance_sum**p`

There are many cases where it is desirable to compute minkowski > 0 and < 1

### Describe alternatives you've considered, if relevant

Write my own kNN classifier with my own minkowski distance calculator:

```
def minkowski_distance(row1, row2, p=3):
    distance = 0.0
    for i in range(len(row1)-1):
        distance += abs(row1[i] - row2[i])**p
    return distance**(1/p)
```

### Additional context

_No response_

COMMENT:
So should we document this one instead of implementing it?

Labels: New Feature, module:neighbors, Needs Info