Talip7/scikit-learn-issues-multilabel huggingface dataset:
KNearestNeighbors dedicated API for unsupervised anomaly detection

### Describe the workflow you want to enable

All unsupervised anomaly detection algorithms share a similar API  (e.g., IsolationForest, OneSVM) but not KNearestNeighbors... And yet, KNN is one of the most used and robust method. (https://arxiv.org/pdf/2206.09426v2.pdf).

Having different API makes it challenging to loop and compare different anomaly detection algorithms... Example: https://scikit-learn.org/0.20/auto_examples/plot_anomaly_comparison.html  (KNN is not present)

Today, you can detect anomalies with KNN with the code below:

```
from sklearn.neighbors import NearestNeighbors
import numpy as np
# my data
data=np.random.normal(0,1,(10,2))
data[6][0], data[6][1] = 4.1, -3.5 # Let's consider the point 6 is anomalous (>3 stdv)

# building the model
model=NearestNeighbors(n_neighbors=3)
model.fit(data)

# predicting
distances, indexes = model.kneighbors(data)
anomaly_score=np.sum(distances,axis=1)
anomaly_thresh=np.quantile(anomaly_score,0.1)
is_inlier = np.ones(data.shape[0], dtype=int)
is_inlier[anomaly_score < anomaly_thresh] = -1
print(is_inlier)# "[ 1  1  1  1  1  1  1 -1  1  1]"
```

### Describe your proposed solution

The API I propose should look like this:
```
model=NearestNeighborsAD(n_neighbors=3,contamination=0.1)
model.fit(data)
is_inline=model.predict(data)
```
Only 3 lines of codes and the same API as the others (e.g., IsolationForest, OneSVM)

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I think that the original article is the following:

https://dl.acm.org/doi/abs/10.1145/342009.335437

```
Ramaswamy, Sridhar, Rajeev Rastogi, and Kyuseok Shim.
"Efficient algorithms for mining outliers from large data sets."
Proceedings of the 2000 ACM SIGMOD international conference on Management of data.
2000.
```

and it is cited ~2800 times.

@albertcthomas did you come across by hazard when developing the other approaches and if this is the case, would you have any feedback that could be relevant for the inclusion of this method?

Labels: New Feature, Needs Decision - Include Feature
KNearestNeighbors dedicated API for unsupervised anomaly detection

### Describe the workflow you want to enable

All unsupervised anomaly detection algorithms share a similar API  (e.g., IsolationForest, OneSVM) but not KNearestNeighbors... And yet, KNN is one of the most used and robust method. (https://arxiv.org/pdf/2206.09426v2.pdf).

Having different API makes it challenging to loop and compare different anomaly detection algorithms... Example: https://scikit-learn.org/0.20/auto_examples/plot_anomaly_comparison.html  (KNN is not present)

Today, you can detect anomalies with KNN with the code below:

```
from sklearn.neighbors import NearestNeighbors
import numpy as np
# my data
data=np.random.normal(0,1,(10,2))
data[6][0], data[6][1] = 4.1, -3.5 # Let's consider the point 6 is anomalous (>3 stdv)

# building the model
model=NearestNeighbors(n_neighbors=3)
model.fit(data)

# predicting
distances, indexes = model.kneighbors(data)
anomaly_score=np.sum(distances,axis=1)
anomaly_thresh=np.quantile(anomaly_score,0.1)
is_inlier = np.ones(data.shape[0], dtype=int)
is_inlier[anomaly_score < anomaly_thresh] = -1
print(is_inlier)# "[ 1  1  1  1  1  1  1 -1  1  1]"
```

### Describe your proposed solution

The API I propose should look like this:
```
model=NearestNeighborsAD(n_neighbors=3,contamination=0.1)
model.fit(data)
is_inline=model.predict(data)
```
Only 3 lines of codes and the same API as the others (e.g., IsolationForest, OneSVM)

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
For me, the question is whether we assume that the user has everything he needs (NearestNeighbors) to define such an outlier detection estimator by himself. It is the same question for deriving an outlier detector based on KernelDensity and I think the answer was, 5 years ago, that a user could easily derive such an estimator. FWIW I have KNN based outlier detectors implemented here https://github.com/albertcthomas/anomaly_tuning/blob/master/anomaly_tuning/estimators.py and I think they satisfy scikit-learn API.

Labels: New Feature, Needs Decision - Include Feature
KNearestNeighbors dedicated API for unsupervised anomaly detection

### Describe the workflow you want to enable

All unsupervised anomaly detection algorithms share a similar API  (e.g., IsolationForest, OneSVM) but not KNearestNeighbors... And yet, KNN is one of the most used and robust method. (https://arxiv.org/pdf/2206.09426v2.pdf).

Having different API makes it challenging to loop and compare different anomaly detection algorithms... Example: https://scikit-learn.org/0.20/auto_examples/plot_anomaly_comparison.html  (KNN is not present)

Today, you can detect anomalies with KNN with the code below:

```
from sklearn.neighbors import NearestNeighbors
import numpy as np
# my data
data=np.random.normal(0,1,(10,2))
data[6][0], data[6][1] = 4.1, -3.5 # Let's consider the point 6 is anomalous (>3 stdv)

# building the model
model=NearestNeighbors(n_neighbors=3)
model.fit(data)

# predicting
distances, indexes = model.kneighbors(data)
anomaly_score=np.sum(distances,axis=1)
anomaly_thresh=np.quantile(anomaly_score,0.1)
is_inlier = np.ones(data.shape[0], dtype=int)
is_inlier[anomaly_score < anomaly_thresh] = -1
print(is_inlier)# "[ 1  1  1  1  1  1  1 -1  1  1]"
```

### Describe your proposed solution

The API I propose should look like this:
```
model=NearestNeighborsAD(n_neighbors=3,contamination=0.1)
model.fit(data)
is_inline=model.predict(data)
```
Only 3 lines of codes and the same API as the others (e.g., IsolationForest, OneSVM)

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
> It is the same question for deriving an outlier detector based on KernelDensity and I think the answer was, 5 years ago, that a user could easily derive such an estimator

I am fine sticking to the decision taken in the past then. Thanks @albertcthomas to bring this piece of info. I am going to close this issue and the associated PR then.

Thanks @PierrickPochelu for proposing this feature.

Labels: New Feature, Needs Decision - Include Feature
KNearestNeighbors dedicated API for unsupervised anomaly detection

### Describe the workflow you want to enable

All unsupervised anomaly detection algorithms share a similar API  (e.g., IsolationForest, OneSVM) but not KNearestNeighbors... And yet, KNN is one of the most used and robust method. (https://arxiv.org/pdf/2206.09426v2.pdf).

Having different API makes it challenging to loop and compare different anomaly detection algorithms... Example: https://scikit-learn.org/0.20/auto_examples/plot_anomaly_comparison.html  (KNN is not present)

Today, you can detect anomalies with KNN with the code below:

```
from sklearn.neighbors import NearestNeighbors
import numpy as np
# my data
data=np.random.normal(0,1,(10,2))
data[6][0], data[6][1] = 4.1, -3.5 # Let's consider the point 6 is anomalous (>3 stdv)

# building the model
model=NearestNeighbors(n_neighbors=3)
model.fit(data)

# predicting
distances, indexes = model.kneighbors(data)
anomaly_score=np.sum(distances,axis=1)
anomaly_thresh=np.quantile(anomaly_score,0.1)
is_inlier = np.ones(data.shape[0], dtype=int)
is_inlier[anomaly_score < anomaly_thresh] = -1
print(is_inlier)# "[ 1  1  1  1  1  1  1 -1  1  1]"
```

### Describe your proposed solution

The API I propose should look like this:
```
model=NearestNeighborsAD(n_neighbors=3,contamination=0.1)
model.fit(data)
is_inline=model.predict(data)
```
Only 3 lines of codes and the same API as the others (e.g., IsolationForest, OneSVM)

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
@PierrickPochelu you might also want to try the pyOD package which implements a kNN based outlier detector https://pyod.readthedocs.io/en/latest/pyod.models.html#module-pyod.models.knn

Labels: New Feature, Needs Decision - Include Feature
KNearestNeighbors dedicated API for unsupervised anomaly detection

### Describe the workflow you want to enable

All unsupervised anomaly detection algorithms share a similar API  (e.g., IsolationForest, OneSVM) but not KNearestNeighbors... And yet, KNN is one of the most used and robust method. (https://arxiv.org/pdf/2206.09426v2.pdf).

Having different API makes it challenging to loop and compare different anomaly detection algorithms... Example: https://scikit-learn.org/0.20/auto_examples/plot_anomaly_comparison.html  (KNN is not present)

Today, you can detect anomalies with KNN with the code below:

```
from sklearn.neighbors import NearestNeighbors
import numpy as np
# my data
data=np.random.normal(0,1,(10,2))
data[6][0], data[6][1] = 4.1, -3.5 # Let's consider the point 6 is anomalous (>3 stdv)

# building the model
model=NearestNeighbors(n_neighbors=3)
model.fit(data)

# predicting
distances, indexes = model.kneighbors(data)
anomaly_score=np.sum(distances,axis=1)
anomaly_thresh=np.quantile(anomaly_score,0.1)
is_inlier = np.ones(data.shape[0], dtype=int)
is_inlier[anomaly_score < anomaly_thresh] = -1
print(is_inlier)# "[ 1  1  1  1  1  1  1 -1  1  1]"
```

### Describe your proposed solution

The API I propose should look like this:
```
model=NearestNeighborsAD(n_neighbors=3,contamination=0.1)
model.fit(data)
is_inline=model.predict(data)
```
Only 3 lines of codes and the same API as the others (e.g., IsolationForest, OneSVM)

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Thank you for your answers @albertcthomas @glemaitre I understand the decision.

Labels: New Feature, Needs Decision - Include Feature
KNearestNeighbors dedicated API for unsupervised anomaly detection

### Describe the workflow you want to enable

All unsupervised anomaly detection algorithms share a similar API  (e.g., IsolationForest, OneSVM) but not KNearestNeighbors... And yet, KNN is one of the most used and robust method. (https://arxiv.org/pdf/2206.09426v2.pdf).

Having different API makes it challenging to loop and compare different anomaly detection algorithms... Example: https://scikit-learn.org/0.20/auto_examples/plot_anomaly_comparison.html  (KNN is not present)

Today, you can detect anomalies with KNN with the code below:

```
from sklearn.neighbors import NearestNeighbors
import numpy as np
# my data
data=np.random.normal(0,1,(10,2))
data[6][0], data[6][1] = 4.1, -3.5 # Let's consider the point 6 is anomalous (>3 stdv)

# building the model
model=NearestNeighbors(n_neighbors=3)
model.fit(data)

# predicting
distances, indexes = model.kneighbors(data)
anomaly_score=np.sum(distances,axis=1)
anomaly_thresh=np.quantile(anomaly_score,0.1)
is_inlier = np.ones(data.shape[0], dtype=int)
is_inlier[anomaly_score < anomaly_thresh] = -1
print(is_inlier)# "[ 1  1  1  1  1  1  1 -1  1  1]"
```

### Describe your proposed solution

The API I propose should look like this:
```
model=NearestNeighborsAD(n_neighbors=3,contamination=0.1)
model.fit(data)
is_inline=model.predict(data)
```
Only 3 lines of codes and the same API as the others (e.g., IsolationForest, OneSVM)

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I have opened a pull request #27337 that adds knn, knn weight, and loop outlier detectors.
If you are interested in seeing this included in core sklearn, you could express this by a thumbs-up on the pull request (I prefer thumbs-up because these do not cause notifications for "me too"s).

Labels: New Feature, Needs Decision - Include Feature
ENH Avoid memoryviews' slicing for `KMeans` Cython implementations

### Summary

Addresses issues raised in https://github.com/scikit-learn/scikit-learn/issues/17299

The proposal is to modify the LOC here: https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/cluster/_k_means_common.pyx#L155-L159. There are currently three places where the `_euclidean_sparse_dense` Cython function is used and can be optimized. 

The issue with the current implementation is that `centers` is a 2D memview and thus passing in `centers[j]` creates a 1D memview. I think going from 1D memview to another 1D memview is okay(?) If not, then we need to also modify the other arguments.

### Proposal
Change the signature of the Cython function `_euclidean_sparse_dense` to this:

```
cdef floating _euclidean_sparse_dense(
        floating[::1] a_data,  # IN
        int[::1] a_indices,    # IN
        floating[::1] b,       # IN
        floating b_squared_norms,
        int b_index,
        bint squared) nogil:
    ...
```

and adjust the unit tests and corresponding Cython code. I will put up a draft PR to demonstrate what is needed there.

### Misc.
cc: @jjerphan who brought this up as a possible Cython improvement for me to help out with.

COMMENT:
Label: Cython, Clustering

Labels: Performance, module:cluster, cython
ENH Avoid memoryviews' slicing for `KMeans` Cython implementations

### Summary

Addresses issues raised in https://github.com/scikit-learn/scikit-learn/issues/17299

The proposal is to modify the LOC here: https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/cluster/_k_means_common.pyx#L155-L159. There are currently three places where the `_euclidean_sparse_dense` Cython function is used and can be optimized. 

The issue with the current implementation is that `centers` is a 2D memview and thus passing in `centers[j]` creates a 1D memview. I think going from 1D memview to another 1D memview is okay(?) If not, then we need to also modify the other arguments.

### Proposal
Change the signature of the Cython function `_euclidean_sparse_dense` to this:

```
cdef floating _euclidean_sparse_dense(
        floating[::1] a_data,  # IN
        int[::1] a_indices,    # IN
        floating[::1] b,       # IN
        floating b_squared_norms,
        int b_index,
        bint squared) nogil:
    ...
```

and adjust the unit tests and corresponding Cython code. I will put up a draft PR to demonstrate what is needed there.

### Misc.
cc: @jjerphan who brought this up as a possible Cython improvement for me to help out with.

COMMENT:
Thank you for opening this issue and proposing a PR, @adam2392.

> The issue with the current implementation is that centers is a 2D memview and thus passing in centers[j] creates a 1D memview.

Yes — cross-referencing https://github.com/scikit-learn/scikit-learn/issues/17299.

> I think going from 1D memview to another 1D memview is okay(?) If not, then we need to also modify the other arguments.

I think 1-D slicing comes with some overhead. This is present in k-means' internals here for instance:

https://github.com/scikit-learn/scikit-learn/blob/798aeebdee1e8f7c7eab1cb1b10c66f37a4eb7d2/sklearn/cluster/_k_means_common.pyx#L155-L159

but it might not be as costly.

I think we can proceed with separate benchmarks for both removing 2D- and 1D-slicing and see if those removals are useful.

Labels: Performance, module:cluster, cython
ENH Avoid memoryviews' slicing for `KMeans` Cython implementations

### Summary

Addresses issues raised in https://github.com/scikit-learn/scikit-learn/issues/17299

The proposal is to modify the LOC here: https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/cluster/_k_means_common.pyx#L155-L159. There are currently three places where the `_euclidean_sparse_dense` Cython function is used and can be optimized. 

The issue with the current implementation is that `centers` is a 2D memview and thus passing in `centers[j]` creates a 1D memview. I think going from 1D memview to another 1D memview is okay(?) If not, then we need to also modify the other arguments.

### Proposal
Change the signature of the Cython function `_euclidean_sparse_dense` to this:

```
cdef floating _euclidean_sparse_dense(
        floating[::1] a_data,  # IN
        int[::1] a_indices,    # IN
        floating[::1] b,       # IN
        floating b_squared_norms,
        int b_index,
        bint squared) nogil:
    ...
```

and adjust the unit tests and corresponding Cython code. I will put up a draft PR to demonstrate what is needed there.

### Misc.
cc: @jjerphan who brought this up as a possible Cython improvement for me to help out with.

COMMENT:
Closing this as there is no performance gain.

Labels: Performance, module:cluster, cython
ENH Avoid memoryviews' slicing for `KMeans` Cython implementations

### Summary

Addresses issues raised in https://github.com/scikit-learn/scikit-learn/issues/17299

The proposal is to modify the LOC here: https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/cluster/_k_means_common.pyx#L155-L159. There are currently three places where the `_euclidean_sparse_dense` Cython function is used and can be optimized. 

The issue with the current implementation is that `centers` is a 2D memview and thus passing in `centers[j]` creates a 1D memview. I think going from 1D memview to another 1D memview is okay(?) If not, then we need to also modify the other arguments.

### Proposal
Change the signature of the Cython function `_euclidean_sparse_dense` to this:

```
cdef floating _euclidean_sparse_dense(
        floating[::1] a_data,  # IN
        int[::1] a_indices,    # IN
        floating[::1] b,       # IN
        floating b_squared_norms,
        int b_index,
        bint squared) nogil:
    ...
```

and adjust the unit tests and corresponding Cython code. I will put up a draft PR to demonstrate what is needed there.

### Misc.
cc: @jjerphan who brought this up as a possible Cython improvement for me to help out with.

COMMENT:
as seen in https://github.com/scikit-learn/scikit-learn/pull/24566 experiments.

Labels: Performance, module:cluster, cython
convert builtin module import to supported form

### Describe the bug

convert builtin module import to supported form

### Steps/Code to Reproduce

import builtins as builtins

### Expected Results

No error is thrown

### Actual Results

import builtins as builtins

### Versions

```shell
import builtins as builtins
```

COMMENT:
Not enough info here to act. Happy to reopen if you leave more info.

Labels: Bug, Needs Triage
SelfTrainingClassifier: use of warm_start in base_estimator

### Describe the issue linked to the documentation

Hi,
I construct an instance of SelfTrainingClassifier with a base_constructor having warm_start set as True. When calling fit in SelfTrainingClassifier, is the classifier trained incrementally, or at every fit information of previous training is overwritten? I tried to figure out in documentation, but I didn't find whether re-use previous training info was allowed. I would like to pre-train a model on unsupervised data and later fine-tune it on labeled data.

Thank you for your support,
Best

### Suggest a potential alternative/fix

_No response_

COMMENT:
`SelfTrainingClassifier.fit` clones `base_estimator` to ensure calling fit twice results in the same model. If you need to have `warm_start` for `base_estimator` you can write a custom estimator which loads your model and calls fit on top of it, and pass that as the `base_estimator` to your `SelfTrainingClassifier`.

Happy to have a link to our meta-estimators glossary item where this is also explained: https://scikit-learn.org/stable/glossary.html#term-metaestimators

Labels: Documentation, good first issue
SelfTrainingClassifier: use of warm_start in base_estimator

### Describe the issue linked to the documentation

Hi,
I construct an instance of SelfTrainingClassifier with a base_constructor having warm_start set as True. When calling fit in SelfTrainingClassifier, is the classifier trained incrementally, or at every fit information of previous training is overwritten? I tried to figure out in documentation, but I didn't find whether re-use previous training info was allowed. I would like to pre-train a model on unsupervised data and later fine-tune it on labeled data.

Thank you for your support,
Best

### Suggest a potential alternative/fix

_No response_

COMMENT:
Hi, I'm new to open source contributions and I'd like to work on this, Could you please assign this to me?

Labels: Documentation, good first issue
SelfTrainingClassifier: use of warm_start in base_estimator

### Describe the issue linked to the documentation

Hi,
I construct an instance of SelfTrainingClassifier with a base_constructor having warm_start set as True. When calling fit in SelfTrainingClassifier, is the classifier trained incrementally, or at every fit information of previous training is overwritten? I tried to figure out in documentation, but I didn't find whether re-use previous training info was allowed. I would like to pre-train a model on unsupervised data and later fine-tune it on labeled data.

Thank you for your support,
Best

### Suggest a potential alternative/fix

_No response_

COMMENT:
Hi @1Aditya7 , feel free to work on it.

Labels: Documentation, good first issue
SelfTrainingClassifier: use of warm_start in base_estimator

### Describe the issue linked to the documentation

Hi,
I construct an instance of SelfTrainingClassifier with a base_constructor having warm_start set as True. When calling fit in SelfTrainingClassifier, is the classifier trained incrementally, or at every fit information of previous training is overwritten? I tried to figure out in documentation, but I didn't find whether re-use previous training info was allowed. I would like to pre-train a model on unsupervised data and later fine-tune it on labeled data.

Thank you for your support,
Best

### Suggest a potential alternative/fix

_No response_

COMMENT:
@adrinjalali Thanks! Will get back to you.

Labels: Documentation, good first issue
SelfTrainingClassifier: use of warm_start in base_estimator

### Describe the issue linked to the documentation

Hi,
I construct an instance of SelfTrainingClassifier with a base_constructor having warm_start set as True. When calling fit in SelfTrainingClassifier, is the classifier trained incrementally, or at every fit information of previous training is overwritten? I tried to figure out in documentation, but I didn't find whether re-use previous training info was allowed. I would like to pre-train a model on unsupervised data and later fine-tune it on labeled data.

Thank you for your support,
Best

### Suggest a potential alternative/fix

_No response_

COMMENT:
Hey, @adrinjalali I want to work on this issue but I am a new to open source contributions, I have some knowledge of machine learning. I want to contribute to this repo. Can you please guide me and assign me this issue

Labels: Documentation, good first issue
SelfTrainingClassifier: use of warm_start in base_estimator

### Describe the issue linked to the documentation

Hi,
I construct an instance of SelfTrainingClassifier with a base_constructor having warm_start set as True. When calling fit in SelfTrainingClassifier, is the classifier trained incrementally, or at every fit information of previous training is overwritten? I tried to figure out in documentation, but I didn't find whether re-use previous training info was allowed. I would like to pre-train a model on unsupervised data and later fine-tune it on labeled data.

Thank you for your support,
Best

### Suggest a potential alternative/fix

_No response_

COMMENT:
@Tribhuvan0 as you see here @1Aditya7 has already started working on it. So you'd need to find another issue to work on :) 

I strongly recommend you going through our contributing guide: https://scikit-learn.org/stable/developers/contributing.html?highlight=contributing+guide

Labels: Documentation, good first issue