Talip7/scikit-learn-issues-multilabel huggingface dataset:
Default value for solving an SVM in primal and dual should be determined automatically

### Describe the workflow you want to enable

Currently the default value for LinearSVC and LinearSVR with regards to solving the problem in dual is True. This means that for users that are unfamiliar with support-vector machines and do not know the importance of this parameter they could be using te framework inefficiently. 

I would propose for the default value to be None and a simple check to occur. If n_samples > n_features it's set to dual and otherwise it is set to primal. 

### Describe your proposed solution

This is a quick and easy fix and would only require a few lines of code in the fit method:

```
        if self.dual != None:
            pass
        else:
            if X.shape[0] < X.shape[1]:
                self.dual = True
            else:
                self.dual = False
```


### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I would like to be involved in the benchmark and the eventual PR if it comes to that @agramfort.

Do I just simulate and arbitrary dataset with sklearn that has n > d using the current default (dual) and post the results in the comments?

I also expect that running the benchmark with dimensions 100000x20 is trivial in primal and will result in a memory issue on most machines in dual.

Labels: Enhancement, module:svm
Default value for solving an SVM in primal and dual should be determined automatically

### Describe the workflow you want to enable

Currently the default value for LinearSVC and LinearSVR with regards to solving the problem in dual is True. This means that for users that are unfamiliar with support-vector machines and do not know the importance of this parameter they could be using te framework inefficiently. 

I would propose for the default value to be None and a simple check to occur. If n_samples > n_features it's set to dual and otherwise it is set to primal. 

### Describe your proposed solution

This is a quick and easy fix and would only require a few lines of code in the fit method:

```
        if self.dual != None:
            pass
        else:
            if X.shape[0] < X.shape[1]:
                self.dual = True
            else:
                self.dual = False
```


### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I think if you have the time then it would be great to contribute a benchmark and the PR @ChidiRnweke

Labels: Enhancement, module:svm
Default value for solving an SVM in primal and dual should be determined automatically

### Describe the workflow you want to enable

Currently the default value for LinearSVC and LinearSVR with regards to solving the problem in dual is True. This means that for users that are unfamiliar with support-vector machines and do not know the importance of this parameter they could be using te framework inefficiently. 

I would propose for the default value to be None and a simple check to occur. If n_samples > n_features it's set to dual and otherwise it is set to primal. 

### Describe your proposed solution

This is a quick and easy fix and would only require a few lines of code in the fit method:

```
        if self.dual != None:
            pass
        else:
            if X.shape[0] < X.shape[1]:
                self.dual = True
            else:
                self.dual = False
```


### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Summary of the benchmarks that I ran:

To put it shortly, for users that don't change the default parameters this is a big improvement for some datasets.

![barplot primal dual](https://user-images.githubusercontent.com/88949660/197419436-0b2d1f20-572b-4bb1-a327-b76dd11d1b01.png)

EDIT: Added a barplot instead of raw data to make the results readable.

Labels: Enhancement, module:svm
Default value for solving an SVM in primal and dual should be determined automatically

### Describe the workflow you want to enable

Currently the default value for LinearSVC and LinearSVR with regards to solving the problem in dual is True. This means that for users that are unfamiliar with support-vector machines and do not know the importance of this parameter they could be using te framework inefficiently. 

I would propose for the default value to be None and a simple check to occur. If n_samples > n_features it's set to dual and otherwise it is set to primal. 

### Describe your proposed solution

This is a quick and easy fix and would only require a few lines of code in the fit method:

```
        if self.dual != None:
            pass
        else:
            if X.shape[0] < X.shape[1]:
                self.dual = True
            else:
                self.dual = False
```


### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
It would be easier to have a bar plot to quickly compare the results :)

Labels: Enhancement, module:svm
Default value for solving an SVM in primal and dual should be determined automatically

### Describe the workflow you want to enable

Currently the default value for LinearSVC and LinearSVR with regards to solving the problem in dual is True. This means that for users that are unfamiliar with support-vector machines and do not know the importance of this parameter they could be using te framework inefficiently. 

I would propose for the default value to be None and a simple check to occur. If n_samples > n_features it's set to dual and otherwise it is set to primal. 

### Describe your proposed solution

This is a quick and easy fix and would only require a few lines of code in the fit method:

```
        if self.dual != None:
            pass
        else:
            if X.shape[0] < X.shape[1]:
                self.dual = True
            else:
                self.dual = False
```


### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Instead of comparing `dual=True` and `dual="auto"`, I think that we want to compare `dual=True/False` and check that for `n_samples < n_features`, `dual=True` will always lead to a faster `fit` time.

Labels: Enhancement, module:svm
Default value for solving an SVM in primal and dual should be determined automatically

### Describe the workflow you want to enable

Currently the default value for LinearSVC and LinearSVR with regards to solving the problem in dual is True. This means that for users that are unfamiliar with support-vector machines and do not know the importance of this parameter they could be using te framework inefficiently. 

I would propose for the default value to be None and a simple check to occur. If n_samples > n_features it's set to dual and otherwise it is set to primal. 

### Describe your proposed solution

This is a quick and easy fix and would only require a few lines of code in the fit method:

```
        if self.dual != None:
            pass
        else:
            if X.shape[0] < X.shape[1]:
                self.dual = True
            else:
                self.dual = False
```


### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I edited my comment to incorporate the feedback in terms of the benchmark and having a plot. Will be working on the changes on the adjustments to the pull request somewhere next week.

Labels: Enhancement, module:svm
Default value for solving an SVM in primal and dual should be determined automatically

### Describe the workflow you want to enable

Currently the default value for LinearSVC and LinearSVR with regards to solving the problem in dual is True. This means that for users that are unfamiliar with support-vector machines and do not know the importance of this parameter they could be using te framework inefficiently. 

I would propose for the default value to be None and a simple check to occur. If n_samples > n_features it's set to dual and otherwise it is set to primal. 

### Describe your proposed solution

This is a quick and easy fix and would only require a few lines of code in the fit method:

```
        if self.dual != None:
            pass
        else:
            if X.shape[0] < X.shape[1]:
                self.dual = True
            else:
                self.dual = False
```


### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Is it stale?

Labels: Enhancement, module:svm
Default value for solving an SVM in primal and dual should be determined automatically

### Describe the workflow you want to enable

Currently the default value for LinearSVC and LinearSVR with regards to solving the problem in dual is True. This means that for users that are unfamiliar with support-vector machines and do not know the importance of this parameter they could be using te framework inefficiently. 

I would propose for the default value to be None and a simple check to occur. If n_samples > n_features it's set to dual and otherwise it is set to primal. 

### Describe your proposed solution

This is a quick and easy fix and would only require a few lines of code in the fit method:

```
        if self.dual != None:
            pass
        else:
            if X.shape[0] < X.shape[1]:
                self.dual = True
            else:
                self.dual = False
```


### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
It looks so. Do you want to take over @glevv ?

Labels: Enhancement, module:svm
Default value for solving an SVM in primal and dual should be determined automatically

### Describe the workflow you want to enable

Currently the default value for LinearSVC and LinearSVR with regards to solving the problem in dual is True. This means that for users that are unfamiliar with support-vector machines and do not know the importance of this parameter they could be using te framework inefficiently. 

I would propose for the default value to be None and a simple check to occur. If n_samples > n_features it's set to dual and otherwise it is set to primal. 

### Describe your proposed solution

This is a quick and easy fix and would only require a few lines of code in the fit method:

```
        if self.dual != None:
            pass
        else:
            if X.shape[0] < X.shape[1]:
                self.dual = True
            else:
                self.dual = False
```


### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I'll look into it.

Is proposed algorithm for auto choosing dual formulation is good enough? Or maybe there are other rules?

Labels: Enhancement, module:svm
Default value for solving an SVM in primal and dual should be determined automatically

### Describe the workflow you want to enable

Currently the default value for LinearSVC and LinearSVR with regards to solving the problem in dual is True. This means that for users that are unfamiliar with support-vector machines and do not know the importance of this parameter they could be using te framework inefficiently. 

I would propose for the default value to be None and a simple check to occur. If n_samples > n_features it's set to dual and otherwise it is set to primal. 

### Describe your proposed solution

This is a quick and easy fix and would only require a few lines of code in the fit method:

```
        if self.dual != None:
            pass
        else:
            if X.shape[0] < X.shape[1]:
                self.dual = True
            else:
                self.dual = False
```


### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
It took a lot more codebase changes than I thought. Can someone review it please?

Labels: Enhancement, module:svm
New algorithm: DP-Means

### Describe the workflow you want to enable

DP-Means is a nonparametric version of K-Means, which is rooted in the Dirichlet Process Mixture Model.
Proposed by[ Kulis and Jordan in 2011](https://arxiv.org/pdf/1111.0352.pdf), it is already a well cited and established algorithm.

Recently, we have published a version of it which scales very good, comparable to K-Means and is equivalent to the original algorithm (https://openreview.net/pdf?id=rnzVBD8jqlq).

Enabling it in scikit-learn would benefit the community, and would be great in my opinion.

### Describe your proposed solution

We have already utilized scikit-learn codebase in order to implement our algorithm efficiently, (https://github.com/BGU-CS-VIL/pdc-dp-means/tree/main/cluster), so I can easily fix this into a pull-request to scikit-learn code base (will need to add the relevant tests and docs as well).

As such, if this feature request is approved, I will work on the relevant pull request in order to add this to the code base.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
can you give an elloboration on how does it is better than the parametized ones

Labels: New Feature, module:cluster, Needs Decision - Include Feature
New algorithm: DP-Means

### Describe the workflow you want to enable

DP-Means is a nonparametric version of K-Means, which is rooted in the Dirichlet Process Mixture Model.
Proposed by[ Kulis and Jordan in 2011](https://arxiv.org/pdf/1111.0352.pdf), it is already a well cited and established algorithm.

Recently, we have published a version of it which scales very good, comparable to K-Means and is equivalent to the original algorithm (https://openreview.net/pdf?id=rnzVBD8jqlq).

Enabling it in scikit-learn would benefit the community, and would be great in my opinion.

### Describe your proposed solution

We have already utilized scikit-learn codebase in order to implement our algorithm efficiently, (https://github.com/BGU-CS-VIL/pdc-dp-means/tree/main/cluster), so I can easily fix this into a pull-request to scikit-learn code base (will need to add the relevant tests and docs as well).

As such, if this feature request is approved, I will work on the relevant pull request in order to add this to the code base.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
also can you give me some refrences on how to approach a machine learning papers so i can really understand it

Labels: New Feature, module:cluster, Needs Decision - Include Feature
New algorithm: DP-Means

### Describe the workflow you want to enable

DP-Means is a nonparametric version of K-Means, which is rooted in the Dirichlet Process Mixture Model.
Proposed by[ Kulis and Jordan in 2011](https://arxiv.org/pdf/1111.0352.pdf), it is already a well cited and established algorithm.

Recently, we have published a version of it which scales very good, comparable to K-Means and is equivalent to the original algorithm (https://openreview.net/pdf?id=rnzVBD8jqlq).

Enabling it in scikit-learn would benefit the community, and would be great in my opinion.

### Describe your proposed solution

We have already utilized scikit-learn codebase in order to implement our algorithm efficiently, (https://github.com/BGU-CS-VIL/pdc-dp-means/tree/main/cluster), so I can easily fix this into a pull-request to scikit-learn code base (will need to add the relevant tests and docs as well).

As such, if this feature request is approved, I will work on the relevant pull request in order to add this to the code base.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
It is not better than the parametrized algorithms, it has a different use case, for example, if you do not know how many clusters you have, but instead know the desired cluster diameter it could be very useful. Another example is that when handling outliers they tend to pull the clusters towards their direction in K-Means, where in DP-Means the outliers would get clusters of there own, and will not affect the "main" clusters.

Not sure what you expect in approaching machine learning papers, this seems like a very general question. In this case I would suggest starting with the basics - e.g. K-Means, and the proceeding to reading the attached papers.

Labels: New Feature, module:cluster, Needs Decision - Include Feature
New algorithm: DP-Means

### Describe the workflow you want to enable

DP-Means is a nonparametric version of K-Means, which is rooted in the Dirichlet Process Mixture Model.
Proposed by[ Kulis and Jordan in 2011](https://arxiv.org/pdf/1111.0352.pdf), it is already a well cited and established algorithm.

Recently, we have published a version of it which scales very good, comparable to K-Means and is equivalent to the original algorithm (https://openreview.net/pdf?id=rnzVBD8jqlq).

Enabling it in scikit-learn would benefit the community, and would be great in my opinion.

### Describe your proposed solution

We have already utilized scikit-learn codebase in order to implement our algorithm efficiently, (https://github.com/BGU-CS-VIL/pdc-dp-means/tree/main/cluster), so I can easily fix this into a pull-request to scikit-learn code base (will need to add the relevant tests and docs as well).

As such, if this feature request is approved, I will work on the relevant pull request in order to add this to the code base.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
ok thank you so much i wil definetly try to dig deeper into the ml and then try to code it

Labels: New Feature, module:cluster, Needs Decision - Include Feature
New algorithm: DP-Means

### Describe the workflow you want to enable

DP-Means is a nonparametric version of K-Means, which is rooted in the Dirichlet Process Mixture Model.
Proposed by[ Kulis and Jordan in 2011](https://arxiv.org/pdf/1111.0352.pdf), it is already a well cited and established algorithm.

Recently, we have published a version of it which scales very good, comparable to K-Means and is equivalent to the original algorithm (https://openreview.net/pdf?id=rnzVBD8jqlq).

Enabling it in scikit-learn would benefit the community, and would be great in my opinion.

### Describe your proposed solution

We have already utilized scikit-learn codebase in order to implement our algorithm efficiently, (https://github.com/BGU-CS-VIL/pdc-dp-means/tree/main/cluster), so I can easily fix this into a pull-request to scikit-learn code base (will need to add the relevant tests and docs as well).

As such, if this feature request is approved, I will work on the relevant pull request in order to add this to the code base.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
and also i am new to open source so i dont know to connect to other devs so please pardon me

Labels: New Feature, module:cluster, Needs Decision - Include Feature
New algorithm: DP-Means

### Describe the workflow you want to enable

DP-Means is a nonparametric version of K-Means, which is rooted in the Dirichlet Process Mixture Model.
Proposed by[ Kulis and Jordan in 2011](https://arxiv.org/pdf/1111.0352.pdf), it is already a well cited and established algorithm.

Recently, we have published a version of it which scales very good, comparable to K-Means and is equivalent to the original algorithm (https://openreview.net/pdf?id=rnzVBD8jqlq).

Enabling it in scikit-learn would benefit the community, and would be great in my opinion.

### Describe your proposed solution

We have already utilized scikit-learn codebase in order to implement our algorithm efficiently, (https://github.com/BGU-CS-VIL/pdc-dp-means/tree/main/cluster), so I can easily fix this into a pull-request to scikit-learn code base (will need to add the relevant tests and docs as well).

As such, if this feature request is approved, I will work on the relevant pull request in order to add this to the code base.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I suppose that the main difference with already have under as [Variational Bayesian](https://scikit-learn.org/stable/modules/mixture.html#bgmm) (with the DP prior) is computational speed by having a more specialized Cython code. Or is there something else more fundamental?

Note that the current license of https://github.com/BGU-CS-VIL/pdc-dp-means (GPL) is not compatible with scikit-learn. So we would need the explicit permissions of all the authors and contributors of `pdc-dp-means` before merging any derivative code into the scikit-learn codebase which is BSD licensed.

Labels: New Feature, module:cluster, Needs Decision - Include Feature
New algorithm: DP-Means

### Describe the workflow you want to enable

DP-Means is a nonparametric version of K-Means, which is rooted in the Dirichlet Process Mixture Model.
Proposed by[ Kulis and Jordan in 2011](https://arxiv.org/pdf/1111.0352.pdf), it is already a well cited and established algorithm.

Recently, we have published a version of it which scales very good, comparable to K-Means and is equivalent to the original algorithm (https://openreview.net/pdf?id=rnzVBD8jqlq).

Enabling it in scikit-learn would benefit the community, and would be great in my opinion.

### Describe your proposed solution

We have already utilized scikit-learn codebase in order to implement our algorithm efficiently, (https://github.com/BGU-CS-VIL/pdc-dp-means/tree/main/cluster), so I can easily fix this into a pull-request to scikit-learn code base (will need to add the relevant tests and docs as well).

As such, if this feature request is approved, I will work on the relevant pull request in order to add this to the code base.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Accepting new estimators in scikit-learn might take time to reach a consensus. In this case the code size does not seem that large and seems like a natural extension to the current K-means code base (I did not look too carefully though) so I wouldn't be too pessimistic.

Still I think a first step would be to turn the [pdc-dp-means](https://github.com/BGU-CS-VIL/pdc-dp-means) repo into a properly tested and well maintained pip installable project with some documentation and usage examples (e.g. as rendered notebooks with with English text explaining the objectives of the notebook and the conclusions on the observed results).

This can be done either in the existing repo, or as a [scikit-learn-contrib](https://github.com/scikit-learn-contrib) project.

This work will not be wasted even if the decision for inclusion the main scikit-learn repo ends up negative as it would allow more end users to productively use this method in their usual Python environment and gain adoption. There is also the possibility to contribute to https://scikit-learn-extra.readthedocs.io/en/stable/.

Labels: New Feature, module:cluster, Needs Decision - Include Feature