Talip7/scikit-learn-issues-multilabel huggingface dataset:
The shape of dual_coef_ of KernelRidge, SVR

### Describe the bug

Fear of potential bugs.

The shape of dual_coef_ does not match in two models that use kernels (e.g. KernelRidge and SVR)

KernelRidge has a shape of (n_samples,)
SVR has a shape of (1, n_samples).

Is there a compelling reason for this discrepancy?

### Steps/Code to Reproduce

```
import sklearn
from sklearn.datasets import make_regression
from sklearn.kernel_ridge import KernelRidge
from sklearn.svm import SVR

X,y  = make_regression(n_samples=10, n_features=6)
kr = KernelRidge()
kr.fit(X,y)
print(kr.dual_coef_.shape)
svr = SVR()
svr.fit(X,y)
print(svr.dual_coef_.shape)
```

### Expected Results

Preferably in the form (n_samples,)

### Actual Results

>print(kr.dual_coef_.shape)
  (10,)

>print(svr.dual_coef_.shape)
  (1,10)

### Versions

```shell
System:
    python: 3.10.8 | packaged by conda-forge | (main, Nov  4 2022, 13:42:51) [MSC v.1916 64 bit (AMD64)]
executable: C:\Users\11665307\Anaconda3\envs\test\python.exe
   machine: Windows-10-10.0.19042-SP0

Python dependencies:
      sklearn: 1.1.3
          pip: 22.2.2
   setuptools: 65.5.0
        numpy: 1.23.5
        scipy: 1.9.3
       Cython: None
       pandas: None
   matplotlib: None
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
         prefix: vcomp
       filepath: C:\Users\11665307\Anaconda3\envs\test\Lib\site-packages\sklearn\.libs\vcomp140.dll
        version: None
    num_threads: 8

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: C:\Users\11665307\Anaconda3\envs\test\Lib\site-packages\numpy\.libs\libopenblas.FB5AE2TYXYH2IJRDKGDGQ3XBKLKTF43H.gfortran-win_amd64.dll
        version: 0.3.20
threading_layer: pthreads
   architecture: SkylakeX
    num_threads: 8

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: C:\Users\11665307\Anaconda3\envs\test\Lib\site-packages\scipy.libs\libopenblas-57db09cfe174768fb409a6bb5a530d4c.dll
        version: 0.3.18
threading_layer: pthreads
   architecture: SkylakeX
    num_threads: 8
```

COMMENT:
The shape observed is in line with our documentation. I assume we have a shape of `(1, n_support_vector)` for `SVR` due to implementation details. This should be the format expected by `libsvm`.

Labels: Bug, Needs Triage
MAINT Convert `samples` parameter in Criterion classes to memory view

## Summary

The `samples` parameter can be changed to a Cython memoryview from its current `SIZE_t* samples` type.

_Originally posted by @adam2392 in https://github.com/scikit-learn/scikit-learn/pull/24994#discussion_r1028818076_

COMMENT:
Hi @adam2392, I appreciate you open an issue but I do not think it is necessary for this one: this small refactoring has already been discussed elsewhere.

Labels: module:tree
⚠️ CI failed on Linux_Nightly_PyPy.pypy3 ⚠️

**CI failed on [Linux_Nightly_PyPy.pypy3](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=49087&view=logs&j=0b16f832-29d6-5b92-1c23-eb006f606a66)** (Nov 21, 2022)
Unable to find junit file. Please see link for details.

COMMENT:
## CI is no longer failing! ✅

[Successful run](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=49126&view=logs&j=0b16f832-29d6-5b92-1c23-eb006f606a66) on Nov 22, 2022

Labels: Needs Triage
⚠️ CI failed on Linux_Nightly_PyPy.pypy3 ⚠️

**CI failed on [Linux_Nightly_PyPy.pypy3](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=49087&view=logs&j=0b16f832-29d6-5b92-1c23-eb006f606a66)** (Nov 21, 2022)
Unable to find junit file. Please see link for details.

COMMENT:
The failure looks like a one time failure during the run, not from the test suite. IT did not trigger again. Let's close

Labels: Needs Triage
TransformerChain

### Describe the workflow you want to enable

1. To be able to encapsulate extraction and encoding of features from a source feature
2. To be able to turn off features derived from a source feature easier than having to turn off the extraction and encoding stages of the pipeline.
3. To be able to see the outcome of chaining many transformers without having to use a dummy estimator at the latest step of the pipeline.

### Describe your proposed solution

To have a new special transformer called `TransformerChain` which basically differs from `Pipeline` in that **all** of its steps have to be transformers. It simply gets a sequence of transformers as its constructor input and each step `fit`s on the transformed input of the previous step.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

To set an example, suppose we have a feature of type `string`, of which we extract 3 separate features using a *custom transformer*. Then we want to encode these features using proper encoders.

Currently, we could extract the features with the combination of a `ColumnTransformer` and a *custom transformer* and then encode those extracted features with proper encoder at the next step of the pipeline. This workflow is depicted in the following figure:
![image](https://user-images.githubusercontent.com/22097587/202909568-b8da4e6e-ab02-4240-99bf-34553816ac3b.png)

As stated above, this workflow has the drawback in its design that the extraction and encoding of those derived features is scattered across two separate steps of the pipeline. Moreover if were to turn off those features, we have to turn off two pipeline steps.

Another solution for user is to encapsulate all the extraction and encoding in the *custom transformer* and take care of many things him/herself. This approach is depicted below. Besides user's having to take care of many things him/herself this approach does not make the most of the Scikit's compositionality and modularity.
![image](https://user-images.githubusercontent.com/22097587/202910439-bf73effe-b271-422a-8c5d-dcf19542c6ac.png)

The proposed solution is shown in the image below. 
![image](https://user-images.githubusercontent.com/22097587/202910611-94f905c8-d565-420d-b2e7-f450bc26826b.png)

COMMENT:
You're right. The point "There is no need for the pipeline to end with a classifier or regressor" was a pivotal one. Thanks.

Labels: New Feature, Needs Triage
Sklearn pip installation broke because of issues with setuptools

### Describe the bug

Apparently because of this other issue https://github.com/pypa/setuptools/issues/3693

Installation of sklearn breaks with pip install right now.

```
File "/tmp/pip-build-env-mmiaw08q/overlay/lib/python3.10/site-packages/numpy/distutils/log.py", line 4, in <module>
           from distutils.log import Log as old_Log
       ImportError: cannot import name 'Log' from 'distutils.log' (/tmp/pip-build-env-mmiaw08q/overlay/lib/python3.10/site-packages/setuptools/_distutils/log.py)
```
I've read some other teams sticking to setuptools 65.5.1 for the moment, scikit learn seems to be aiming at 60+


### Steps/Code to Reproduce

```
pip install scikit_learn
```

### Expected Results

No error should be thrown

### Actual Results

```
File "/tmp/pip-build-env-mmiaw08q/overlay/lib/python3.10/site-packages/numpy/distutils/log.py", line 4, in <module>
           from distutils.log import Log as old_Log
       ImportError: cannot import name 'Log' from 'distutils.log' (/tmp/pip-build-env-mmiaw08q/overlay/lib/python3.10/site-packages/setuptools/_distutils/log.py)
```

### Versions

```shell
all versions depending on setuptools 60+
```

COMMENT:
Since #24563 this shouldn't be an issue anymore as numpy.distutils isn't used.

Labels: Bug, Needs Triage
Sklearn pip installation broke because of issues with setuptools

### Describe the bug

Apparently because of this other issue https://github.com/pypa/setuptools/issues/3693

Installation of sklearn breaks with pip install right now.

```
File "/tmp/pip-build-env-mmiaw08q/overlay/lib/python3.10/site-packages/numpy/distutils/log.py", line 4, in <module>
           from distutils.log import Log as old_Log
       ImportError: cannot import name 'Log' from 'distutils.log' (/tmp/pip-build-env-mmiaw08q/overlay/lib/python3.10/site-packages/setuptools/_distutils/log.py)
```
I've read some other teams sticking to setuptools 65.5.1 for the moment, scikit learn seems to be aiming at 60+


### Steps/Code to Reproduce

```
pip install scikit_learn
```

### Expected Results

No error should be thrown

### Actual Results

```
File "/tmp/pip-build-env-mmiaw08q/overlay/lib/python3.10/site-packages/numpy/distutils/log.py", line 4, in <module>
           from distutils.log import Log as old_Log
       ImportError: cannot import name 'Log' from 'distutils.log' (/tmp/pip-build-env-mmiaw08q/overlay/lib/python3.10/site-packages/setuptools/_distutils/log.py)
```

### Versions

```shell
all versions depending on setuptools 60+
```

COMMENT:
Yep, sorry guys, this works if I update the sklearn version to one that includes https://github.com/scikit-learn/scikit-learn/pull/24563 - the problem is probably only affecting previous versions but I guess that's fair.

Labels: Bug, Needs Triage
NEW FEATURE: MarginalSumsRegression

### Describe the workflow you want to enable

Adding Marginal Sums as a regression estimator. Marginal Sums are used in actuarial science to construct a multiplicative estimator: f1 * f2 * ... * fn * b = y with fi being a factor for every feature and b being a base value (the mean of the target variable).

### Describe your proposed solution

Currently working on a possible implementation in #24989

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

Actuarial science is working with big data for a long time and there are a lot of established methods to calculate premiums. Unfortunately, they are often implemented in proprietary languages, that are around for decades (e.g. SAS). The change towards open source is very hard. Especially, if you can't reproduce your existing models in the open source framework. The proposed new feature is one of these methods. Adding this to sklearn would allow actuarial scientists to port existing models into sklearn and compare them with more modern estimators (e.g. GradientBoostingRegressor).

Additionally teaching in actuarial science often requires lightweight/demo versions of said proprietary tools. I believe that unis should always be able to teach in open source languages.

COMMENT:
@lorentzenchr any thoughts regarding this method?

Labels: New Feature, Needs Triage
NEW FEATURE: MarginalSumsRegression

### Describe the workflow you want to enable

Adding Marginal Sums as a regression estimator. Marginal Sums are used in actuarial science to construct a multiplicative estimator: f1 * f2 * ... * fn * b = y with fi being a factor for every feature and b being a base value (the mean of the target variable).

### Describe your proposed solution

Currently working on a possible implementation in #24989

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

Actuarial science is working with big data for a long time and there are a lot of established methods to calculate premiums. Unfortunately, they are often implemented in proprietary languages, that are around for decades (e.g. SAS). The change towards open source is very hard. Especially, if you can't reproduce your existing models in the open source framework. The proposed new feature is one of these methods. Adding this to sklearn would allow actuarial scientists to port existing models into sklearn and compare them with more modern estimators (e.g. GradientBoostingRegressor).

Additionally teaching in actuarial science often requires lightweight/demo versions of said proprietary tools. I believe that unis should always be able to teach in open source languages.

COMMENT:
The method of marginal totals (MMT) predates GLMs, but is equivalent (same point estimates). MMT with a multiplicative structure is equivalent to a Poisson GLM with log link, i.e. [`PoissonRegressor`](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.PoissonRegressor.html#sklearn.linear_model.PoissonRegressor). If it is of interest, I can provide some references.

Labels: New Feature, Needs Triage
NEW FEATURE: MarginalSumsRegression

### Describe the workflow you want to enable

Adding Marginal Sums as a regression estimator. Marginal Sums are used in actuarial science to construct a multiplicative estimator: f1 * f2 * ... * fn * b = y with fi being a factor for every feature and b being a base value (the mean of the target variable).

### Describe your proposed solution

Currently working on a possible implementation in #24989

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

Actuarial science is working with big data for a long time and there are a lot of established methods to calculate premiums. Unfortunately, they are often implemented in proprietary languages, that are around for decades (e.g. SAS). The change towards open source is very hard. Especially, if you can't reproduce your existing models in the open source framework. The proposed new feature is one of these methods. Adding this to sklearn would allow actuarial scientists to port existing models into sklearn and compare them with more modern estimators (e.g. GradientBoostingRegressor).

Additionally teaching in actuarial science often requires lightweight/demo versions of said proprietary tools. I believe that unis should always be able to teach in open source languages.

COMMENT:
> The method of marginal totals (MMT) predates GLMs, but is equivalent (same point estimates). MMT with a multiplicative structure is equivalent to a Poisson GLM with log link, i.e. [`PoissonRegressor`](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.PoissonRegressor.html#sklearn.linear_model.PoissonRegressor). If it is of interest, I can provide some references.

Oh, that's actually good news, if it's already there. I'd be very happy, if you could provide some resources.

Labels: New Feature, Needs Triage
NEW FEATURE: MarginalSumsRegression

### Describe the workflow you want to enable

Adding Marginal Sums as a regression estimator. Marginal Sums are used in actuarial science to construct a multiplicative estimator: f1 * f2 * ... * fn * b = y with fi being a factor for every feature and b being a base value (the mean of the target variable).

### Describe your proposed solution

Currently working on a possible implementation in #24989

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

Actuarial science is working with big data for a long time and there are a lot of established methods to calculate premiums. Unfortunately, they are often implemented in proprietary languages, that are around for decades (e.g. SAS). The change towards open source is very hard. Especially, if you can't reproduce your existing models in the open source framework. The proposed new feature is one of these methods. Adding this to sklearn would allow actuarial scientists to port existing models into sklearn and compare them with more modern estimators (e.g. GradientBoostingRegressor).

Additionally teaching in actuarial science often requires lightweight/demo versions of said proprietary tools. I believe that unis should always be able to teach in open source languages.

COMMENT:
Ohlsson and Johansson "Non-Life Insurance Pricing with Generalized Linear Models" EAA Series Springer 2010
MMT was introduced by Baily "Insurance rates with minimum bias" Proceedings of the Casualty Actuarial Society, vol L (1963)

Labels: New Feature, Needs Triage
NEW FEATURE: MarginalSumsRegression

### Describe the workflow you want to enable

Adding Marginal Sums as a regression estimator. Marginal Sums are used in actuarial science to construct a multiplicative estimator: f1 * f2 * ... * fn * b = y with fi being a factor for every feature and b being a base value (the mean of the target variable).

### Describe your proposed solution

Currently working on a possible implementation in #24989

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

Actuarial science is working with big data for a long time and there are a lot of established methods to calculate premiums. Unfortunately, they are often implemented in proprietary languages, that are around for decades (e.g. SAS). The change towards open source is very hard. Especially, if you can't reproduce your existing models in the open source framework. The proposed new feature is one of these methods. Adding this to sklearn would allow actuarial scientists to port existing models into sklearn and compare them with more modern estimators (e.g. GradientBoostingRegressor).

Additionally teaching in actuarial science often requires lightweight/demo versions of said proprietary tools. I believe that unis should always be able to teach in open source languages.

COMMENT:
> Ohlsson and Johansson "Non-Life Insurance Pricing with Generalized Linear Models" EAA Series Springer 2010
> 
> MMT was introduced by Baily "Insurance rates with minimum bias" Proceedings of the Casualty Actuarial Society, vol L (1963)

Thank you very much!

Labels: New Feature, Needs Triage
Control default behavior of PR curve

### Describe the workflow you want to enable

Display the recall as a function of the predicted positive rate (PP) using `sklearn.metrics.precision_recall_curve` to compute the recall and PP as a quantiles of the threshold scores. Currently not possible to perform consistently as `sklearn.metrics.precision_recall_curve` drops a lot of threshold values corresponding recall = 1. This behavior has been recently introduced.

### Describe your proposed solution

Enable a `drop_intermediate` parameter in `sklearn.metrics.precision_recall_curve` similar to `sklearn.metrics.roc_curve` with default value = False and keep the extreme values of the threshold to avoid side effects.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

The documentation of `sklearn.metrics.precision_recall_curve` describes `n_thresholds = len(np.unique(probas_pred))`. This is not anymore the behavior of this function. This might cause a lot of backward incompatibility hence the suggested default value False.

Code to reproduce:

import numpy as np
import numpy.random as npr
import sklearn as sk
from sklearn.metrics import precision_recall_curve

scoresPredictor = np.arange(100)
groundTruth = np.concatenate((np.zeros(50), np.ones(50)))

precision_PR, recall_PR, thresholds_PR  = precision_recall_curve(groundTruth, scoresPredictor)

print(len(np.unique(scoresPredictor)))
print(len(thresholds_PR))

COMMENT:
There is a PR #24668 that implements the suggested fix. The issue that lead to the PR is #21825.

However the issue of values being dropped at recall=1 was also reported in #23213 and fixed in #23214.

I don't know when the behaviour of dropping recall=1 values was introduced.

Overall I think one of the reported issues here was already fixed and for the idea of `drop_intermediate` there is a PR already.

Could you test with [a nightly build](https://scikit-learn.org/stable/developers/advanced_installation.html#installing-nightly-builds) if the problem has been resolved?

Labels: New Feature, Needs Triage
Control default behavior of PR curve

### Describe the workflow you want to enable

Display the recall as a function of the predicted positive rate (PP) using `sklearn.metrics.precision_recall_curve` to compute the recall and PP as a quantiles of the threshold scores. Currently not possible to perform consistently as `sklearn.metrics.precision_recall_curve` drops a lot of threshold values corresponding recall = 1. This behavior has been recently introduced.

### Describe your proposed solution

Enable a `drop_intermediate` parameter in `sklearn.metrics.precision_recall_curve` similar to `sklearn.metrics.roc_curve` with default value = False and keep the extreme values of the threshold to avoid side effects.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

The documentation of `sklearn.metrics.precision_recall_curve` describes `n_thresholds = len(np.unique(probas_pred))`. This is not anymore the behavior of this function. This might cause a lot of backward incompatibility hence the suggested default value False.

Code to reproduce:

import numpy as np
import numpy.random as npr
import sklearn as sk
from sklearn.metrics import precision_recall_curve

scoresPredictor = np.arange(100)
groundTruth = np.concatenate((np.zeros(50), np.ones(50)))

precision_PR, recall_PR, thresholds_PR  = precision_recall_curve(groundTruth, scoresPredictor)

print(len(np.unique(scoresPredictor)))
print(len(thresholds_PR))

COMMENT:
The issue is fixed in `1.2.dev0` and `1.1.2`. I should have checked. This was the main purpose of the request, I think the issue can be closed.

Labels: New Feature, Needs Triage
⚠️ CI failed on Linux_Nightly_PyPy.pypy3 ⚠️

**CI failed on [Linux_Nightly_PyPy.pypy3](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=48973&view=logs&j=0b16f832-29d6-5b92-1c23-eb006f606a66)** (Nov 18, 2022)
- test_fit_and_score_verbosity[False-scorer2-10-split_prg2-cdt_prg2-\\[CV 2/3; 1/1\\] END ....... sc1: \\(test=3.421\\) sc2: \\(test=3.421\\) total time=   0.\\ds]

COMMENT:
## CI is no longer failing! ✅

[Successful run](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=49068&view=logs&j=0b16f832-29d6-5b92-1c23-eb006f606a66) on Nov 20, 2022

Labels: Needs Triage
⚠️ CI failed on Linux_Nightly_PyPy.pypy3 ⚠️

**CI failed on [Linux_Nightly_PyPy.pypy3](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=48973&view=logs&j=0b16f832-29d6-5b92-1c23-eb006f606a66)** (Nov 18, 2022)
- test_fit_and_score_verbosity[False-scorer2-10-split_prg2-cdt_prg2-\\[CV 2/3; 1/1\\] END ....... sc1: \\(test=3.421\\) sc2: \\(test=3.421\\) total time=   0.\\ds]

COMMENT:
If you say so

Labels: Needs Triage