Talip7/scikit-learn-issues-multilabel huggingface dataset:
GLM doesn't have an offset option

### Describe the workflow you want to enable

As described at #24155, GLM should also support the offset option.

### Describe your proposed solution

For example
```python
from sklearn.linear_model import PoissonRegressor

glm = PoissonRegressor(
                        alpha=0,
                        fit_intercept=False, 
                        max_iter=300,
                        offset=A # offset option
                      )
glm.fit(X_train, y_train)
```

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
The main problem with number 2 is that it requires to know the offsets at model creation time, which might not always be the case. In the pipeline example I posted above, where offsets are computed by the `CustomTransformation` based on the input data + some configuration, this would not work.

About number 1: that might work, we'll give it a spin and see how that would look 👍 

Spoiler: it will probably require creating an offset-aware wrapper of `LinearLoss`

Labels: New Feature, Needs Decision, module:linear_model, Needs Decision - Include Feature
GLM doesn't have an offset option

### Describe the workflow you want to enable

As described at #24155, GLM should also support the offset option.

### Describe your proposed solution

For example
```python
from sklearn.linear_model import PoissonRegressor

glm = PoissonRegressor(
                        alpha=0,
                        fit_intercept=False, 
                        max_iter=300,
                        offset=A # offset option
                      )
glm.fit(X_train, y_train)
```

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
For reference, glum uses the 3rd approach, i.e. argument in fit and predict, see https://glum.readthedocs.io/en/latest/glm.html#glum.GeneralizedLinearRegressor.fit.

Labels: New Feature, Needs Decision, module:linear_model, Needs Decision - Include Feature
GLM doesn't have an offset option

### Describe the workflow you want to enable

As described at #24155, GLM should also support the offset option.

### Describe your proposed solution

For example
```python
from sklearn.linear_model import PoissonRegressor

glm = PoissonRegressor(
                        alpha=0,
                        fit_intercept=False, 
                        max_iter=300,
                        offset=A # offset option
                      )
glm.fit(X_train, y_train)
```

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
> The main problem is that whenever offsets are not known a priori (e.g. computed during a previous step in a pipeline), it would be impossible to pass them to the GLM without creating a custom wrapper.

Is this a problem? Personally I would see `offset` as being very similar to `sample_weight`, so if it's fine to pass `sample_weight` into `.fit`, it should be fine to pass `offset`.

Labels: New Feature, Needs Decision, module:linear_model, Needs Decision - Include Feature
⚠️ CI failed on Wheel builder ⚠️

**CI is still failing on [Wheel builder](https://github.com/scikit-learn/scikit-learn/actions/runs/3278795901)** (Oct 19, 2022)

COMMENT:
The main problem that causes the windows test runs to fail (when importing scikit-learn modules to collect the tests) seems to be a missing compiled extension:

```python-traceback
  [...]
  Python\lib\site-packages\sklearn\datasets\__init__.py:22: in <module>
      from ._twenty_newsgroups import fetch_20newsgroups
  Python\lib\site-packages\sklearn\datasets\_twenty_newsgroups.py:46: in <module>
      from ..feature_extraction.text import CountVectorizer
  Python\lib\site-packages\sklearn\feature_extraction\__init__.py:8: in <module>
      from ._hash import FeatureHasher
  Python\lib\site-packages\sklearn\feature_extraction\_hash.py:10: in <module>
      from ._hashing_fast import transform as _hashing_transform
  E   ImportError: DLL load failed while importing _hashing_fast: The specified module could not be found.
```

Labels: Bug, Build / CI, Blocker
⚠️ CI failed on Wheel builder ⚠️

**CI is still failing on [Wheel builder](https://github.com/scikit-learn/scikit-learn/actions/runs/3278795901)** (Oct 19, 2022)

COMMENT:
The absence of compiled extension is the source of failure for the Windows 64bit wheels only, that is:
https://github.com/scikit-learn/scikit-learn/blob/fd0e8158a408b79bc7c43b4de7891640bb0fe809/.github/workflows/wheels.yml#L58-L69

The other Windows 32bit wheels, that is:
https://github.com/scikit-learn/scikit-learn/blob/fd0e8158a408b79bc7c43b4de7891640bb0fe809/.github/workflows/wheels.yml#L71-L79

aren't built because SciPy needs to be built from source and a Fortran compilation error occurs as stated in https://github.com/scikit-learn/scikit-learn/pull/24446#issuecomment-1272472108:

```
 ..\..\meson.build:49:0: ERROR: Executables created by Fortran compiler gfortran are not runnable.
```

this might be related to the lack of a compiler toolchain for win32 as stated in [this thread in Scientific Python's discourse](https://discuss.scientific-python.org/t/releasing-or-not-32-bit-windows-wheels/282):

> For SciPy 1.9.0 we are switching to Meson as the default build system, with the numpy.distutils based build still available as a backup. Currently there is no compiler toolchain which works with the Meson build and allows us to build 32-bit wheels for distribution.

Labels: Bug, Build / CI, Blocker
⚠️ CI failed on Wheel builder ⚠️

**CI is still failing on [Wheel builder](https://github.com/scikit-learn/scikit-learn/actions/runs/3278795901)** (Oct 19, 2022)

COMMENT:
Thanks. It's weird because there is a 32 bit windows build for scipy 1.9.1 but not for 1.9.2:

https://pypi.org/project/scipy/1.9.1/#files

I am pinning 1.9.1 for the regular CI in #24621 but I guess we will need something similar for the wheels or maybe we can temporarily (?) stop building wheels for 32 bit windows while scipy does not support it.

Labels: Bug, Build / CI, Blocker
⚠️ CI failed on Wheel builder ⚠️

**CI is still failing on [Wheel builder](https://github.com/scikit-learn/scikit-learn/actions/runs/3278795901)** (Oct 19, 2022)

COMMENT:
> The absence of compiled extension is the source of failure for the Windows 64bit wheels only.

This point still needs investigation though.

Labels: Bug, Build / CI, Blocker
⚠️ CI failed on Wheel builder ⚠️

**CI is still failing on [Wheel builder](https://github.com/scikit-learn/scikit-learn/actions/runs/3278795901)** (Oct 19, 2022)

COMMENT:
As discussed in #24621, we probably should find a way to let pip use `--prefer-binary` when installing the build and test dependencies. Not sure how to do that with `cibuildwheel` though.

Labels: Bug, Build / CI, Blocker
⚠️ CI failed on Wheel builder ⚠️

**CI is still failing on [Wheel builder](https://github.com/scikit-learn/scikit-learn/actions/runs/3278795901)** (Oct 19, 2022)

COMMENT:
@ogrisel Could this be a feature request for [cibuildwheel](https://github.com/pypa/cibuildwheel) itself?

Labels: Bug, Build / CI, Blocker
⚠️ CI failed on Wheel builder ⚠️

**CI is still failing on [Wheel builder](https://github.com/scikit-learn/scikit-learn/actions/runs/3278795901)** (Oct 19, 2022)

COMMENT:
> > The absence of compiled extension is the source of failure for the Windows 64bit wheels only.
> 
> This point still needs investigation though.

This point is solved by https://github.com/scikit-learn/scikit-learn/pull/24446/files#diff-e0a07d85249fa03c06895043f2d57076b4a1a30b195631e1682b5470db6a5ac3
Should I offer a separate PR for that?

Sorry for cross-posting, trying to make myself read... ;)

Labels: Bug, Build / CI, Blocker
⚠️ CI failed on Wheel builder ⚠️

**CI is still failing on [Wheel builder](https://github.com/scikit-learn/scikit-learn/actions/runs/3278795901)** (Oct 19, 2022)

COMMENT:
Looking at the fix in https://github.com/scikit-learn/scikit-learn/pull/24631, I think the issue with Windows 64bit is that SciPy 1.9.2 no longer ships with `msvcp140.dll`. Unfortunately, I suspect that earlier versions of scikit-learn will not work with SciPy 1.9.2 either.

**Edit**: I opened an issue about this before in SciPy: https://github.com/scipy/scipy/issues/14998

Labels: Bug, Build / CI, Blocker
⚠️ CI failed on Wheel builder ⚠️

**CI is still failing on [Wheel builder](https://github.com/scikit-learn/scikit-learn/actions/runs/3278795901)** (Oct 19, 2022)

COMMENT:
> This point is solved by https://github.com/scikit-learn/scikit-learn/pull/24446/files#diff-e0a07d85249fa03c06895043f2d57076b4a1a30b195631e1682b5470db6a5ac3
Should I offer a separate PR for that?

I missed that. I think it's up to scipy to fix their release artifacts. We just have to wait a bit.

Labels: Bug, Build / CI, Blocker
⚠️ CI failed on Wheel builder ⚠️

**CI is still failing on [Wheel builder](https://github.com/scikit-learn/scikit-learn/actions/runs/3278795901)** (Oct 19, 2022)

COMMENT:
I opened a new issue for SciPy 1.9.2 here: https://github.com/scipy/scipy/issues/17191. I suspect that SciPy removed `msvcp140.dll` intentionally, which means that we need to develop our own workaround. For SciPy, it looks like they determined that `msvcp140.dll` is not needed anymore.

XREF: https://github.com/scipy/oldest-supported-numpy/issues/45

Labels: Bug, Build / CI, Blocker
⚠️ CI failed on Wheel builder ⚠️

**CI is still failing on [Wheel builder](https://github.com/scikit-learn/scikit-learn/actions/runs/3278795901)** (Oct 19, 2022)

COMMENT:
So the claim in https://github.com/scipy/oldest-supported-numpy/issues/45 is _All versions of Python currently supported should already have `msvcp140.dll`._ If that is true, what I guess is happening is that scikit-learn depends on it implicitly, and is counting on it being loaded in memory by Python or SciPy. However, if Python and SciPy both don't actually need the DLL anymore because they're now relying on the Windows [Universal Runtime](https://learn.microsoft.com/en-us/cpp/windows/universal-crt-deployment?view=msvc-170) (`ucrtbase.dll` & co), then it won't help scikit-learn to have the DLL bundled into SciPy. It just sits there in some random directory that's not on the library search path. 

Both newer MSVC versions and Mingw-w64 (which is what the SciPy wheels are now built with) should depend on the UCRT rather than on the older `msvcp140.dll`. So this explanation seems plausible.

You may also get end user issues who happen to not have any other program installed that already dumped the DLL into a system directory (from the last time SciPy got a bug report about that, this is rare though).

Labels: Bug, Build / CI, Blocker
⚠️ CI failed on Wheel builder ⚠️

**CI is still failing on [Wheel builder](https://github.com/scikit-learn/scikit-learn/actions/runs/3278795901)** (Oct 19, 2022)

COMMENT:
## CI is no longer failing! ✅

[Successful run](https://github.com/scikit-learn/scikit-learn/actions/runs/3286761573) on Oct 20, 2022

Labels: Bug, Build / CI, Blocker
MiniBatchDictionaryLearning confusion over U,V

### Describe the issue linked to the documentation

https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.MiniBatchDictionaryLearning.html

The optimization problem in the documentation is described as

$$
\arg\min \frac12 \Vert X-UV\Vert_F^2+\alpha \Vert U\Vert_1
$$
 
such that $\Vert V_k\Vert_2\leq 1$ for all $k$.

This is very confusing, because generally $U$ is the dictionary matrix, and $V$ is the representation matrix (e.g. see the paper linked in the documentation https://www.di.ens.fr/sierra/pdfs/icml09.pdf)

So it should read

$$
\arg\min \frac12 \Vert X-UV\Vert_F^2+\alpha \Vert V\Vert_1
$$
 
such that $\Vert U_k\Vert_2\leq 1$ for all $k$; here $U_k$ denotes the columns of $U$. 

In other words, the $L_1$ regularization should be performed on the representation matrix $V$, and the $L_2$ norm constraints on the columns (basis vectors) of the matrix $U$.

### Suggest a potential alternative/fix

It should read

$$
\arg\min \frac12 \Vert X-UV\Vert_F^2+\alpha \Vert V\Vert_1
$$
 
such that $\Vert U_k\Vert_2\leq 1$ for all $k$; here $U_k$ denotes the columns of $U$.

COMMENT:
On further thought, I think the documentation as written is correct; however it is confusing in that $V$ is actually the dictionary matrix and $U$ is the representation matrix (I would expect it to be the other way around).

Labels: Documentation
MiniBatchDictionaryLearning confusion over U,V

### Describe the issue linked to the documentation

https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.MiniBatchDictionaryLearning.html

The optimization problem in the documentation is described as

$$
\arg\min \frac12 \Vert X-UV\Vert_F^2+\alpha \Vert U\Vert_1
$$
 
such that $\Vert V_k\Vert_2\leq 1$ for all $k$.

This is very confusing, because generally $U$ is the dictionary matrix, and $V$ is the representation matrix (e.g. see the paper linked in the documentation https://www.di.ens.fr/sierra/pdfs/icml09.pdf)

So it should read

$$
\arg\min \frac12 \Vert X-UV\Vert_F^2+\alpha \Vert V\Vert_1
$$
 
such that $\Vert U_k\Vert_2\leq 1$ for all $k$; here $U_k$ denotes the columns of $U$. 

In other words, the $L_1$ regularization should be performed on the representation matrix $V$, and the $L_2$ norm constraints on the columns (basis vectors) of the matrix $U$.

### Suggest a potential alternative/fix

It should read

$$
\arg\min \frac12 \Vert X-UV\Vert_F^2+\alpha \Vert V\Vert_1
$$
 
such that $\Vert U_k\Vert_2\leq 1$ for all $k$; here $U_k$ denotes the columns of $U$.

COMMENT:
Hi @Pilex1, your confusion probably comes from the fact that X, the dictionary and the representation are sometimes transposed in the litterature compared to scikit-learn. In sklearn we have the general convention to have X of shape (n_samples, n_features), the dictionary (n_components, n_features) and the representation (n_samples, n_components).

Labels: Documentation
MiniBatchDictionaryLearning confusion over U,V

### Describe the issue linked to the documentation

https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.MiniBatchDictionaryLearning.html

The optimization problem in the documentation is described as

$$
\arg\min \frac12 \Vert X-UV\Vert_F^2+\alpha \Vert U\Vert_1
$$
 
such that $\Vert V_k\Vert_2\leq 1$ for all $k$.

This is very confusing, because generally $U$ is the dictionary matrix, and $V$ is the representation matrix (e.g. see the paper linked in the documentation https://www.di.ens.fr/sierra/pdfs/icml09.pdf)

So it should read

$$
\arg\min \frac12 \Vert X-UV\Vert_F^2+\alpha \Vert V\Vert_1
$$
 
such that $\Vert U_k\Vert_2\leq 1$ for all $k$; here $U_k$ denotes the columns of $U$. 

In other words, the $L_1$ regularization should be performed on the representation matrix $V$, and the $L_2$ norm constraints on the columns (basis vectors) of the matrix $U$.

### Suggest a potential alternative/fix

It should read

$$
\arg\min \frac12 \Vert X-UV\Vert_F^2+\alpha \Vert V\Vert_1
$$
 
such that $\Vert U_k\Vert_2\leq 1$ for all $k$; here $U_k$ denotes the columns of $U$.

COMMENT:
Ah I see, thanks!

Labels: Documentation
Remove `sum_over_features` from `sklearn.metrics.manhattan_distances`

`sklearn.metrics.manhattan_distances` with `sum_over_features=False` does not return distances, looks unused, and is also unlikely used by users.

I think the behavior for `sum_over_features=True` should be the unique one for `manhattan_distances`; in this case, `sum_over_features` would be removed this from the public API after [a deprecation cycle](https://scikit-learn.org/stable/developers/contributing.html#deprecation).

This was first discussed in https://github.com/scikit-learn/scikit-learn/pull/23958#discussion_r983137884.

COMMENT:
+1

Labels: module:metrics
Remove `sum_over_features` from `sklearn.metrics.manhattan_distances`

`sklearn.metrics.manhattan_distances` with `sum_over_features=False` does not return distances, looks unused, and is also unlikely used by users.

I think the behavior for `sum_over_features=True` should be the unique one for `manhattan_distances`; in this case, `sum_over_features` would be removed this from the public API after [a deprecation cycle](https://scikit-learn.org/stable/developers/contributing.html#deprecation).

This was first discussed in https://github.com/scikit-learn/scikit-learn/pull/23958#discussion_r983137884.

COMMENT:
I'd like to take this up. Should I just check whether the `sum_over_features` parameter is being used by the user and if it is, just display a deprecation warning and continue to return output for `sum_over_features=True`, irrespective of what the user set its value to? Or should I keep the function behavior the same and just display a deprecation warning if the user has used the parameter?

Labels: module:metrics
Remove `sum_over_features` from `sklearn.metrics.manhattan_distances`

`sklearn.metrics.manhattan_distances` with `sum_over_features=False` does not return distances, looks unused, and is also unlikely used by users.

I think the behavior for `sum_over_features=True` should be the unique one for `manhattan_distances`; in this case, `sum_over_features` would be removed this from the public API after [a deprecation cycle](https://scikit-learn.org/stable/developers/contributing.html#deprecation).

This was first discussed in https://github.com/scikit-learn/scikit-learn/pull/23958#discussion_r983137884.

COMMENT:
Thanks for this proposal, @rusdes.

The suitable way to go through a deprecation cycle, that is to support `sum_over_features=False` for two more minor releases whilst raising a warning in this period.

Regarding introducing a deprecating cycle, see:
 - [this section of documentation](https://scikit-learn.org/stable/developers/contributing.html#deprecation) for a comprehensive overview
 - https://github.com/scikit-learn/scikit-learn/pull/24523/ for an example

Labels: module:metrics
`LabelEncoder` is not compatible with `Pipeline`

### Describe the bug

Using the `LabelEncoder` inside a pipeline leads to errors.

The possible solution could be to introduce an auxiliary parameter to the `fit` function, similarly as it was done for `OneHotEncoder`:
https://github.com/scikit-learn/scikit-learn/blob/8694eb00f8a3c0dede331fe60c0415bfaafef631/sklearn/preprocessing/_encoders.py#L794-L806

### Steps/Code to Reproduce

```python
from sklearn import preprocessing, pipeline

label_encoder = preprocessing.LabelEncoder()
pipe = pipeline.Pipeline(steps=[('encoder', label_encoder)])
pipe.fit_transform([['a'], ['b'], ['c']])
```

### Expected Results

No error is thrown, `pipe.fit_transform` returns `[[0], [1], [2]]`

### Actual Results

```
Traceback (most recent call last):
  File ".../issue-demo.py", line 5, in <module>
    pipe.fit_transform([['a'], ['b'], ['c']])
  File ".../python3.10/site-packages/sklearn/pipeline.py", line 422, in fit_transform
    return last_step.fit_transform(Xt, y, **fit_params_last_step)
TypeError: LabelEncoder.fit_transform() takes 2 positional arguments but 3 were given
```

### Versions

```shell
System:
    python: 3.10.6 (main, Aug  9 2022, 18:38:26) [Clang 13.1.6 (clang-1316.0.21.2.5)]
executable: /reductedbin/python
   machine: macOS-12.4-x86_64-i386-64bit

Python dependencies:
      sklearn: 1.2.dev0
          pip: 22.2.2
   setuptools: 65.3.0
        numpy: 1.23.3
        scipy: 1.9.1
       Cython: None
       pandas: None
   matplotlib: None
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: False

threadpoolctl info:
       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /reductedlib/python3.10/site-packages/numpy/.dylibs/libopenblas64_.0.dylib
        version: 0.3.20
threading_layer: pthreads
   architecture: Haswell
    num_threads: 4

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /reductedlib/python3.10/site-packages/scipy/.dylibs/libopenblas.0.dylib
        version: 0.3.18
threading_layer: pthreads
   architecture: Haswell
    num_threads: 4
```

COMMENT:
Meanwhile I can suggest the following workaround:
```
def apply_fix():
    """
    LabelEncoder is not compatible with Pipeline, this is a workaround
    see https://github.com/scikit-learn/scikit-learn/issues/24596
    """
    fit = preprocessing.LabelEncoder.fit
    fit_fix = lambda self, X, y = None: fit(self, X)
    preprocessing.LabelEncoder.fit = fit_fix

    fit_transform = preprocessing.LabelEncoder.fit_transform
    fit_transform_fix = lambda self, X, y = None: fit_transform(self, X)
    preprocessing.LabelEncoder.fit_transform = fit_transform_fix
```

Labels: Bug, Needs Triage
`LabelEncoder` is not compatible with `Pipeline`

### Describe the bug

Using the `LabelEncoder` inside a pipeline leads to errors.

The possible solution could be to introduce an auxiliary parameter to the `fit` function, similarly as it was done for `OneHotEncoder`:
https://github.com/scikit-learn/scikit-learn/blob/8694eb00f8a3c0dede331fe60c0415bfaafef631/sklearn/preprocessing/_encoders.py#L794-L806

### Steps/Code to Reproduce

```python
from sklearn import preprocessing, pipeline

label_encoder = preprocessing.LabelEncoder()
pipe = pipeline.Pipeline(steps=[('encoder', label_encoder)])
pipe.fit_transform([['a'], ['b'], ['c']])
```

### Expected Results

No error is thrown, `pipe.fit_transform` returns `[[0], [1], [2]]`

### Actual Results

```
Traceback (most recent call last):
  File ".../issue-demo.py", line 5, in <module>
    pipe.fit_transform([['a'], ['b'], ['c']])
  File ".../python3.10/site-packages/sklearn/pipeline.py", line 422, in fit_transform
    return last_step.fit_transform(Xt, y, **fit_params_last_step)
TypeError: LabelEncoder.fit_transform() takes 2 positional arguments but 3 were given
```

### Versions

```shell
System:
    python: 3.10.6 (main, Aug  9 2022, 18:38:26) [Clang 13.1.6 (clang-1316.0.21.2.5)]
executable: /reductedbin/python
   machine: macOS-12.4-x86_64-i386-64bit

Python dependencies:
      sklearn: 1.2.dev0
          pip: 22.2.2
   setuptools: 65.3.0
        numpy: 1.23.3
        scipy: 1.9.1
       Cython: None
       pandas: None
   matplotlib: None
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: False

threadpoolctl info:
       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /reductedlib/python3.10/site-packages/numpy/.dylibs/libopenblas64_.0.dylib
        version: 0.3.20
threading_layer: pthreads
   architecture: Haswell
    num_threads: 4

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /reductedlib/python3.10/site-packages/scipy/.dylibs/libopenblas.0.dylib
        version: 0.3.18
threading_layer: pthreads
   architecture: Haswell
    num_threads: 4
```

COMMENT:
A `LabelEncoder` is not supposed to be applied on `X` (only on `y` as a preprocessing) and therefore should not be included in a `Pipeline`. Use a `OneHotEncoder` for this purpose.

Labels: Bug, Needs Triage