Talip7/scikit-learn-issues-multilabel huggingface dataset:
RFC SLEP006: allow users to enable a "strict" mode in metadata routing

ref: https://github.com/scikit-learn/scikit-learn/pull/22986/files#r862344847

Along the way we've had discussions on whether we should raise if a metadata is requested but not provided.

Mirroring the existing behavior, SLEP006 and the implementation won't raise in the following case:

``` python
pipeline = make_pipeline(LogisticRegression().set_fit_request(sample_weight=True)).fit(X, y)
```

Should we allow a strict mode, either globally or on the estimator level, which would make the above code raise?

COMMENT:
Some options out there:

- Rename `RequestType.REQUESTED` to `RequestType.EXPECTED` and add `RequestType.ACCEPTED`. The current `True` value can be the proxy for either of them.
- Introduce a `"strict"` mode, the default can be either `True` or `False`
- rename `set_fit_request` to `set_fit_metadata_acceptance` (or similar)
- Do nothing for now

I'm personally happy with either of the above solutions.

Labels: API, RFC
RFC SLEP006: allow users to enable a "strict" mode in metadata routing

ref: https://github.com/scikit-learn/scikit-learn/pull/22986/files#r862344847

Along the way we've had discussions on whether we should raise if a metadata is requested but not provided.

Mirroring the existing behavior, SLEP006 and the implementation won't raise in the following case:

``` python
pipeline = make_pipeline(LogisticRegression().set_fit_request(sample_weight=True)).fit(X, y)
```

Should we allow a strict mode, either globally or on the estimator level, which would make the above code raise?

COMMENT:
`set_fit_receives`? I don't see a lot of risk (albeit some user confusion) from these staying as optional.

Labels: API, RFC
RFC SLEP006: allow users to enable a "strict" mode in metadata routing

ref: https://github.com/scikit-learn/scikit-learn/pull/22986/files#r862344847

Along the way we've had discussions on whether we should raise if a metadata is requested but not provided.

Mirroring the existing behavior, SLEP006 and the implementation won't raise in the following case:

``` python
pipeline = make_pipeline(LogisticRegression().set_fit_request(sample_weight=True)).fit(X, y)
```

Should we allow a strict mode, either globally or on the estimator level, which would make the above code raise?

COMMENT:
The more we think about this, and in combination with the work on default/auto routing efforts, it seems we're not too excited about this feature, and it doesn't really enable a usecase where it's not possible right now.

The issue comes back to the fact that `set_{method}_request` is not really a good name for what it's doing, which is "if you put me in a router, route this metadata, if provided, to me, in this way". We haven't succeeded at finding a nice short method name for this.

Labels: API, RFC
Refactor metadata routing classes used in tests

The work related to metadata routing has introduced quite a few classes in the test files in order to test different functionalities and different cases.

Once we have advanced more in the implementation, we should refactor those classes, find common patterns, and clean them up, and maybe move some of them beside the other classes used in testing where the `MinimalClassifier` is also implemented.

Related to https://github.com/scikit-learn/scikit-learn/issues/22893

COMMENT:
Did https://github.com/scikit-learn/scikit-learn/pull/22986 fix this? What else is there to do?

Labels: Refactor
Refactor metadata routing classes used in tests

The work related to metadata routing has introduced quite a few classes in the test files in order to test different functionalities and different cases.

Once we have advanced more in the implementation, we should refactor those classes, find common patterns, and clean them up, and maybe move some of them beside the other classes used in testing where the `MinimalClassifier` is also implemented.

Related to https://github.com/scikit-learn/scikit-learn/issues/22893

COMMENT:
https://github.com/scikit-learn/scikit-learn/pull/26894 worked towards this, @thomasjpfan is the code now at a state where you'd consider this issue fixed?

Labels: Refactor
Refactor metadata routing classes used in tests

The work related to metadata routing has introduced quite a few classes in the test files in order to test different functionalities and different cases.

Once we have advanced more in the implementation, we should refactor those classes, find common patterns, and clean them up, and maybe move some of them beside the other classes used in testing where the `MinimalClassifier` is also implemented.

Related to https://github.com/scikit-learn/scikit-learn/issues/22893

COMMENT:
Yup, I think we can consider this issue fixed.

Labels: Refactor
KNeighborsTransformer and RadiusNeighborsTransformer have n_jobs=1 as default (instead of None)

### Discussed in https://github.com/scikit-learn/scikit-learn/discussions/23888

<div type='discussions-op-text'>

<sup>Originally posted by **Valentin-Laurent** July 12, 2022</sup>
While working on #22348, I realized that attribute `n_jobs` of classes `KNeighborsTransformer` and `RadiusNeighborsTransformer` is `1` as default (see [_graph.py](https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/neighbors/_graph.py)).

These are the only 2 examples of public classes that doesn't have `n_jobs=None` as default (which can result in a different behavior than `n_jobs=1` if using joblib).

Is this on purpose, or should we fix it (along with the doc)? I'd be glad to take care of it (new contributor here :))</div>

COMMENT:
I'm okay with moving this to `n_jobs=None`. Based on the [joblib.Parallel](https://joblib.readthedocs.io/en/latest/generated/joblib.Parallel.html) docs the behavior for `None` is:

> None is a marker for ‘unset’ that will be interpreted as n_jobs=1 (sequential execution) unless the call is performed under a [parallel_backend()](https://joblib.readthedocs.io/en/latest/parallel.html#joblib.parallel_backend) context manager that sets another value for n_jobs.

Because of this technically changing to `None` is backward incompatible, but I see setting `n_jobs=None` more as a bug fix. For me, `parallel_backend`, should work.

Labels: module:neighbors
KNeighborsTransformer and RadiusNeighborsTransformer have n_jobs=1 as default (instead of None)

### Discussed in https://github.com/scikit-learn/scikit-learn/discussions/23888

<div type='discussions-op-text'>

<sup>Originally posted by **Valentin-Laurent** July 12, 2022</sup>
While working on #22348, I realized that attribute `n_jobs` of classes `KNeighborsTransformer` and `RadiusNeighborsTransformer` is `1` as default (see [_graph.py](https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/neighbors/_graph.py)).

These are the only 2 examples of public classes that doesn't have `n_jobs=None` as default (which can result in a different behavior than `n_jobs=1` if using joblib).

Is this on purpose, or should we fix it (along with the doc)? I'd be glad to take care of it (new contributor here :))</div>

COMMENT:
I opened a PR (#24075) considering this is a bug fix. Happy to update the PR with a deprecation cycle if needed.

Labels: module:neighbors
Add fit_predict to linear_models?

### Describe the workflow you want to enable

AFAICT, the regressor classes in `sklearn.linear_models` do not implement a fit_predict method (which is present in many other estimator classes).  Having it would be a nice convenience.

### Describe your proposed solution

Add something like `def fit_predict(self, X, y): self.fit(X, y); return self.predict(X)` to an appropriately determined base class.

### Describe alternatives you've considered, if relevant

One can always call `fit` and `predict` directly, but given that the fit_predict convenience method has been added to some estimators, it seems a pity not to have it everywhere applicable.

### Additional context

_No response_

COMMENT:
`fit_predict` for supervised learning is not a great idea. Indeed, you are generally not interested in fitting and predicting on the same set. The general pattern would be to `fit` on a training set and predict on a test set, so:

```
model.fit(X_train, y_train).predict(X_test)
```

The only place where we allow `fit_predict` is in clustering (i.e. unsupervised learning) where you will be interested to do fit-predict on the same set.

If you really want to get predictions on the training set, you can still pipeline both calls:

```python
model.fit(X_train, y_train).predict(X_train)
```

So I am closing because I am sure that we don't want to change our API here because it could become confusing to have `fit_predict` for supervised models.

Labels: New Feature, Needs Triage
AUC differences between cross_validate and cross_val_predict

### Describe the bug

I am working through a binary classification problem and I need to generate an ROC curve. When comparing the performances of the function cross_validate (which returns performance metrics), with cross_val_predict (which can return probabilities), the AUC can vary largely from these two functions when being given the same model and data.

This is especially difficult to work with when cross_validate returns much better roc_auc values than cross_val_predict.

This has previously been discussed here:
https://stackoverflow.com/questions/45074392/differences-in-roc-auc-values-with-cross-val-score-and-cross-val-predict

### Steps/Code to Reproduce


model = LogisticRegression(class_weight='balanced', penalty='l1', solver='liblinear')
scores = cross_validate(clone(model), x, y, cv=5, scoring="roc_auc")
print(scores)

predictions = cross_val_predict(model, x, y, cv=5, method="predict_proba")

print(roc_auc_score(y, predictions[:, 1])






### Expected Results

1     roc_auc  0.727598  0.056468

0.7275985663082437

### Actual Results

1     roc_auc  0.768172  0.056468

0.7275985663082437

### Versions

```shell
System:
    python: 3.7.13 (default, Mar 28 2022, 08:03:21) [MSC v.1916 64 bit (AMD64)]
executable: C:\envs\vision\python.exe
   machine: Windows-10-10.0.19041-SP0

Python dependencies:
          pip: 22.1.2
   setuptools: 61.2.0
      sklearn: 1.0.2
        numpy: 1.21.5
        scipy: 1.7.3
       Cython: None
       pandas: 1.3.4
   matplotlib: 3.5.1
       joblib: 1.1.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

COMMENT:
In addition to @TomDLT, you should not use `cross_val_predict` for evaluation in general (as shown in the documentation).

Labels: Bug, Needs Triage
Issue with affinity propagation and dtype np.float32

While working on https://github.com/scikit-learn/scikit-learn/pull/23838, it seems that the issue observed in https://github.com/scikit-learn/scikit-learn/issues/10832 was not solved since it breaks for different random seeds.

Therefore, we should revisit the original issue.

COMMENT:
The snippet reported in #10832
```py
import sklearn.cluster
import numpy as np

k = np.array([[1,0,0,0],
              [0,1,1,0],
              [0,1,1,0],
              [0,0,0,1]], dtype='float32')

afp = sklearn.cluster.AffinityPropagation(preference=1, affinity='precomputed').fit(k)
print(afp.labels_)
```
expects 3 clusters, with labels [0,1,1,2]. It actually has nothing to do with float32 or float64: it breaks for some random seeds whether k is float64 or float32.

I'm not familiar with this estimator so I don't know if this clustering should be expected regardless of the noise added during fit.

Labels: Bug, module:cluster
Vector Quantization Example increases rather than decreases memory use

### Describe the issue linked to the documentation

The [Vector Quantization Example](https://scikit-learn.org/stable/auto_examples/cluster/plot_face_compress.html) doesn't seem to demonstrate Vector Quantization.  

As written, the kmeans clustering approach used in the example converts a grey scale `uint8` `face` to an `int32` representation (`labels`). This increases the image memory use by 4x. 

```python
print(f'face dtype: {face.dtype}')
print(f'face bytes: {face.nbytes}')
print(f'labels dtype: {labels.dtype}')
print(f'labels bytes: {labels.nbytes}')
```
```
face dtype: uint8
face bytes: 786432
labels dtype: int32
labels bytes: 3145728
```

**Expected output**
Vector quantization output demonstrates a decrease in memory use.


**Additional details**
From [Wikipedia](https://en.wikipedia.org/wiki/Vector_quantization): "Vector quantization, also called "block quantization" or "pattern matching quantization" is often used in lossy data compression. It works by encoding values from a multidimensional vector space into a finite set of values from a discrete subspace of lower dimension. A lower-space vector requires less storage space, so the data is compressed."

I'm guessing kmeans outputs an `int32` by default.  The cluster `labels` are in the range 0,1,2,3,4. While this could be compressed to a 4 bit integer, `uint8` is as small as we can go with numpy so the example does not effectively illustrate the data compression. 

Perhaps the tutorial assumption is that the values contained in `labels` could be compressed through some other algorithm (e.g. outside of numpy). However, for someone unfamiliar with Vector Quantization, it may seem odd why someone would quantize a vector in a way that both loses information and increases memory use.

### Suggest a potential alternative/fix

1. add a comment to clarify that the quantized representation of the original face could be further compressed by another algorithm
2. replace the gray image with a color image and cast the kmeans output to a `uint8` to demonstrate the compression. Converting 3, 8 bit channels to one 8 bit channel would reduce nbytes by 67%.

COMMENT:
Indeed, this tutorial should be revised.

Regarding the point raised by @BlaneG, I think that we can illustrate the compression by counting the number of unique values. We should as well comment regarding the data type when it comes to the in-memory compression.

Regarding the code, we should replace `KMeans` with the `KBinsDiscretizer` that allows to switch from `kmeans` to `uniform` quantization and would require only calling `transform`. This will make the code easier to understand, and the discretization is actually the job of `KBinsDiscretizer` :)

@BlaneG do you wish to do a PR to improve this example?

Labels: Documentation, good first issue
Vector Quantization Example increases rather than decreases memory use

### Describe the issue linked to the documentation

The [Vector Quantization Example](https://scikit-learn.org/stable/auto_examples/cluster/plot_face_compress.html) doesn't seem to demonstrate Vector Quantization.  

As written, the kmeans clustering approach used in the example converts a grey scale `uint8` `face` to an `int32` representation (`labels`). This increases the image memory use by 4x. 

```python
print(f'face dtype: {face.dtype}')
print(f'face bytes: {face.nbytes}')
print(f'labels dtype: {labels.dtype}')
print(f'labels bytes: {labels.nbytes}')
```
```
face dtype: uint8
face bytes: 786432
labels dtype: int32
labels bytes: 3145728
```

**Expected output**
Vector quantization output demonstrates a decrease in memory use.


**Additional details**
From [Wikipedia](https://en.wikipedia.org/wiki/Vector_quantization): "Vector quantization, also called "block quantization" or "pattern matching quantization" is often used in lossy data compression. It works by encoding values from a multidimensional vector space into a finite set of values from a discrete subspace of lower dimension. A lower-space vector requires less storage space, so the data is compressed."

I'm guessing kmeans outputs an `int32` by default.  The cluster `labels` are in the range 0,1,2,3,4. While this could be compressed to a 4 bit integer, `uint8` is as small as we can go with numpy so the example does not effectively illustrate the data compression. 

Perhaps the tutorial assumption is that the values contained in `labels` could be compressed through some other algorithm (e.g. outside of numpy). However, for someone unfamiliar with Vector Quantization, it may seem odd why someone would quantize a vector in a way that both loses information and increases memory use.

### Suggest a potential alternative/fix

1. add a comment to clarify that the quantized representation of the original face could be further compressed by another algorithm
2. replace the gray image with a color image and cast the kmeans output to a `uint8` to demonstrate the compression. Converting 3, 8 bit channels to one 8 bit channel would reduce nbytes by 67%.

COMMENT:
Hello! If no one is already working on this, can I take this up?

Labels: Documentation, good first issue
Vector Quantization Example increases rather than decreases memory use

### Describe the issue linked to the documentation

The [Vector Quantization Example](https://scikit-learn.org/stable/auto_examples/cluster/plot_face_compress.html) doesn't seem to demonstrate Vector Quantization.  

As written, the kmeans clustering approach used in the example converts a grey scale `uint8` `face` to an `int32` representation (`labels`). This increases the image memory use by 4x. 

```python
print(f'face dtype: {face.dtype}')
print(f'face bytes: {face.nbytes}')
print(f'labels dtype: {labels.dtype}')
print(f'labels bytes: {labels.nbytes}')
```
```
face dtype: uint8
face bytes: 786432
labels dtype: int32
labels bytes: 3145728
```

**Expected output**
Vector quantization output demonstrates a decrease in memory use.


**Additional details**
From [Wikipedia](https://en.wikipedia.org/wiki/Vector_quantization): "Vector quantization, also called "block quantization" or "pattern matching quantization" is often used in lossy data compression. It works by encoding values from a multidimensional vector space into a finite set of values from a discrete subspace of lower dimension. A lower-space vector requires less storage space, so the data is compressed."

I'm guessing kmeans outputs an `int32` by default.  The cluster `labels` are in the range 0,1,2,3,4. While this could be compressed to a 4 bit integer, `uint8` is as small as we can go with numpy so the example does not effectively illustrate the data compression. 

Perhaps the tutorial assumption is that the values contained in `labels` could be compressed through some other algorithm (e.g. outside of numpy). However, for someone unfamiliar with Vector Quantization, it may seem odd why someone would quantize a vector in a way that both loses information and increases memory use.

### Suggest a potential alternative/fix

1. add a comment to clarify that the quantized representation of the original face could be further compressed by another algorithm
2. replace the gray image with a color image and cast the kmeans output to a `uint8` to demonstrate the compression. Converting 3, 8 bit channels to one 8 bit channel would reduce nbytes by 67%.

COMMENT:
You can go ahead.

Labels: Documentation, good first issue
Vector Quantization Example increases rather than decreases memory use

### Describe the issue linked to the documentation

The [Vector Quantization Example](https://scikit-learn.org/stable/auto_examples/cluster/plot_face_compress.html) doesn't seem to demonstrate Vector Quantization.  

As written, the kmeans clustering approach used in the example converts a grey scale `uint8` `face` to an `int32` representation (`labels`). This increases the image memory use by 4x. 

```python
print(f'face dtype: {face.dtype}')
print(f'face bytes: {face.nbytes}')
print(f'labels dtype: {labels.dtype}')
print(f'labels bytes: {labels.nbytes}')
```
```
face dtype: uint8
face bytes: 786432
labels dtype: int32
labels bytes: 3145728
```

**Expected output**
Vector quantization output demonstrates a decrease in memory use.


**Additional details**
From [Wikipedia](https://en.wikipedia.org/wiki/Vector_quantization): "Vector quantization, also called "block quantization" or "pattern matching quantization" is often used in lossy data compression. It works by encoding values from a multidimensional vector space into a finite set of values from a discrete subspace of lower dimension. A lower-space vector requires less storage space, so the data is compressed."

I'm guessing kmeans outputs an `int32` by default.  The cluster `labels` are in the range 0,1,2,3,4. While this could be compressed to a 4 bit integer, `uint8` is as small as we can go with numpy so the example does not effectively illustrate the data compression. 

Perhaps the tutorial assumption is that the values contained in `labels` could be compressed through some other algorithm (e.g. outside of numpy). However, for someone unfamiliar with Vector Quantization, it may seem odd why someone would quantize a vector in a way that both loses information and increases memory use.

### Suggest a potential alternative/fix

1. add a comment to clarify that the quantized representation of the original face could be further compressed by another algorithm
2. replace the gray image with a color image and cast the kmeans output to a `uint8` to demonstrate the compression. Converting 3, 8 bit channels to one 8 bit channel would reduce nbytes by 67%.

COMMENT:
@MocktaiLEngineer Do let me know if you are working on this otherwise I would like to pick this up.

Labels: Documentation, good first issue
Vector Quantization Example increases rather than decreases memory use

### Describe the issue linked to the documentation

The [Vector Quantization Example](https://scikit-learn.org/stable/auto_examples/cluster/plot_face_compress.html) doesn't seem to demonstrate Vector Quantization.  

As written, the kmeans clustering approach used in the example converts a grey scale `uint8` `face` to an `int32` representation (`labels`). This increases the image memory use by 4x. 

```python
print(f'face dtype: {face.dtype}')
print(f'face bytes: {face.nbytes}')
print(f'labels dtype: {labels.dtype}')
print(f'labels bytes: {labels.nbytes}')
```
```
face dtype: uint8
face bytes: 786432
labels dtype: int32
labels bytes: 3145728
```

**Expected output**
Vector quantization output demonstrates a decrease in memory use.


**Additional details**
From [Wikipedia](https://en.wikipedia.org/wiki/Vector_quantization): "Vector quantization, also called "block quantization" or "pattern matching quantization" is often used in lossy data compression. It works by encoding values from a multidimensional vector space into a finite set of values from a discrete subspace of lower dimension. A lower-space vector requires less storage space, so the data is compressed."

I'm guessing kmeans outputs an `int32` by default.  The cluster `labels` are in the range 0,1,2,3,4. While this could be compressed to a 4 bit integer, `uint8` is as small as we can go with numpy so the example does not effectively illustrate the data compression. 

Perhaps the tutorial assumption is that the values contained in `labels` could be compressed through some other algorithm (e.g. outside of numpy). However, for someone unfamiliar with Vector Quantization, it may seem odd why someone would quantize a vector in a way that both loses information and increases memory use.

### Suggest a potential alternative/fix

1. add a comment to clarify that the quantized representation of the original face could be further compressed by another algorithm
2. replace the gray image with a color image and cast the kmeans output to a `uint8` to demonstrate the compression. Converting 3, 8 bit channels to one 8 bit channel would reduce nbytes by 67%.

COMMENT:
@MrinalTyagi Please feel free to take over.

Labels: Documentation, good first issue