Talip7/scikit-learn-issues-multilabel huggingface dataset:
Docs say parameter sample_weight of LinearRegression.fit must be array but number is also valid

### Describe the issue linked to the documentation

The documentation page for the `fit` method of the `LinearRegression` class mentions that the `sample_weight` parameter must be of type `array_like` or `None` ([docs](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html#sklearn.linear_model.LinearRegression.fit)). However this is not entirely true since we can also pass `float` or `int` for this parameter. Floats or ints get transformed into an array of that same value repeating n times. Code snippet here:
https://github.com/scikit-learn/scikit-learn/blob/f59c50301bc725fe3da75ac3b5e8614ff9a26f98/sklearn/utils/validation.py#L2000-L2003
This makes it that a sample weight of `float` or `int` is essentially equal to `None` since they all have the same relative weight (not sure if I'm overseeing something, but could not think of any case where a float or int for `sample_weight` could be meaningful).

### Suggest a potential alternative/fix

I see two possible fixes:
- Change the documentation to address the fact that numbers are valid values for `sample_weight` however they have no effect since there is no difference in the relative weight of the samples.
- Change the code so that an error or warning is raised if the `sample_weight` parameter is a `float` or an `int`.

COMMENT:
> Some tests just check that passing a float works. Some check consistency, i.e. passing a float has the same effect as passing None.

Is that something that should be done, i.e., use the `_check_sample_wieght` function each time the `fit` method of an estimator accepts the `sample_weight` parameter? Suppose that could help ensure consistency of the API

I think if this was being done from scratch my inclination would be to only accept either the `array` or `None`. Seems a tiny bit more coherent with the underlying concept. But since `float` is currently accepted, also don't see much harm in sticking with that, since nothing is inherently wrong about the current logic.

Perhaps the only downside may be undetected bugs that could occur. Not sure if any of the maintainers as some intuition for how likely you believe a bug of this kind would be to occur in someone's code? Doesn't seem too likely to me, but I'm quite new to open-source

Labels: Documentation, Needs Decision
Update index handling in `PandasAdapter`

### Describe the workflow you want to enable

As noted in #27037, handling the index of an input container can be hairy. The solution implemented in #27044 works, but it excludes `pandas.Series` input types. I'd like to modify the logic in the [:method:`PandasAdapter.create_container`](https://github.com/scikit-learn/scikit-learn/blob/f59c50301bc725fe3da75ac3b5e8614ff9a26f98/sklearn/utils/_set_output.py#L124) so that it checks if the `X_original` is a `pandas.DataFrame` **_or_** `pandas.Series`. This would allow transformers that accept 1-dimensional inputs and output 2-dimensional dataframes to persist their indices.

### Describe your proposed solution

I'd like to change [line 124](https://github.com/scikit-learn/scikit-learn/blob/f59c50301bc725fe3da75ac3b5e8614ff9a26f98/sklearn/utils/_set_output.py#L124) from this:

```python
            elif isinstance(X_original, pd.DataFrame):
```

To this:

```python
            elif isinstance(X_original, (pd.DataFrame, pd.Series)):
```

### Describe alternatives you've considered, if relevant

User sets the index on their own:

```python
some_series = pd.Series(...)
trf = SomeTransformer().set_output(transform="pandas")
out_frame = trf.fit_transform(some_series).set_index(some_series.index)
```

### Additional context

I recognize _most_ transformers in `scikit-learn` expect 2-dimensional inputs. But some packages that depend on `scikit-learn` (like [`mlxtend`](https://rasbt.github.io/mlxtend/)) have transformers that transform 1-dimensional input into 2-dimensional output. I believe this would greatly benefit them. See the [newly updated `TransactionEncoder`](https://github.com/rasbt/mlxtend/pull/1087) for an example.

I'm willing to submit a PR if this is an acceptable enhancement.

COMMENT:
This is currently outside our API scope and I don't think that we want to make some code that we are not using internally. I think this is a design purpose that scikit-learn define `X` to be a 2D container because it remove some ambiguity regarding features/samples.

We might be revisit this if we come to think of transformer that make independent column-wise transform.

So I'm -1 for the inclusion right now.

Labels: New Feature
Update index handling in `PandasAdapter`

### Describe the workflow you want to enable

As noted in #27037, handling the index of an input container can be hairy. The solution implemented in #27044 works, but it excludes `pandas.Series` input types. I'd like to modify the logic in the [:method:`PandasAdapter.create_container`](https://github.com/scikit-learn/scikit-learn/blob/f59c50301bc725fe3da75ac3b5e8614ff9a26f98/sklearn/utils/_set_output.py#L124) so that it checks if the `X_original` is a `pandas.DataFrame` **_or_** `pandas.Series`. This would allow transformers that accept 1-dimensional inputs and output 2-dimensional dataframes to persist their indices.

### Describe your proposed solution

I'd like to change [line 124](https://github.com/scikit-learn/scikit-learn/blob/f59c50301bc725fe3da75ac3b5e8614ff9a26f98/sklearn/utils/_set_output.py#L124) from this:

```python
            elif isinstance(X_original, pd.DataFrame):
```

To this:

```python
            elif isinstance(X_original, (pd.DataFrame, pd.Series)):
```

### Describe alternatives you've considered, if relevant

User sets the index on their own:

```python
some_series = pd.Series(...)
trf = SomeTransformer().set_output(transform="pandas")
out_frame = trf.fit_transform(some_series).set_index(some_series.index)
```

### Additional context

I recognize _most_ transformers in `scikit-learn` expect 2-dimensional inputs. But some packages that depend on `scikit-learn` (like [`mlxtend`](https://rasbt.github.io/mlxtend/)) have transformers that transform 1-dimensional input into 2-dimensional output. I believe this would greatly benefit them. See the [newly updated `TransactionEncoder`](https://github.com/rasbt/mlxtend/pull/1087) for an example.

I'm willing to submit a PR if this is an acceptable enhancement.

COMMENT:
> This is currently outside our API scope and I don't think that we want to make some code that we are not using internally. I think this is a design purpose that scikit-learn define `X` to be a 2D container because it remove some ambiguity regarding features/samples.
> 
> We might be revisit this if we come to think of transformer that make independent column-wise transform.
> 
> So I'm -1 for the inclusion right now.

Makes sense. Perhaps this would be useful for the [`CountVectorizer`](https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.CountVectorizer.html), [`TfidfVectorizer`](https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html), and [`TfidfTransformer`](https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfTransformer.html) as they expect a 1D container? Of course, they output sparse matrices by default, but maybe an option in the future.

Labels: New Feature
Update index handling in `PandasAdapter`

### Describe the workflow you want to enable

As noted in #27037, handling the index of an input container can be hairy. The solution implemented in #27044 works, but it excludes `pandas.Series` input types. I'd like to modify the logic in the [:method:`PandasAdapter.create_container`](https://github.com/scikit-learn/scikit-learn/blob/f59c50301bc725fe3da75ac3b5e8614ff9a26f98/sklearn/utils/_set_output.py#L124) so that it checks if the `X_original` is a `pandas.DataFrame` **_or_** `pandas.Series`. This would allow transformers that accept 1-dimensional inputs and output 2-dimensional dataframes to persist their indices.

### Describe your proposed solution

I'd like to change [line 124](https://github.com/scikit-learn/scikit-learn/blob/f59c50301bc725fe3da75ac3b5e8614ff9a26f98/sklearn/utils/_set_output.py#L124) from this:

```python
            elif isinstance(X_original, pd.DataFrame):
```

To this:

```python
            elif isinstance(X_original, (pd.DataFrame, pd.Series)):
```

### Describe alternatives you've considered, if relevant

User sets the index on their own:

```python
some_series = pd.Series(...)
trf = SomeTransformer().set_output(transform="pandas")
out_frame = trf.fit_transform(some_series).set_index(some_series.index)
```

### Additional context

I recognize _most_ transformers in `scikit-learn` expect 2-dimensional inputs. But some packages that depend on `scikit-learn` (like [`mlxtend`](https://rasbt.github.io/mlxtend/)) have transformers that transform 1-dimensional input into 2-dimensional output. I believe this would greatly benefit them. See the [newly updated `TransactionEncoder`](https://github.com/rasbt/mlxtend/pull/1087) for an example.

I'm willing to submit a PR if this is an acceptable enhancement.

COMMENT:
> they expect a 1D container?

As input but not as output. Here, I think that this is more on the output side that you want to act.

Labels: New Feature
Update index handling in `PandasAdapter`

### Describe the workflow you want to enable

As noted in #27037, handling the index of an input container can be hairy. The solution implemented in #27044 works, but it excludes `pandas.Series` input types. I'd like to modify the logic in the [:method:`PandasAdapter.create_container`](https://github.com/scikit-learn/scikit-learn/blob/f59c50301bc725fe3da75ac3b5e8614ff9a26f98/sklearn/utils/_set_output.py#L124) so that it checks if the `X_original` is a `pandas.DataFrame` **_or_** `pandas.Series`. This would allow transformers that accept 1-dimensional inputs and output 2-dimensional dataframes to persist their indices.

### Describe your proposed solution

I'd like to change [line 124](https://github.com/scikit-learn/scikit-learn/blob/f59c50301bc725fe3da75ac3b5e8614ff9a26f98/sklearn/utils/_set_output.py#L124) from this:

```python
            elif isinstance(X_original, pd.DataFrame):
```

To this:

```python
            elif isinstance(X_original, (pd.DataFrame, pd.Series)):
```

### Describe alternatives you've considered, if relevant

User sets the index on their own:

```python
some_series = pd.Series(...)
trf = SomeTransformer().set_output(transform="pandas")
out_frame = trf.fit_transform(some_series).set_index(some_series.index)
```

### Additional context

I recognize _most_ transformers in `scikit-learn` expect 2-dimensional inputs. But some packages that depend on `scikit-learn` (like [`mlxtend`](https://rasbt.github.io/mlxtend/)) have transformers that transform 1-dimensional input into 2-dimensional output. I believe this would greatly benefit them. See the [newly updated `TransactionEncoder`](https://github.com/rasbt/mlxtend/pull/1087) for an example.

I'm willing to submit a PR if this is an acceptable enhancement.

COMMENT:
> > they expect a 1D container?
> 
> As input but not as output. Here, I think that this is more on the output side that you want to act.

Correct—those transformers take 1D input and then output a 2D object. I figure if they had access to the `set_output` method and the user chose to return a dense object rather than sparse, we'd want to persist the index.

Labels: New Feature
Update index handling in `PandasAdapter`

### Describe the workflow you want to enable

As noted in #27037, handling the index of an input container can be hairy. The solution implemented in #27044 works, but it excludes `pandas.Series` input types. I'd like to modify the logic in the [:method:`PandasAdapter.create_container`](https://github.com/scikit-learn/scikit-learn/blob/f59c50301bc725fe3da75ac3b5e8614ff9a26f98/sklearn/utils/_set_output.py#L124) so that it checks if the `X_original` is a `pandas.DataFrame` **_or_** `pandas.Series`. This would allow transformers that accept 1-dimensional inputs and output 2-dimensional dataframes to persist their indices.

### Describe your proposed solution

I'd like to change [line 124](https://github.com/scikit-learn/scikit-learn/blob/f59c50301bc725fe3da75ac3b5e8614ff9a26f98/sklearn/utils/_set_output.py#L124) from this:

```python
            elif isinstance(X_original, pd.DataFrame):
```

To this:

```python
            elif isinstance(X_original, (pd.DataFrame, pd.Series)):
```

### Describe alternatives you've considered, if relevant

User sets the index on their own:

```python
some_series = pd.Series(...)
trf = SomeTransformer().set_output(transform="pandas")
out_frame = trf.fit_transform(some_series).set_index(some_series.index)
```

### Additional context

I recognize _most_ transformers in `scikit-learn` expect 2-dimensional inputs. But some packages that depend on `scikit-learn` (like [`mlxtend`](https://rasbt.github.io/mlxtend/)) have transformers that transform 1-dimensional input into 2-dimensional output. I believe this would greatly benefit them. See the [newly updated `TransactionEncoder`](https://github.com/rasbt/mlxtend/pull/1087) for an example.

I'm willing to submit a PR if this is an acceptable enhancement.

COMMENT:
I'm therefore closing since we will not act on it for the moment.

Labels: New Feature
⚠️ CI failed on Linux_nogil.pylatest_pip_nogil ⚠️

**CI failed on [Linux_nogil.pylatest_pip_nogil](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=65430&view=logs&j=67fbb25f-e417-50be-be55-3b1e9637fce5)** (Mar 31, 2024)
Unable to find junit file. Please see link for details.

COMMENT:
## CI is no longer failing! ✅

[Successful run](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=65439&view=logs&j=67fbb25f-e417-50be-be55-3b1e9637fce5) on Apr 01, 2024

Labels: Needs Triage
Is there any way to see alphas/coefs/intercept associated with *all* scenarios tested within ElasticNetCV

### Describe the workflow you want to enable

I like that ElasticNetCV outputs the MSE path for CV folds/alphas but is there any way to similarly track associated model params (ie, coef/intercept) for each scenario and include them as part of output.

I get that it's easier to just output 'best' estimators/params but would be useful to add granularity to allow identifying a 'sweet spot', either via MSE curve or something else, which would make outputting all params additive.   



### Describe your proposed solution

As described, run existing scenarios as is but instead of holding only through evaluation of 'best' model, save all model params/outputs and return in an additional data object/structure.   

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Maybe you want to have a look at the path function: https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.enet_path.html#sklearn.linear_model.enet_path

Labels: New Feature
Is there any way to see alphas/coefs/intercept associated with *all* scenarios tested within ElasticNetCV

### Describe the workflow you want to enable

I like that ElasticNetCV outputs the MSE path for CV folds/alphas but is there any way to similarly track associated model params (ie, coef/intercept) for each scenario and include them as part of output.

I get that it's easier to just output 'best' estimators/params but would be useful to add granularity to allow identifying a 'sweet spot', either via MSE curve or something else, which would make outputting all params additive.   



### Describe your proposed solution

As described, run existing scenarios as is but instead of holding only through evaluation of 'best' model, save all model params/outputs and return in an additional data object/structure.   

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Just so everyone is aware, a colleague and I are currently looking into this feature and plan to work on an implementation. We'll keep the thread updated with any progress or questions as they arise.

Labels: New Feature
Is there any way to see alphas/coefs/intercept associated with *all* scenarios tested within ElasticNetCV

### Describe the workflow you want to enable

I like that ElasticNetCV outputs the MSE path for CV folds/alphas but is there any way to similarly track associated model params (ie, coef/intercept) for each scenario and include them as part of output.

I get that it's easier to just output 'best' estimators/params but would be useful to add granularity to allow identifying a 'sweet spot', either via MSE curve or something else, which would make outputting all params additive.   



### Describe your proposed solution

As described, run existing scenarios as is but instead of holding only through evaluation of 'best' model, save all model params/outputs and return in an additional data object/structure.   

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Hi! The patch is ready. 
Could someone please review it and let us know if everything looks good or if any changes are needed? 
Thanks!

Labels: New Feature
kNN classifier - `predict`/`predict_proba` inefficient?

### Describe the workflow you want to enable

In a lot of cases, when calling `classifier.predict()` we may want probabilities as well via `classifier.predict_proba()`.

To enable that use-case, it seems [the code](https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/neighbors/_classification.py#L240-L408) would recompute a lot of the same logic, and call `neigh_dist, neigh_ind = self.kneighbors(X)` twice, etc.

This seems like wasted computation, given that everything up to the line `weights = _get_weights(neigh_dist, self.weights)` could be the same in a typical case. I admittedly haven't ran a benchmark to see how much time would be saved if this computation weren't performed twice.

I wonder: Is there a way to get "predictions with probabilities" while avoiding duplicate recomputation?

### Describe your proposed solution

If there is not, would a `predict_with_proba` (or similar), or another way of preventing duplicate computation, be difficult to implement?

### Describe alternatives you've considered, if relevant

I suppose one could always just override these functions and write their own version of the classifier. Perhaps that's the advised approach for this scenario, I'm not sure.

### Additional context

_No response_

COMMENT:
If you want both hard predictions and estimated class membership probabilities you can always do:


```python
y_probs = classifier.predict_proba()
y_pred = y_probs.argmax(axis=1)
```

This should be pretty efficient, I don't really see the need to expand the scikit-learn public API in this case.

Labels: New Feature, Needs Triage
kNN classifier - `predict`/`predict_proba` inefficient?

### Describe the workflow you want to enable

In a lot of cases, when calling `classifier.predict()` we may want probabilities as well via `classifier.predict_proba()`.

To enable that use-case, it seems [the code](https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/neighbors/_classification.py#L240-L408) would recompute a lot of the same logic, and call `neigh_dist, neigh_ind = self.kneighbors(X)` twice, etc.

This seems like wasted computation, given that everything up to the line `weights = _get_weights(neigh_dist, self.weights)` could be the same in a typical case. I admittedly haven't ran a benchmark to see how much time would be saved if this computation weren't performed twice.

I wonder: Is there a way to get "predictions with probabilities" while avoiding duplicate recomputation?

### Describe your proposed solution

If there is not, would a `predict_with_proba` (or similar), or another way of preventing duplicate computation, be difficult to implement?

### Describe alternatives you've considered, if relevant

I suppose one could always just override these functions and write their own version of the classifier. Perhaps that's the advised approach for this scenario, I'm not sure.

### Additional context

_No response_

COMMENT:
I agree with @ogrisel, I don't think that we want to extend the API because it can be done in a one-liner by post-processing the probabilities.

Labels: New Feature, Needs Triage
Node Splitting Proxy Improvement

### Describe the issue linked to the documentation

While exploring the splitter pyx file in the library's tree folder, I discovered this [current proxy improvement](https://github.com/scikit-learn/scikit-learn/blob/1f46775f7d87538fe00b38f230426c8a7371b11e/sklearn/tree/_splitter.pyx#L476), which speeds up the process of finding the best split prior to calculating the actual impurity improvement of this best estimated split. I am wondering if there could be some explanation in the documentation for the surrogate proxy improvement. And if this was documented in a paper, perhaps? We were curious about it, and we apologise if we did not see anything in the document or elsewhere that discussed it. The documentation for the proxy improvement function does not appear to be "thorough".

It would be greatly appreciated.

```python
current_proxy_improvement = criterion.proxy_impurity_improvement()
```

Cheers,


### Suggest a potential alternative/fix

Either in the code itself or in the documentation, explain briefly that the process of finding the best split differs slightly from the original idea, but that the actual outcome is expected to be the same given that the best estimated split receives the original impurity improvement equation calculated on its split.

If interested, we could provide such a pull request, but we would appreciate clarification to see that we are in line with your intention and actual behaviour from what we understood.

Cheers.

COMMENT:
> I am wondering if there could be some explanation in the documentation for the surrogate proxy improvement. And if this was documented in a paper, perhaps? 

This is an implementation details where we remove usually mathematical operations (multiplication, division, addition, etc.) with constant terms. Once we find the split, we compute the true improvement.

I'm -1 with adding this details since it does not impact the end-user.

Labels: Documentation, Needs Triage
RFC New parameters for penalties in LogisticRegression

Based on the comment https://github.com/scikit-learn/scikit-learn/pull/28706#discussion_r1541184840:

Currently, `LogisticRegression` uses `C` as inverse penalization strength, `penalty` to select the type of penalty and `l1_ratio` to control the ration between l1 and l2 penalties.
I propose the following:
1. Add `alpha` (as in `Ridge`, `ElasticNet`, `PoissonRegressor` ...) instead of `C`.
  Fail if both are given at the same time.
2. Deprecate `C`.
3. Deprecate `penalty` which is redundant. `alpha` and `l1_ratio` are enough.

COMMENT:
ping @scikit-learn/core-devs for more visibility.

Labels: API, RFC, module:linear_model