Talip7/scikit-learn-issues-multilabel huggingface dataset:
Frisch-Newton Interior Point Solver for Quantile Regression

### Describe the workflow you want to enable

Hi @ scikit-learn devs! 

Over at [pyfixest](https://github.com/py-econometrics/pyfixest), we have implemented a Frisch-Newton Interior Point solver to fit quantile regressions. The algorithm goes back to work from Koenker. In practice, we have followed Koenker and Ng ["A Frisch-Newton Algorithm for Sparse Quantile Regression". ](https://link.springer.com/article/10.1007/s10255-005-0231-1)

The code is licensed under MIT and available [here](https://github.com/py-econometrics/pyfixest/blob/master/pyfixest/estimation/quantreg/frisch_newton_ip.py#L70). 

We (@apoorvalal) have collected some benchmarks [here](https://gist.github.com/apoorvalal/3e18eea79c6e9e8e8ee380e0fc0bab1f) - the FN solver seems to outperform the scikit default solver by an order of a magnitude.  

<img width="1362" height="534" alt="Image" src="https://github.com/user-attachments/assets/f25eb315-60d8-464f-80f8-bf1c6aedce3b" />

Would you be interested in a PR that adds the FN solver as a new estimation method to the quantile regression class? 

We've also implemented algorithms from [Chernozhukov et al ](https://arxiv.org/abs/1909.05782)that can drastically speed up estimation of the entire **quantile regression process**. 

All the best, Alex

### Describe your proposed solution

I open a PR and add a new solver "fn" to `QuantileRegressor`.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

Related to https://github.com/scikit-learn/scikit-learn/issues/20132

COMMENT:
@s3alfisc Thanks, that looks promising.

Would it be possible to include either an L1 or L2 penalty to the Frisch-Newton solver? In the original PR #9978 that introduced `QuantileRegressor`, I derived the linear programming formulation with L1 penalty myself, see https://github.com/scikit-learn/scikit-learn/pull/9978#issuecomment-744059556.

Labels: New Feature, Needs Decision - Include Feature
Frisch-Newton Interior Point Solver for Quantile Regression

### Describe the workflow you want to enable

Hi @ scikit-learn devs! 

Over at [pyfixest](https://github.com/py-econometrics/pyfixest), we have implemented a Frisch-Newton Interior Point solver to fit quantile regressions. The algorithm goes back to work from Koenker. In practice, we have followed Koenker and Ng ["A Frisch-Newton Algorithm for Sparse Quantile Regression". ](https://link.springer.com/article/10.1007/s10255-005-0231-1)

The code is licensed under MIT and available [here](https://github.com/py-econometrics/pyfixest/blob/master/pyfixest/estimation/quantreg/frisch_newton_ip.py#L70). 

We (@apoorvalal) have collected some benchmarks [here](https://gist.github.com/apoorvalal/3e18eea79c6e9e8e8ee380e0fc0bab1f) - the FN solver seems to outperform the scikit default solver by an order of a magnitude.  

<img width="1362" height="534" alt="Image" src="https://github.com/user-attachments/assets/f25eb315-60d8-464f-80f8-bf1c6aedce3b" />

Would you be interested in a PR that adds the FN solver as a new estimation method to the quantile regression class? 

We've also implemented algorithms from [Chernozhukov et al ](https://arxiv.org/abs/1909.05782)that can drastically speed up estimation of the entire **quantile regression process**. 

All the best, Alex

### Describe your proposed solution

I open a PR and add a new solver "fn" to `QuantileRegressor`.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

Related to https://github.com/scikit-learn/scikit-learn/issues/20132

COMMENT:
What I like about the current `QuantileRegressor` is that it delegates the computation to [`scipy.optimize.linprog`](https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.linprog.html). I have some reservations of implementing our own linear programming solver. Note that this is a much harder problem than least squares or GLMs.

Note that it seems that HIGHS is developing a new interior point method, see https://workshop25.highs.dev/schedule.

@jajhall friendly ping as you might have some insights about the HIGHS development and also why the Frisch-Newton solver (interior point barrier method for quantile regression) proposed in this issue seems to be favorable compared to "highs" (for LP problems stemming from quantile regression).

Labels: New Feature, Needs Decision - Include Feature
Frisch-Newton Interior Point Solver for Quantile Regression

### Describe the workflow you want to enable

Hi @ scikit-learn devs! 

Over at [pyfixest](https://github.com/py-econometrics/pyfixest), we have implemented a Frisch-Newton Interior Point solver to fit quantile regressions. The algorithm goes back to work from Koenker. In practice, we have followed Koenker and Ng ["A Frisch-Newton Algorithm for Sparse Quantile Regression". ](https://link.springer.com/article/10.1007/s10255-005-0231-1)

The code is licensed under MIT and available [here](https://github.com/py-econometrics/pyfixest/blob/master/pyfixest/estimation/quantreg/frisch_newton_ip.py#L70). 

We (@apoorvalal) have collected some benchmarks [here](https://gist.github.com/apoorvalal/3e18eea79c6e9e8e8ee380e0fc0bab1f) - the FN solver seems to outperform the scikit default solver by an order of a magnitude.  

<img width="1362" height="534" alt="Image" src="https://github.com/user-attachments/assets/f25eb315-60d8-464f-80f8-bf1c6aedce3b" />

Would you be interested in a PR that adds the FN solver as a new estimation method to the quantile regression class? 

We've also implemented algorithms from [Chernozhukov et al ](https://arxiv.org/abs/1909.05782)that can drastically speed up estimation of the entire **quantile regression process**. 

All the best, Alex

### Describe your proposed solution

I open a PR and add a new solver "fn" to `QuantileRegressor`.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

Related to https://github.com/scikit-learn/scikit-learn/issues/20132

COMMENT:
@s3alfisc Could you pass CSC sparse arrays directly to `QuantileRegressor(solver="highs").fit(X_csc)` such that the data conversion from numpy ndarray to scipy CSC is not accounted for? I have a feeling...

Labels: New Feature, Needs Decision - Include Feature
Frisch-Newton Interior Point Solver for Quantile Regression

### Describe the workflow you want to enable

Hi @ scikit-learn devs! 

Over at [pyfixest](https://github.com/py-econometrics/pyfixest), we have implemented a Frisch-Newton Interior Point solver to fit quantile regressions. The algorithm goes back to work from Koenker. In practice, we have followed Koenker and Ng ["A Frisch-Newton Algorithm for Sparse Quantile Regression". ](https://link.springer.com/article/10.1007/s10255-005-0231-1)

The code is licensed under MIT and available [here](https://github.com/py-econometrics/pyfixest/blob/master/pyfixest/estimation/quantreg/frisch_newton_ip.py#L70). 

We (@apoorvalal) have collected some benchmarks [here](https://gist.github.com/apoorvalal/3e18eea79c6e9e8e8ee380e0fc0bab1f) - the FN solver seems to outperform the scikit default solver by an order of a magnitude.  

<img width="1362" height="534" alt="Image" src="https://github.com/user-attachments/assets/f25eb315-60d8-464f-80f8-bf1c6aedce3b" />

Would you be interested in a PR that adds the FN solver as a new estimation method to the quantile regression class? 

We've also implemented algorithms from [Chernozhukov et al ](https://arxiv.org/abs/1909.05782)that can drastically speed up estimation of the entire **quantile regression process**. 

All the best, Alex

### Describe your proposed solution

I open a PR and add a new solver "fn" to `QuantileRegressor`.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

Related to https://github.com/scikit-learn/scikit-learn/issues/20132

COMMENT:
I can't say why a Frisch-Newton solver might be better than our IPM solver, but I asked Jacek Gondzio today, and we'll talk properly about the issue at a later date. 

Our new IPM solver will be available in the next release of HiGHS, but not via Python for a while. Since it's a "traditional" IPM solver (ie via direct factorization of the augmented or Newton system) it will (soon) extend readily to QP. This may make it valuable when you have an L2 penalty.

Labels: New Feature, Needs Decision - Include Feature
Frisch-Newton Interior Point Solver for Quantile Regression

### Describe the workflow you want to enable

Hi @ scikit-learn devs! 

Over at [pyfixest](https://github.com/py-econometrics/pyfixest), we have implemented a Frisch-Newton Interior Point solver to fit quantile regressions. The algorithm goes back to work from Koenker. In practice, we have followed Koenker and Ng ["A Frisch-Newton Algorithm for Sparse Quantile Regression". ](https://link.springer.com/article/10.1007/s10255-005-0231-1)

The code is licensed under MIT and available [here](https://github.com/py-econometrics/pyfixest/blob/master/pyfixest/estimation/quantreg/frisch_newton_ip.py#L70). 

We (@apoorvalal) have collected some benchmarks [here](https://gist.github.com/apoorvalal/3e18eea79c6e9e8e8ee380e0fc0bab1f) - the FN solver seems to outperform the scikit default solver by an order of a magnitude.  

<img width="1362" height="534" alt="Image" src="https://github.com/user-attachments/assets/f25eb315-60d8-464f-80f8-bf1c6aedce3b" />

Would you be interested in a PR that adds the FN solver as a new estimation method to the quantile regression class? 

We've also implemented algorithms from [Chernozhukov et al ](https://arxiv.org/abs/1909.05782)that can drastically speed up estimation of the entire **quantile regression process**. 

All the best, Alex

### Describe your proposed solution

I open a PR and add a new solver "fn" to `QuantileRegressor`.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

Related to https://github.com/scikit-learn/scikit-learn/issues/20132

COMMENT:
> one alternative could also be that I simply write a small utils package that users can switch to once they realize the alpha = 0 solver in scikit does not scale for their problem?

I think that would be welcomed to unlock the value of your contrib for advanced scikit-learn users while we wait for an assessment for HiGHS new method to check if it can rival Frish-Newton in the unpenalized case and whether any of those methods can be generalized to support l2 or elasticnet penalties.

Labels: New Feature, Needs Decision - Include Feature
EmpiricalCovariance user guide assume_centered tip incorrect

### Describe the issue linked to the documentation

The [user guide documentation](https://scikit-learn.org/stable/modules/covariance.html#empirical-covariance) for EmpiricalCovariance currently states:

> More precisely, if `assume_centered=False`, then the test set is supposed to have the same mean vector as the training set. If not, both should be centered by the user, and `assume_centered=True` should be used.

It doesn't make sense, however, that `assume_centered=False` would require data to be centered.  Likewise, it would seem that the user would need to center the data OR use `assume_centered=True` -- not both.

Additionally, it doesn't seem like there are separate training and testing data for this.

### Suggest a potential alternative/fix

I think it should read:

>More precisely, if `assume_centered=True`, then the data set's mean vector should be zero. If not, the data should be centered by the user, or `assume_centered=False` should be used.

COMMENT:
Hi! I'm new to open source and would love to contribute. This issue looks like a good place to start — I'd like to take this on and will be submitting a PR soon.

Labels: Documentation
EmpiricalCovariance user guide assume_centered tip incorrect

### Describe the issue linked to the documentation

The [user guide documentation](https://scikit-learn.org/stable/modules/covariance.html#empirical-covariance) for EmpiricalCovariance currently states:

> More precisely, if `assume_centered=False`, then the test set is supposed to have the same mean vector as the training set. If not, both should be centered by the user, and `assume_centered=True` should be used.

It doesn't make sense, however, that `assume_centered=False` would require data to be centered.  Likewise, it would seem that the user would need to center the data OR use `assume_centered=True` -- not both.

Additionally, it doesn't seem like there are separate training and testing data for this.

### Suggest a potential alternative/fix

I think it should read:

>More precisely, if `assume_centered=True`, then the data set's mean vector should be zero. If not, the data should be centered by the user, or `assume_centered=False` should be used.

COMMENT:
I agree that this section is confusing and could be improved.

It is possible that 

> the test set is supposed to have the same mean vector as the training set

 was talking about the `score` method:

https://github.com/scikit-learn/scikit-learn/blob/bd5d5f6b6f94a8e99c69fba5a14c3de0d39c949e/sklearn/covariance/_empirical_covariance.py#L285-L286

(note `self.location_ = X.mean(0)`)

What about something like:

> More precisely, if `assume_centered=True`, then all features in `X` should have a mean of zero. If not, the data should be centered by the user, or `assume_centered=False` should be used.

We can then also mention the assumption made about the test and train set when using the `score` method.

Labels: Documentation
Pipelines are permitted to have no steps and are displayed as fitted

### Describe the bug

Pipeline without defined steps is displayed in HTML as fitted.  




### Steps/Code to Reproduce


```
from sklearn.pipeline import Pipeline

pipe = Pipeline([])

pipe
```

### Expected Results

Maybe empty list should not be accepted. And it should rise a ValueError with a message asking to add steps.




### Actual Results

Using vscode jupyter extension:

<img width="401" alt="Image" src="https://github.com/user-attachments/assets/f0ad0033-1b86-4a91-a30b-969a5d2ea22e" />

Note: Accepting an empty list is one issue, and showing that it is fitted is another.
The former occurs when a `Pipeline` is initialized. The latter, I believe, is a design flaw in `sklearn/utils/_repr_html/estimator.py.`

### Versions

```shell
System:
    python: 3.12.2 (v3.12.2:6abddd9f6a, Feb  6 2024, 17:02:06) [Clang 13.0.0 (clang-1300.0.29.30)]
executable: /Users/dealeon/projects/scikit-learn/sklearn-env/bin/python
   machine: macOS-15.5-arm64-arm-64bit

Python dependencies:
      sklearn: 1.8.dev0
          pip: 25.1
   setuptools: 75.8.0
        numpy: 2.1.1
        scipy: 1.14.1
       Cython: 3.0.11
       pandas: 2.2.3
   matplotlib: 3.9.2
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libomp
       filepath: /opt/homebrew/Cellar/libomp/19.1.7/lib/libomp.dylib
        version: None
```

COMMENT:
I think the root cause is:

In the `Pipeline` class, the `__sklearn_is_fitted__` method determines whether the pipeline is fitted by checking the last non-`"passthrough"` step. Here's the relevant part:

```python
def __sklearn_is_fitted__(self):
    last_step = None
    for _, estimator in reversed(self.steps):
        if estimator != "passthrough":
            last_step = estimator
            break

    if last_step is None:
        # All steps are 'passthrough', so the pipeline is considered fitted
        return True
```

However, if `self.steps` is an **empty list**, `last_step` remains `None`, and the method returns `True` - treating it as a passthrough-only pipeline. This is incorrect because the pipeline has no steps at all, not just passthrough steps.

So the observed behavior:
```python
pipe = Pipeline([])
pipe  # shows as fitted
```
…is due to this incorrect assumption.

A suggested fix is to add a check in `Pipeline.__init__` to raise a `ValueError` if `steps` is empty. Alternatively (or additionally), update `__sklearn_is_fitted__` to raise an error or return `False` when `self.steps` is empty.

Let me know if you'd like me to submit a PR for this! @DeaMariaLeon @StefanieSenger

Labels: Bug
Pipelines are permitted to have no steps and are displayed as fitted

### Describe the bug

Pipeline without defined steps is displayed in HTML as fitted.  




### Steps/Code to Reproduce


```
from sklearn.pipeline import Pipeline

pipe = Pipeline([])

pipe
```

### Expected Results

Maybe empty list should not be accepted. And it should rise a ValueError with a message asking to add steps.




### Actual Results

Using vscode jupyter extension:

<img width="401" alt="Image" src="https://github.com/user-attachments/assets/f0ad0033-1b86-4a91-a30b-969a5d2ea22e" />

Note: Accepting an empty list is one issue, and showing that it is fitted is another.
The former occurs when a `Pipeline` is initialized. The latter, I believe, is a design flaw in `sklearn/utils/_repr_html/estimator.py.`

### Versions

```shell
System:
    python: 3.12.2 (v3.12.2:6abddd9f6a, Feb  6 2024, 17:02:06) [Clang 13.0.0 (clang-1300.0.29.30)]
executable: /Users/dealeon/projects/scikit-learn/sklearn-env/bin/python
   machine: macOS-15.5-arm64-arm-64bit

Python dependencies:
      sklearn: 1.8.dev0
          pip: 25.1
   setuptools: 75.8.0
        numpy: 2.1.1
        scipy: 1.14.1
       Cython: 3.0.11
       pandas: 2.2.3
   matplotlib: 3.9.2
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libomp
       filepath: /opt/homebrew/Cellar/libomp/19.1.7/lib/libomp.dylib
        version: None
```

COMMENT:
> Let me know if you'd like me to submit a PR for this! @DeaMariaLeon @StefanieSenger 

Thanks for looking at this, @SwathiR1999! I won't have the bandwidth to guide a new contributor on this issue. Reviewer time is in high demand and very tight overall, unfortunately. If you open a thoughtful PR that meets quality standards and passes CI, it will be much easier for someone to review it. 

But @DeaMariaLeon was probably already on it and wanted to get an assessment from the team. I agree this is a problem and thanks for taking this on, @DeaMariaLeon, hence my thumbs-up from earlier.

This issue still needs triage, though.

Labels: Bug
Pipelines are permitted to have no steps and are displayed as fitted

### Describe the bug

Pipeline without defined steps is displayed in HTML as fitted.  




### Steps/Code to Reproduce


```
from sklearn.pipeline import Pipeline

pipe = Pipeline([])

pipe
```

### Expected Results

Maybe empty list should not be accepted. And it should rise a ValueError with a message asking to add steps.




### Actual Results

Using vscode jupyter extension:

<img width="401" alt="Image" src="https://github.com/user-attachments/assets/f0ad0033-1b86-4a91-a30b-969a5d2ea22e" />

Note: Accepting an empty list is one issue, and showing that it is fitted is another.
The former occurs when a `Pipeline` is initialized. The latter, I believe, is a design flaw in `sklearn/utils/_repr_html/estimator.py.`

### Versions

```shell
System:
    python: 3.12.2 (v3.12.2:6abddd9f6a, Feb  6 2024, 17:02:06) [Clang 13.0.0 (clang-1300.0.29.30)]
executable: /Users/dealeon/projects/scikit-learn/sklearn-env/bin/python
   machine: macOS-15.5-arm64-arm-64bit

Python dependencies:
      sklearn: 1.8.dev0
          pip: 25.1
   setuptools: 75.8.0
        numpy: 2.1.1
        scipy: 1.14.1
       Cython: 3.0.11
       pandas: 2.2.3
   matplotlib: 3.9.2
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libomp
       filepath: /opt/homebrew/Cellar/libomp/19.1.7/lib/libomp.dylib
        version: None
```

COMMENT:
Hi @SwathiR1999, like Stefanie said, I was already working on this issue. Could you please work on another one? 

BTW I would avoid taking an issue with a label "needs triage".

Labels: Bug
Pipelines are permitted to have no steps and are displayed as fitted

### Describe the bug

Pipeline without defined steps is displayed in HTML as fitted.  




### Steps/Code to Reproduce


```
from sklearn.pipeline import Pipeline

pipe = Pipeline([])

pipe
```

### Expected Results

Maybe empty list should not be accepted. And it should rise a ValueError with a message asking to add steps.




### Actual Results

Using vscode jupyter extension:

<img width="401" alt="Image" src="https://github.com/user-attachments/assets/f0ad0033-1b86-4a91-a30b-969a5d2ea22e" />

Note: Accepting an empty list is one issue, and showing that it is fitted is another.
The former occurs when a `Pipeline` is initialized. The latter, I believe, is a design flaw in `sklearn/utils/_repr_html/estimator.py.`

### Versions

```shell
System:
    python: 3.12.2 (v3.12.2:6abddd9f6a, Feb  6 2024, 17:02:06) [Clang 13.0.0 (clang-1300.0.29.30)]
executable: /Users/dealeon/projects/scikit-learn/sklearn-env/bin/python
   machine: macOS-15.5-arm64-arm-64bit

Python dependencies:
      sklearn: 1.8.dev0
          pip: 25.1
   setuptools: 75.8.0
        numpy: 2.1.1
        scipy: 1.14.1
       Cython: 3.0.11
       pandas: 2.2.3
   matplotlib: 3.9.2
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libomp
       filepath: /opt/homebrew/Cellar/libomp/19.1.7/lib/libomp.dylib
        version: None
```

COMMENT:
Seems to me like the Null Object pattern is generally useful and idiomatic Python, and it would be far better to improve the class, which needs to be fixed anyway.

Labels: Bug
Pipelines are permitted to have no steps and are displayed as fitted

### Describe the bug

Pipeline without defined steps is displayed in HTML as fitted.  




### Steps/Code to Reproduce


```
from sklearn.pipeline import Pipeline

pipe = Pipeline([])

pipe
```

### Expected Results

Maybe empty list should not be accepted. And it should rise a ValueError with a message asking to add steps.




### Actual Results

Using vscode jupyter extension:

<img width="401" alt="Image" src="https://github.com/user-attachments/assets/f0ad0033-1b86-4a91-a30b-969a5d2ea22e" />

Note: Accepting an empty list is one issue, and showing that it is fitted is another.
The former occurs when a `Pipeline` is initialized. The latter, I believe, is a design flaw in `sklearn/utils/_repr_html/estimator.py.`

### Versions

```shell
System:
    python: 3.12.2 (v3.12.2:6abddd9f6a, Feb  6 2024, 17:02:06) [Clang 13.0.0 (clang-1300.0.29.30)]
executable: /Users/dealeon/projects/scikit-learn/sklearn-env/bin/python
   machine: macOS-15.5-arm64-arm-64bit

Python dependencies:
      sklearn: 1.8.dev0
          pip: 25.1
   setuptools: 75.8.0
        numpy: 2.1.1
        scipy: 1.14.1
       Cython: 3.0.11
       pandas: 2.2.3
   matplotlib: 3.9.2
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libomp
       filepath: /opt/homebrew/Cellar/libomp/19.1.7/lib/libomp.dylib
        version: None
```

COMMENT:
/take

Labels: Bug