Talip7/scikit-learn-issues-multilabel huggingface dataset:
⚠️ CI failed on Linux_Runs.pylatest_conda_forge_mkl (last failure: Jul 09, 2025) ⚠️

**CI failed on [Linux_Runs.pylatest_conda_forge_mkl](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=78075&view=logs&j=dde5042c-7464-5d47-9507-31bdd2ee0a3a)** (Jul 09, 2025)
- Test Collection Failure

COMMENT:
## CI is no longer failing! ✅

[Successful run](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=78097&view=logs&j=dde5042c-7464-5d47-9507-31bdd2ee0a3a) on Jul 10, 2025

Labels: Needs Triage
Frisch-Newton Interior Point Solver for Quantile Regression

### Describe the workflow you want to enable

Hi @ scikit-learn devs! 

Over at [pyfixest](https://github.com/py-econometrics/pyfixest), we have implemented a Frisch-Newton Interior Point solver to fit quantile regressions. The algorithm goes back to work from Koenker. In practice, we have followed Koenker and Ng ["A Frisch-Newton Algorithm for Sparse Quantile Regression". ](https://link.springer.com/article/10.1007/s10255-005-0231-1)

The code is licensed under MIT and available [here](https://github.com/py-econometrics/pyfixest/blob/master/pyfixest/estimation/quantreg/frisch_newton_ip.py#L70). 

We (@apoorvalal) have collected some benchmarks [here](https://gist.github.com/apoorvalal/3e18eea79c6e9e8e8ee380e0fc0bab1f) - the FN solver seems to outperform the scikit default solver by an order of a magnitude.  

<img width="1362" height="534" alt="Image" src="https://github.com/user-attachments/assets/f25eb315-60d8-464f-80f8-bf1c6aedce3b" />

Would you be interested in a PR that adds the FN solver as a new estimation method to the quantile regression class? 

We've also implemented algorithms from [Chernozhukov et al ](https://arxiv.org/abs/1909.05782)that can drastically speed up estimation of the entire **quantile regression process**. 

All the best, Alex

### Describe your proposed solution

I open a PR and add a new solver "fn" to `QuantileRegressor`.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

Related to https://github.com/scikit-learn/scikit-learn/issues/20132

COMMENT:
The benchmark results sound promising! A 3-order of magnitude speed-up is really impressive.

Have you checked that this solver converges to the same results as the one in scikit-learn? In particular, does it use the same convergence criterion?

If so, I would be interested in seeing a draft PR personally.

cc @lorentzenchr @glemaitre.

Labels: New Feature, Needs Decision - Include Feature
Frisch-Newton Interior Point Solver for Quantile Regression

### Describe the workflow you want to enable

Hi @ scikit-learn devs! 

Over at [pyfixest](https://github.com/py-econometrics/pyfixest), we have implemented a Frisch-Newton Interior Point solver to fit quantile regressions. The algorithm goes back to work from Koenker. In practice, we have followed Koenker and Ng ["A Frisch-Newton Algorithm for Sparse Quantile Regression". ](https://link.springer.com/article/10.1007/s10255-005-0231-1)

The code is licensed under MIT and available [here](https://github.com/py-econometrics/pyfixest/blob/master/pyfixest/estimation/quantreg/frisch_newton_ip.py#L70). 

We (@apoorvalal) have collected some benchmarks [here](https://gist.github.com/apoorvalal/3e18eea79c6e9e8e8ee380e0fc0bab1f) - the FN solver seems to outperform the scikit default solver by an order of a magnitude.  

<img width="1362" height="534" alt="Image" src="https://github.com/user-attachments/assets/f25eb315-60d8-464f-80f8-bf1c6aedce3b" />

Would you be interested in a PR that adds the FN solver as a new estimation method to the quantile regression class? 

We've also implemented algorithms from [Chernozhukov et al ](https://arxiv.org/abs/1909.05782)that can drastically speed up estimation of the entire **quantile regression process**. 

All the best, Alex

### Describe your proposed solution

I open a PR and add a new solver "fn" to `QuantileRegressor`.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

Related to https://github.com/scikit-learn/scikit-learn/issues/20132

COMMENT:
Also: have you considered contributing this solver upstream in SciPy instead? That might make more sense.

Labels: New Feature, Needs Decision - Include Feature
Frisch-Newton Interior Point Solver for Quantile Regression

### Describe the workflow you want to enable

Hi @ scikit-learn devs! 

Over at [pyfixest](https://github.com/py-econometrics/pyfixest), we have implemented a Frisch-Newton Interior Point solver to fit quantile regressions. The algorithm goes back to work from Koenker. In practice, we have followed Koenker and Ng ["A Frisch-Newton Algorithm for Sparse Quantile Regression". ](https://link.springer.com/article/10.1007/s10255-005-0231-1)

The code is licensed under MIT and available [here](https://github.com/py-econometrics/pyfixest/blob/master/pyfixest/estimation/quantreg/frisch_newton_ip.py#L70). 

We (@apoorvalal) have collected some benchmarks [here](https://gist.github.com/apoorvalal/3e18eea79c6e9e8e8ee380e0fc0bab1f) - the FN solver seems to outperform the scikit default solver by an order of a magnitude.  

<img width="1362" height="534" alt="Image" src="https://github.com/user-attachments/assets/f25eb315-60d8-464f-80f8-bf1c6aedce3b" />

Would you be interested in a PR that adds the FN solver as a new estimation method to the quantile regression class? 

We've also implemented algorithms from [Chernozhukov et al ](https://arxiv.org/abs/1909.05782)that can drastically speed up estimation of the entire **quantile regression process**. 

All the best, Alex

### Describe your proposed solution

I open a PR and add a new solver "fn" to `QuantileRegressor`.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

Related to https://github.com/scikit-learn/scikit-learn/issues/20132

COMMENT:
As @ogrisel mentioned, I would prefer to ask for addition upstream in scipy's `linprog` solver.

Then about the benchmark: Could you pass CSC sparse arrays/matrices to scikit-learn's `QuantileRegressor`, I guess a lot of time is just used for data conversion. Then, again as @ogrisel mentioned, could you also provide the value of the mean pinball/quantile loss that each solver achieves? For optimization problems, one often plots time vs subobtimality (achieved loss minus loss at minimum) or just vs the achieved loss.

Labels: New Feature, Needs Decision - Include Feature
Frisch-Newton Interior Point Solver for Quantile Regression

### Describe the workflow you want to enable

Hi @ scikit-learn devs! 

Over at [pyfixest](https://github.com/py-econometrics/pyfixest), we have implemented a Frisch-Newton Interior Point solver to fit quantile regressions. The algorithm goes back to work from Koenker. In practice, we have followed Koenker and Ng ["A Frisch-Newton Algorithm for Sparse Quantile Regression". ](https://link.springer.com/article/10.1007/s10255-005-0231-1)

The code is licensed under MIT and available [here](https://github.com/py-econometrics/pyfixest/blob/master/pyfixest/estimation/quantreg/frisch_newton_ip.py#L70). 

We (@apoorvalal) have collected some benchmarks [here](https://gist.github.com/apoorvalal/3e18eea79c6e9e8e8ee380e0fc0bab1f) - the FN solver seems to outperform the scikit default solver by an order of a magnitude.  

<img width="1362" height="534" alt="Image" src="https://github.com/user-attachments/assets/f25eb315-60d8-464f-80f8-bf1c6aedce3b" />

Would you be interested in a PR that adds the FN solver as a new estimation method to the quantile regression class? 

We've also implemented algorithms from [Chernozhukov et al ](https://arxiv.org/abs/1909.05782)that can drastically speed up estimation of the entire **quantile regression process**. 

All the best, Alex

### Describe your proposed solution

I open a PR and add a new solver "fn" to `QuantileRegressor`.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

Related to https://github.com/scikit-learn/scikit-learn/issues/20132

COMMENT:
Hi both, thanks for your feedback! 

I think upstream integration into SciPy does not work - the Frisch-Newton Algorithm is a specialised solver for quantile regression and thereby not sufficiently "general interest" for SciPy?  

For convergence, we check if the duality gap is smaller than 1e-06 (following R's quantreg package). 

We check against multiple solvers in a range of tests (mostly against R-quantreg with absolute tolerance) and have a comparison of fitted coefs against a set of solvers available in Python in the gist (attached below): 

<img width="1808" height="1550" alt="Image" src="https://github.com/user-attachments/assets/f42baa01-10fa-4c49-a7d4-fdef0a522935" />

I'll tackle all the points raised by @lorentzenchr - csc inputs and time vs submodality. You can expect to hear back from me by the end of this week =)

Labels: New Feature, Needs Decision - Include Feature
Frisch-Newton Interior Point Solver for Quantile Regression

### Describe the workflow you want to enable

Hi @ scikit-learn devs! 

Over at [pyfixest](https://github.com/py-econometrics/pyfixest), we have implemented a Frisch-Newton Interior Point solver to fit quantile regressions. The algorithm goes back to work from Koenker. In practice, we have followed Koenker and Ng ["A Frisch-Newton Algorithm for Sparse Quantile Regression". ](https://link.springer.com/article/10.1007/s10255-005-0231-1)

The code is licensed under MIT and available [here](https://github.com/py-econometrics/pyfixest/blob/master/pyfixest/estimation/quantreg/frisch_newton_ip.py#L70). 

We (@apoorvalal) have collected some benchmarks [here](https://gist.github.com/apoorvalal/3e18eea79c6e9e8e8ee380e0fc0bab1f) - the FN solver seems to outperform the scikit default solver by an order of a magnitude.  

<img width="1362" height="534" alt="Image" src="https://github.com/user-attachments/assets/f25eb315-60d8-464f-80f8-bf1c6aedce3b" />

Would you be interested in a PR that adds the FN solver as a new estimation method to the quantile regression class? 

We've also implemented algorithms from [Chernozhukov et al ](https://arxiv.org/abs/1909.05782)that can drastically speed up estimation of the entire **quantile regression process**. 

All the best, Alex

### Describe your proposed solution

I open a PR and add a new solver "fn" to `QuantileRegressor`.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

Related to https://github.com/scikit-learn/scikit-learn/issues/20132

COMMENT:
@s3alfisc Note that an such an addition that improves by an order of magnitude is, for sure, welcome. But note that this usually involves quite an investment of time and, in this case, benchmarking.

About benchmarking: One usually plots suboptimality (current objective function minus minimum) on the y-axis and time on the x-axis. And then different colored lines for different solvers, see e.g. https://github.com/scikit-learn/scikit-learn/pull/24767.

Labels: New Feature, Needs Decision - Include Feature
Frisch-Newton Interior Point Solver for Quantile Regression

### Describe the workflow you want to enable

Hi @ scikit-learn devs! 

Over at [pyfixest](https://github.com/py-econometrics/pyfixest), we have implemented a Frisch-Newton Interior Point solver to fit quantile regressions. The algorithm goes back to work from Koenker. In practice, we have followed Koenker and Ng ["A Frisch-Newton Algorithm for Sparse Quantile Regression". ](https://link.springer.com/article/10.1007/s10255-005-0231-1)

The code is licensed under MIT and available [here](https://github.com/py-econometrics/pyfixest/blob/master/pyfixest/estimation/quantreg/frisch_newton_ip.py#L70). 

We (@apoorvalal) have collected some benchmarks [here](https://gist.github.com/apoorvalal/3e18eea79c6e9e8e8ee380e0fc0bab1f) - the FN solver seems to outperform the scikit default solver by an order of a magnitude.  

<img width="1362" height="534" alt="Image" src="https://github.com/user-attachments/assets/f25eb315-60d8-464f-80f8-bf1c6aedce3b" />

Would you be interested in a PR that adds the FN solver as a new estimation method to the quantile regression class? 

We've also implemented algorithms from [Chernozhukov et al ](https://arxiv.org/abs/1909.05782)that can drastically speed up estimation of the entire **quantile regression process**. 

All the best, Alex

### Describe your proposed solution

I open a PR and add a new solver "fn" to `QuantileRegressor`.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

Related to https://github.com/scikit-learn/scikit-learn/issues/20132

COMMENT:
@s3alfisc Thanks for your great update.
I am not 100% sure with linear programming, but I have the feeling that without the same precision of the two solutions your plots, as nice as they are, are somewhat meaningless. The `options` argument of scipy's `linprog` solver can be passed to [`QuantileRegressor`](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.QuantileRegressor.html#sklearn.linear_model.QuantileRegressor) via `solver_options`, e.g. `QuantileRegressor(solver_options={"tol": 1e-10})` (or use `"maxiter"` to control it).

Could you make a plot that show `pinball loss + L1` on the x-axis and timing on the y-axis?

As you already have the code, opening a PR would make sense. Then you profit from all the already written tests.
About inclusion or not, however, we are not yet settled.

Labels: New Feature, Needs Decision - Include Feature
Frisch-Newton Interior Point Solver for Quantile Regression

### Describe the workflow you want to enable

Hi @ scikit-learn devs! 

Over at [pyfixest](https://github.com/py-econometrics/pyfixest), we have implemented a Frisch-Newton Interior Point solver to fit quantile regressions. The algorithm goes back to work from Koenker. In practice, we have followed Koenker and Ng ["A Frisch-Newton Algorithm for Sparse Quantile Regression". ](https://link.springer.com/article/10.1007/s10255-005-0231-1)

The code is licensed under MIT and available [here](https://github.com/py-econometrics/pyfixest/blob/master/pyfixest/estimation/quantreg/frisch_newton_ip.py#L70). 

We (@apoorvalal) have collected some benchmarks [here](https://gist.github.com/apoorvalal/3e18eea79c6e9e8e8ee380e0fc0bab1f) - the FN solver seems to outperform the scikit default solver by an order of a magnitude.  

<img width="1362" height="534" alt="Image" src="https://github.com/user-attachments/assets/f25eb315-60d8-464f-80f8-bf1c6aedce3b" />

Would you be interested in a PR that adds the FN solver as a new estimation method to the quantile regression class? 

We've also implemented algorithms from [Chernozhukov et al ](https://arxiv.org/abs/1909.05782)that can drastically speed up estimation of the entire **quantile regression process**. 

All the best, Alex

### Describe your proposed solution

I open a PR and add a new solver "fn" to `QuantileRegressor`.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

Related to https://github.com/scikit-learn/scikit-learn/issues/20132

COMMENT:
> but I have the feeling that without the same precision of the two solutions your plots, as nice as they are, are somewhat meaningless.

That's a fair point! I'll have to take a closer look into the highs solver options to figure out what is being done exactly, at first glance it did not seem straightforward to fix a tolerance for the duality gap / avg pinball loss. 

> Could you make a plot that show pinball loss + L1 on the x-axis and timing on the y-axis?

Here you mean pinball loss over iteration time? I.e. loss after 1 seconds, 2 seconds, etc? The challenge here is that linprog does not expose the pinball loss until convergence and returns none when early stopping, i,e, via maxiter. Will have to look at the highspy docs to do this. 

With L1 you mean the L1 norm? If yes I am afraid I did not make this explicit enough - the FN solver as of now only works without penalization, which opens the question of how useful it even is for the `QuantileRegressor` method (which defaults to L1?) and your users. From personal experience, I would assume the `QuantileRegressor` is often used for large N, small k problems, where over fitting should not be as much of an issue? 

> As you already have the code, opening a PR would make sense. Then you profit from all the already written tests.

I'll do this then!

>  About inclusion or not, however, we are not yet settled.

Yes, absolutely! I still have some homework to do =)

Labels: New Feature, Needs Decision - Include Feature
Frisch-Newton Interior Point Solver for Quantile Regression

### Describe the workflow you want to enable

Hi @ scikit-learn devs! 

Over at [pyfixest](https://github.com/py-econometrics/pyfixest), we have implemented a Frisch-Newton Interior Point solver to fit quantile regressions. The algorithm goes back to work from Koenker. In practice, we have followed Koenker and Ng ["A Frisch-Newton Algorithm for Sparse Quantile Regression". ](https://link.springer.com/article/10.1007/s10255-005-0231-1)

The code is licensed under MIT and available [here](https://github.com/py-econometrics/pyfixest/blob/master/pyfixest/estimation/quantreg/frisch_newton_ip.py#L70). 

We (@apoorvalal) have collected some benchmarks [here](https://gist.github.com/apoorvalal/3e18eea79c6e9e8e8ee380e0fc0bab1f) - the FN solver seems to outperform the scikit default solver by an order of a magnitude.  

<img width="1362" height="534" alt="Image" src="https://github.com/user-attachments/assets/f25eb315-60d8-464f-80f8-bf1c6aedce3b" />

Would you be interested in a PR that adds the FN solver as a new estimation method to the quantile regression class? 

We've also implemented algorithms from [Chernozhukov et al ](https://arxiv.org/abs/1909.05782)that can drastically speed up estimation of the entire **quantile regression process**. 

All the best, Alex

### Describe your proposed solution

I open a PR and add a new solver "fn" to `QuantileRegressor`.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

Related to https://github.com/scikit-learn/scikit-learn/issues/20132

COMMENT:
> Could you make a plot that show pinball loss + L1 on the x-axis and timing on the y-axis?

Let‘s neglect L1 as you explained. One possibility is to run with increasing `maxiter`, record the time and the pinball loss of each run and plot the loss (of a run) on the y-axis and the time (of a run) on the x-axis.

Labels: New Feature, Needs Decision - Include Feature
Frisch-Newton Interior Point Solver for Quantile Regression

### Describe the workflow you want to enable

Hi @ scikit-learn devs! 

Over at [pyfixest](https://github.com/py-econometrics/pyfixest), we have implemented a Frisch-Newton Interior Point solver to fit quantile regressions. The algorithm goes back to work from Koenker. In practice, we have followed Koenker and Ng ["A Frisch-Newton Algorithm for Sparse Quantile Regression". ](https://link.springer.com/article/10.1007/s10255-005-0231-1)

The code is licensed under MIT and available [here](https://github.com/py-econometrics/pyfixest/blob/master/pyfixest/estimation/quantreg/frisch_newton_ip.py#L70). 

We (@apoorvalal) have collected some benchmarks [here](https://gist.github.com/apoorvalal/3e18eea79c6e9e8e8ee380e0fc0bab1f) - the FN solver seems to outperform the scikit default solver by an order of a magnitude.  

<img width="1362" height="534" alt="Image" src="https://github.com/user-attachments/assets/f25eb315-60d8-464f-80f8-bf1c6aedce3b" />

Would you be interested in a PR that adds the FN solver as a new estimation method to the quantile regression class? 

We've also implemented algorithms from [Chernozhukov et al ](https://arxiv.org/abs/1909.05782)that can drastically speed up estimation of the entire **quantile regression process**. 

All the best, Alex

### Describe your proposed solution

I open a PR and add a new solver "fn" to `QuantileRegressor`.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

Related to https://github.com/scikit-learn/scikit-learn/issues/20132

COMMENT:
Hi, I've now done the following: 

- for the highs solver, iterate over multiple **tolerances**, not maxiters - the reason is that if the highs solver does not converge, it does not return any results.
- for the FN solver, I simply loop over different maxiter tresholds. 

This produces the following figures (note that both are a little fuzzy as I cannot trace the full log of a single iteration, but combine multiple interations). 

<img width="3956" height="1978" alt="Image" src="https://github.com/user-attachments/assets/ad015846-dbb5-449f-8fee-a1dcd3505afa" />


I've also opened a draft PR: https://github.com/scikit-learn/scikit-learn/pull/32002

Labels: New Feature, Needs Decision - Include Feature
Frisch-Newton Interior Point Solver for Quantile Regression

### Describe the workflow you want to enable

Hi @ scikit-learn devs! 

Over at [pyfixest](https://github.com/py-econometrics/pyfixest), we have implemented a Frisch-Newton Interior Point solver to fit quantile regressions. The algorithm goes back to work from Koenker. In practice, we have followed Koenker and Ng ["A Frisch-Newton Algorithm for Sparse Quantile Regression". ](https://link.springer.com/article/10.1007/s10255-005-0231-1)

The code is licensed under MIT and available [here](https://github.com/py-econometrics/pyfixest/blob/master/pyfixest/estimation/quantreg/frisch_newton_ip.py#L70). 

We (@apoorvalal) have collected some benchmarks [here](https://gist.github.com/apoorvalal/3e18eea79c6e9e8e8ee380e0fc0bab1f) - the FN solver seems to outperform the scikit default solver by an order of a magnitude.  

<img width="1362" height="534" alt="Image" src="https://github.com/user-attachments/assets/f25eb315-60d8-464f-80f8-bf1c6aedce3b" />

Would you be interested in a PR that adds the FN solver as a new estimation method to the quantile regression class? 

We've also implemented algorithms from [Chernozhukov et al ](https://arxiv.org/abs/1909.05782)that can drastically speed up estimation of the entire **quantile regression process**. 

All the best, Alex

### Describe your proposed solution

I open a PR and add a new solver "fn" to `QuantileRegressor`.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

Related to https://github.com/scikit-learn/scikit-learn/issues/20132

COMMENT:
Thanks! Those pinball-loss vs time plots look very promising. I will try to find some time helping with the review of the PR.

Labels: New Feature, Needs Decision - Include Feature