Talip7/scikit-learn-issues-multilabel huggingface dataset:
Intercepts of Newton-Cholesky logistic regression get corrupted when warm starting

### Describe the bug

When using multinomial logistic regression with warm starts from a previous iteration, the final coefficients in the model are correct, but the intercepts somehow get filled with incorrect numbers somewhere.

As a result, predictions from a warm-started model differ from those of a cold-start model that has more iterations on the same data.

The issue appears to have been introduced recently as it works fine with version 1.5, but not with 1.6 or 1.7.

### Steps/Code to Reproduce

```python
import numpy as np
from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
X, y = load_iris(return_X_y=True)

model1 = LogisticRegression(
    solver="newton-cholesky",
    max_iter=2
).fit(X, y)
model2 = LogisticRegression(
    solver="newton-cholesky",
    max_iter=1,
    warm_start=True
).fit(X, y).fit(X, y)

np.testing.assert_almost_equal(
    model1.coef_,
    model2.coef_
)

np.testing.assert_almost_equal(
    model1.predict_proba(X[:5]),
    model2.predict_proba(X[:5])
)
```

### Expected Results

Intercepts should be the same, up to shifting by a constant if needed.

### Actual Results

Intercepts are different, as are predicted probabilities

### Versions

```shell
System:
    python: 3.12.6 | packaged by conda-forge | (main, Sep 22 2024, 14:16:49) [GCC 13.3.0]
executable: /home/david/miniforge3/bin/python
   machine: Linux-6.12.33+deb12-amd64-x86_64-with-glibc2.36

Python dependencies:
      sklearn: 1.7.1
          pip: 24.2
   setuptools: 74.1.2
        numpy: 2.0.1
        scipy: 1.14.1
       Cython: 3.1.0
       pandas: 2.2.3
   matplotlib: 3.9.2
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 20
         prefix: libscipy_openblas
       filepath: /home/david/.local/lib/python3.12/site-packages/numpy.libs/libscipy_openblas64_-99b71e71.so
        version: 0.3.27
threading_layer: pthreads
   architecture: Haswell

       user_api: blas
   internal_api: mkl
    num_threads: 14
         prefix: libmkl_rt
       filepath: /home/david/miniforge3/lib/libmkl_rt.so.2
        version: 2023.2-Product
threading_layer: gnu

       user_api: openmp
   internal_api: openmp
    num_threads: 20
         prefix: libgomp
       filepath: /home/david/miniforge3/lib/libgomp.so.1.0.0
        version: None
```

COMMENT:
> After iteration 2, the predicted probabilities differ

They do not differ:
```python
import numpy as np
from scipy.special import softmax
coefs = np.array([2, -2])
intercepts1 = np.array([-3, 5])
intercepts2 = np.array([-2, 6])

rng = np.random.default_rng(seed=123)
x = rng.standard_normal(size=(10,1))
raw1 = x@coefs.reshape((1,2)) + intercepts1
raw2 = x@coefs.reshape((1,2)) + intercepts2

np.testing.assert_almost_equal(
    softmax(raw1),
    softmax(raw2)
)
```

Labels: Bug, module:linear_model
Intercepts of Newton-Cholesky logistic regression get corrupted when warm starting

### Describe the bug

When using multinomial logistic regression with warm starts from a previous iteration, the final coefficients in the model are correct, but the intercepts somehow get filled with incorrect numbers somewhere.

As a result, predictions from a warm-started model differ from those of a cold-start model that has more iterations on the same data.

The issue appears to have been introduced recently as it works fine with version 1.5, but not with 1.6 or 1.7.

### Steps/Code to Reproduce

```python
import numpy as np
from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
X, y = load_iris(return_X_y=True)

model1 = LogisticRegression(
    solver="newton-cholesky",
    max_iter=2
).fit(X, y)
model2 = LogisticRegression(
    solver="newton-cholesky",
    max_iter=1,
    warm_start=True
).fit(X, y).fit(X, y)

np.testing.assert_almost_equal(
    model1.coef_,
    model2.coef_
)

np.testing.assert_almost_equal(
    model1.predict_proba(X[:5]),
    model2.predict_proba(X[:5])
)
```

### Expected Results

Intercepts should be the same, up to shifting by a constant if needed.

### Actual Results

Intercepts are different, as are predicted probabilities

### Versions

```shell
System:
    python: 3.12.6 | packaged by conda-forge | (main, Sep 22 2024, 14:16:49) [GCC 13.3.0]
executable: /home/david/miniforge3/bin/python
   machine: Linux-6.12.33+deb12-amd64-x86_64-with-glibc2.36

Python dependencies:
      sklearn: 1.7.1
          pip: 24.2
   setuptools: 74.1.2
        numpy: 2.0.1
        scipy: 1.14.1
       Cython: 3.1.0
       pandas: 2.2.3
   matplotlib: 3.9.2
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 20
         prefix: libscipy_openblas
       filepath: /home/david/.local/lib/python3.12/site-packages/numpy.libs/libscipy_openblas64_-99b71e71.so
        version: 0.3.27
threading_layer: pthreads
   architecture: Haswell

       user_api: blas
   internal_api: mkl
    num_threads: 14
         prefix: libmkl_rt
       filepath: /home/david/miniforge3/lib/libmkl_rt.so.2
        version: 2023.2-Product
threading_layer: gnu

       user_api: openmp
   internal_api: openmp
    num_threads: 20
         prefix: libgomp
       filepath: /home/david/miniforge3/lib/libgomp.so.1.0.0
        version: None
```

COMMENT:
> as users are not supposed to use the solver that way

Broader problem is: it makes warm starts perform worse than cold starts.

Labels: Bug, module:linear_model
Intercepts of Newton-Cholesky logistic regression get corrupted when warm starting

### Describe the bug

When using multinomial logistic regression with warm starts from a previous iteration, the final coefficients in the model are correct, but the intercepts somehow get filled with incorrect numbers somewhere.

As a result, predictions from a warm-started model differ from those of a cold-start model that has more iterations on the same data.

The issue appears to have been introduced recently as it works fine with version 1.5, but not with 1.6 or 1.7.

### Steps/Code to Reproduce

```python
import numpy as np
from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
X, y = load_iris(return_X_y=True)

model1 = LogisticRegression(
    solver="newton-cholesky",
    max_iter=2
).fit(X, y)
model2 = LogisticRegression(
    solver="newton-cholesky",
    max_iter=1,
    warm_start=True
).fit(X, y).fit(X, y)

np.testing.assert_almost_equal(
    model1.coef_,
    model2.coef_
)

np.testing.assert_almost_equal(
    model1.predict_proba(X[:5]),
    model2.predict_proba(X[:5])
)
```

### Expected Results

Intercepts should be the same, up to shifting by a constant if needed.

### Actual Results

Intercepts are different, as are predicted probabilities

### Versions

```shell
System:
    python: 3.12.6 | packaged by conda-forge | (main, Sep 22 2024, 14:16:49) [GCC 13.3.0]
executable: /home/david/miniforge3/bin/python
   machine: Linux-6.12.33+deb12-amd64-x86_64-with-glibc2.36

Python dependencies:
      sklearn: 1.7.1
          pip: 24.2
   setuptools: 74.1.2
        numpy: 2.0.1
        scipy: 1.14.1
       Cython: 3.1.0
       pandas: 2.2.3
   matplotlib: 3.9.2
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 20
         prefix: libscipy_openblas
       filepath: /home/david/.local/lib/python3.12/site-packages/numpy.libs/libscipy_openblas64_-99b71e71.so
        version: 0.3.27
threading_layer: pthreads
   architecture: Haswell

       user_api: blas
   internal_api: mkl
    num_threads: 14
         prefix: libmkl_rt
       filepath: /home/david/miniforge3/lib/libmkl_rt.so.2
        version: 2023.2-Product
threading_layer: gnu

       user_api: openmp
   internal_api: openmp
    num_threads: 20
         prefix: libgomp
       filepath: /home/david/miniforge3/lib/libgomp.so.1.0.0
        version: None
```

COMMENT:
There is indeed a bug in the code...I'm investigating.

Labels: Bug, module:linear_model
Extend make file to inlcude initial setup installations.

### Describe the workflow you want to enable

I recently made my first contribution to sklearn and found it a bit tidious to do the initial setup after cloning the repo. I think that extending the make file to include something similar to `make inital setup` to install the dependencies in [step 4 of the Contributing > Contributing code > How to contribute](https://scikit-learn.org/stable/developers/contributing.html) would be benefitial. Additionnaly adding a script ot run the git commands. I'd love to implement this so please, let me know if this is something of interest! 

### Describe your proposed solution

Extending the make file to include something similar to `make inital setup` to install the dependencies in [step 4 of the Contributing > Contributing code > How to contribute](https://scikit-learn.org/stable/developers/contributing.html)

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Congrats for your first scikit-learn contribution 🎉 and thanks for taking the time to try to improve the contributor experience 🙏!

I would agree there are quite a few manual steps to get set-up but I am not sure it's easy to improve the situation unfortunately. There are plenty off trade-offs to be aware of and I would say that maintainers are best placed to make the right decisions about them and improve the situation.

I am going to close this one for the following reasons:
- we have contributors on Windows and `Makefile` is not Windows friendly
- the `Makefile` is a kind of a small undocumented convenience wrapper, and I don't think we plan to change this in the near future

Labels: New Feature, Needs Triage
SkLearn IQR function

### Describe the workflow you want to enable

Recently, I was working on a machine learning project with a dataset that was quite skewed. I repeatedly had to compute the interquartile range (IQR), calculate the 25th and 75th percentiles, visualize the box plot, and then remove outliers — all manually.

While this wasn't an issue at first, it became tedious to write the same code over and over again for different rows and columns. This made me wonder: Wouldn’t it be much more efficient if scikit-learn offered a built-in utility to calculate the IQR and optionally remove or flag outliers?

I believe this kind of functionality could significantly streamline the preprocessing workflow for many users.

### Describe your proposed solution

I’d like to suggest adding a simple utility class to scikit-learn (or as part of a preprocessing module), called OutlierRemoval. This class would encapsulate all IQR-related preprocessing logic and expose a clean interface for users to apply it.

```py
class OutlierRemoval:
    def __init__(self, multiplier: float = 1.5):
        # Multiplier for the IQR rule (default is 1.5)
        ...

    def get_q1(self, X, column):
        # Returns the 25th percentile for a column
        ...

    def get_q3(self, X, column):
        # Returns the 75th percentile for a column
        ...

    def calculate_iqr(self, X, column):
        # Returns IQR = Q3 - Q1
        ...

    def plot_boxplot(self, X, column):
        # Displays a boxplot for the column
        ...

    def remove_outliers(self, X, column):
        # Removes rows with outliers from the dataset
        ...
```

Prevents redundant code when handling outliers across multiple projects

Encourages best practices in preprocessing pipelines

Makes exploratory data analysis (EDA) cleaner and more intuitive

Aligns with scikit-learn’s emphasis on reusable, composable preprocessing tools

### Describe alternatives you've considered, if relevant

I've used pandas and numpy to manually calculate percentiles and remove outliers column by column. While effective, it becomes repetitive and error-prone in large or automated workflows. A reusable, built-in utility in scikit-learn would improve consistency and save time.

### Additional context

I’d be happy to contribute a first version of this feature and refine it based on maintainer feedback. I’m currently exploring scikit-learn as a first open-source contribution and this idea came directly from real-world project pain points.

COMMENT:
We don't really support estimators which change the number of samples (long story). So this can't be an estimator right now, and it's more of a utility function. I don't think it's something we'd end up including any time soon realistically speaking.

Labels: New Feature
CI: Enable GitHub Actions App for ppc64le (Power architecture) support

Hi scikit-learn team,

We’re reaching out to propose enabling CI support for the ppc64le (IBM Power) architecture in your repository, as part of a broader effort to ensure cross-platform compatibility in the scientific Python ecosystem.

We’re using a GitHub Actions (GHA)-based runner service provided and maintained by IBM to run jobs for the ppc64le architecture. This setup has already been successfully integrated into projects like:

✅ [cryptography](https://github.com/pyca/cryptography/issues/13086)

📌 [Tracking issue in NumPy](https://github.com/numpy/numpy/issues/29125)

We’d now like to propose enabling the GitHub Actions app in this repository to allow running CI jobs for ppc64le directly via GitHub Actions. This would support upstream compatibility and help ensure continued support for the Power architecture in scikit-learn.

Key Benefits:
🔒 Ephemeral and secure runners, isolated per job

🛠️ Maintained by IBM, requires no setup effort from your side

🔁 Integrates with existing GitHub Actions workflows

📚 Technical documentation and usage details:
https://github.com/IBM/actionspz/tree/main/docs

We’re happy to assist with the setup or provide any additional details the team may need.

Thanks so much!

COMMENT:
Thanks for opening this issue and having put in work to create testing infrastructure for open-source projects.

My initial reaction to this is to ask "where do we stand with respect to ppc64le support right now? Does it work? Is it a little broken? Is it a lot broken?" and "Do we want to promise to support this platform?". These are questions for @scikit-learn/core-devs 

To add a bit of context, maybe @sandeepgupta12 you can give a short overview of what ppc64le is, where you'd fine processors like that, etc - I have to admit that I know the term Power PC, but that is about all (I think it is some ARM based thing?)

Could you explain why we need to add a GitHub App in order to use the runners? Asking because adding an app ussualy means granting the app some permissions/access rights - something that needs a bit of thinking about.

I saw in the linked Numpy thread that they raised the topic of not having (release) secrets in the jobs that end up running on the IBM runner. I think this is a good input for us to think about as well - can we, in general, reduce the amount of secrets available to runners? Do we need to do any work here?

Labels: Build / CI, Needs Decision
CI: Enable GitHub Actions App for ppc64le (Power architecture) support

Hi scikit-learn team,

We’re reaching out to propose enabling CI support for the ppc64le (IBM Power) architecture in your repository, as part of a broader effort to ensure cross-platform compatibility in the scientific Python ecosystem.

We’re using a GitHub Actions (GHA)-based runner service provided and maintained by IBM to run jobs for the ppc64le architecture. This setup has already been successfully integrated into projects like:

✅ [cryptography](https://github.com/pyca/cryptography/issues/13086)

📌 [Tracking issue in NumPy](https://github.com/numpy/numpy/issues/29125)

We’d now like to propose enabling the GitHub Actions app in this repository to allow running CI jobs for ppc64le directly via GitHub Actions. This would support upstream compatibility and help ensure continued support for the Power architecture in scikit-learn.

Key Benefits:
🔒 Ephemeral and secure runners, isolated per job

🛠️ Maintained by IBM, requires no setup effort from your side

🔁 Integrates with existing GitHub Actions workflows

📚 Technical documentation and usage details:
https://github.com/IBM/actionspz/tree/main/docs

We’re happy to assist with the setup or provide any additional details the team may need.

Thanks so much!

COMMENT:
@betatim 

Thanks for the thoughtful response!

To provide some context: ppc64le stands for PowerPC 64-bit Little Endian. It's the modern form of the PowerPC architecture and is primarily used in enterprise and high-performance computing (HPC) environments. You’ll commonly find it on IBM Power systems, including servers designed for AI workloads, scientific computing, and other data-intensive applications. It’s not ARM-based—though both are RISC architectures, they are entirely separate.

Regarding the GitHub App—yes, the runners we're using (hosted on IBM Cloud) require installing a GitHub App to securely authenticate jobs and associate them with the requesting repository. The app is scoped narrowly: it only permits workflow execution and does not access repository contents or secrets unless explicitly configured to do so. Projects like NumPy and Cryptography already use it successfully.

Also worth noting: these runners are ephemeral, meaning no data persists after the job completes. You can find more details in the FAQ here:
👉 https://github.com/IBM/actionspz/blob/main/docs/FAQ.md

Labels: Build / CI, Needs Decision
CI: Enable GitHub Actions App for ppc64le (Power architecture) support

Hi scikit-learn team,

We’re reaching out to propose enabling CI support for the ppc64le (IBM Power) architecture in your repository, as part of a broader effort to ensure cross-platform compatibility in the scientific Python ecosystem.

We’re using a GitHub Actions (GHA)-based runner service provided and maintained by IBM to run jobs for the ppc64le architecture. This setup has already been successfully integrated into projects like:

✅ [cryptography](https://github.com/pyca/cryptography/issues/13086)

📌 [Tracking issue in NumPy](https://github.com/numpy/numpy/issues/29125)

We’d now like to propose enabling the GitHub Actions app in this repository to allow running CI jobs for ppc64le directly via GitHub Actions. This would support upstream compatibility and help ensure continued support for the Power architecture in scikit-learn.

Key Benefits:
🔒 Ephemeral and secure runners, isolated per job

🛠️ Maintained by IBM, requires no setup effort from your side

🔁 Integrates with existing GitHub Actions workflows

📚 Technical documentation and usage details:
https://github.com/IBM/actionspz/tree/main/docs

We’re happy to assist with the setup or provide any additional details the team may need.

Thanks so much!

COMMENT:
> My initial reaction to this is to ask "where do we stand with respect to ppc64le support right now? Does it work? Is it a little broken? Is it a lot broken?" and "Do we want to promise to support this platform?".

I'd say that ppc64le support is very very light right now. We don't ship wheels for this platform, however the conda-forge builds do build for this platform. But we skip the test suite on the ppc64le conda-forge builds (that might be for historical reasons that no longer applies, we could test enabling it again).

Besides that, we sometimes get issues for failing tests on ppc64le, but it's been quite some time that I didn't see a new one.

I think adding a CI job for this platform would be interesting, at least to see how far we are from being able to fully support this platform.

Labels: Build / CI, Needs Decision
CI: Enable GitHub Actions App for ppc64le (Power architecture) support

Hi scikit-learn team,

We’re reaching out to propose enabling CI support for the ppc64le (IBM Power) architecture in your repository, as part of a broader effort to ensure cross-platform compatibility in the scientific Python ecosystem.

We’re using a GitHub Actions (GHA)-based runner service provided and maintained by IBM to run jobs for the ppc64le architecture. This setup has already been successfully integrated into projects like:

✅ [cryptography](https://github.com/pyca/cryptography/issues/13086)

📌 [Tracking issue in NumPy](https://github.com/numpy/numpy/issues/29125)

We’d now like to propose enabling the GitHub Actions app in this repository to allow running CI jobs for ppc64le directly via GitHub Actions. This would support upstream compatibility and help ensure continued support for the Power architecture in scikit-learn.

Key Benefits:
🔒 Ephemeral and secure runners, isolated per job

🛠️ Maintained by IBM, requires no setup effort from your side

🔁 Integrates with existing GitHub Actions workflows

📚 Technical documentation and usage details:
https://github.com/IBM/actionspz/tree/main/docs

We’re happy to assist with the setup or provide any additional details the team may need.

Thanks so much!

COMMENT:
@sandeepgupta12 could you explain a little bit who is behind this effort (I have seen that many similar issues/PRs have been opened in PyTorch, matplotlib, numpy, etc ...)? Is it IBM who cares about better support in the scientific Python ecosystem?

Honestly for this kind of "exotic" platform request, the first thing I check is whether there is  a numpy or scipy wheel, which does not seem to be the case.

Once there is a numpy and scipy wheel, I guess we may consider thinking about the "is it worth it?" question. How many of our users use `ppc64le` and is it worth the maintenance cost?

There are also security-related aspects to take into account because I am guessing this is using GitHub self-hosted runners [^1] which has implications in terms of security. The GitHub [doc](https://docs.github.com/en/actions/reference/secure-use-reference#hardening-for-self-hosted-runners) says

> self-hosted runners should almost [never be used for public repositories](https://docs.github.com/en/actions/security-for-github-actions/security-guides/security-hardening-for-github-actions) on GitHub.

[^1]: The [`IBM/actionspz` onboarding doc](https://github.com/IBM/actionspz/blob/main/docs/onboarding.md) seems to agree  although "considered self-hosted runners" wording should be clarified

Labels: Build / CI, Needs Decision
CI: Enable GitHub Actions App for ppc64le (Power architecture) support

Hi scikit-learn team,

We’re reaching out to propose enabling CI support for the ppc64le (IBM Power) architecture in your repository, as part of a broader effort to ensure cross-platform compatibility in the scientific Python ecosystem.

We’re using a GitHub Actions (GHA)-based runner service provided and maintained by IBM to run jobs for the ppc64le architecture. This setup has already been successfully integrated into projects like:

✅ [cryptography](https://github.com/pyca/cryptography/issues/13086)

📌 [Tracking issue in NumPy](https://github.com/numpy/numpy/issues/29125)

We’d now like to propose enabling the GitHub Actions app in this repository to allow running CI jobs for ppc64le directly via GitHub Actions. This would support upstream compatibility and help ensure continued support for the Power architecture in scikit-learn.

Key Benefits:
🔒 Ephemeral and secure runners, isolated per job

🛠️ Maintained by IBM, requires no setup effort from your side

🔁 Integrates with existing GitHub Actions workflows

📚 Technical documentation and usage details:
https://github.com/IBM/actionspz/tree/main/docs

We’re happy to assist with the setup or provide any additional details the team may need.

Thanks so much!

COMMENT:
@sandeepgupta12 do you  know if IBM (or those enterprises using the platform) are willing to pay for the support? This is not something our usual users need, and it would introduce significant maintenance cost on our side. So the effort needs to somewhat be justified.

Labels: Build / CI, Needs Decision