Talip7/scikit-learn-issues-multilabel huggingface dataset:
Array API backends support for MLX

It would be great to get the scikit-learn Array API back-end to be compatible with MLX (which is mostly conformant with the array API). 

Here is an example which currently does not work for a few reasons:

```python
from sklearn.datasets import make_classification
from sklearn import config_context
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis
import mlx.core as mx

X_np, y_np = make_classification(random_state=0)
X_mx = mx.array(X_np)
y_mx = mx.array(y_np)

with config_context(array_api_dispatch=True):
    lda = LinearDiscriminantAnalysis()
    X_trans = lda.fit_transform(X_mx, y_mx)

print(type(X_trans))
```

The reasons it does not work:

- MLX does not have a `float64` data type (similar to PyTorch MPS backend). It's a bit hacky to set `mx.float64 = mx.float32` so maybe good to handle this in the scikit or in a compatibility layer.

- MLX does not support operations with data-dependent output shapes, e.g. [`unique_values`](https://data-apis.org/array-api/2022.12/API_specification/generated/array_api.unique_values.html). Since these are optional in the array API should we attempt to avoid using them in scikit to get maximal compatibility with other frameworks?

- There are still a couple functions missing in MLX like `mx.asarray` and `mx.isdtype` (those are pretty easy for us to add)

Relevant discussion in MLX https://github.com/ml-explore/mlx/pull/1289

CC @betatim

COMMENT:
I think someone has to investigate more the details about the actual status of array API spec support in MLX and summarize the main discrepancies and check whether they can be addressed either in `array-api-compat` or `array-api-extra` to be able to support MLX in scikit-learn.

Note that the value dependent output shapes problem seems to be similar to the dask array API compatibility problem. We do not see any easy resolution plan for this, unfortunately.

Maybe the JAX-specific helpers of `array-api-extra` can help design similar helpers to address those constraints in MLX. That also requires investigation.

Labels: New Feature, Needs Investigation, Array API
`fetch_20newsgroups_vectorized` gives HTTP Error 403 Forbidden

### Describe the bug

This was also recently reported on [StackOverflow](https://stackoverflow.com/questions/78398259/lda-in-python-shows-403-error-in-fetching-20newsgroups-dataset). It appears that https://ndownloader.figshare.com is down.

### Steps/Code to Reproduce

```python
from sklearn.datasets import fetch_20newsgroups_vectorized
newsgroups_vectorized = fetch_20newsgroups_vectorized(subset='test')
```


### Expected Results

The dataset is downloaded.

### Actual Results

```
urllib.error.HTTPError: HTTP Error 403: Forbidden
```

### Versions

```shell
>>> import sklearn; sklearn.show_versions()

System:
    python: 3.9.4 (tags/v3.9.4:1f2e308, Apr  6 2021, 13:40:21) [MSC v.1928 64 bit (AMD64)]
executable: C:\Users\Asus\AppData\Local\Programs\Python\Python39\python.exe
   machine: Windows-10-10.0.19041-SP0

Python dependencies:
      sklearn: 1.5.1
          pip: 24.1
   setuptools: 69.5.1
        numpy: 1.26.4
        scipy: 1.13.1
       Cython: 0.29.32
       pandas: 2.2.2
   matplotlib: 3.9.0
       joblib: 1.3.2
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
         prefix: vcomp
       filepath: C:\Users\Asus\AppData\Local\Programs\Python\Python39\Lib\site-packages\sklearn\.libs\vcomp140.dll
        version: None
    num_threads: 16

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: C:\Users\Asus\AppData\Local\Programs\Python\Python39\Lib\site-packages\numpy.libs\libopenblas64__v0.3.23-293-gc2f4bdbb-gcc_10_3_0-2bde3a66a51006b2b53eb373ff767a3f.dll
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: Zen
    num_threads: 16

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: C:\Users\Asus\AppData\Local\Programs\Python\Python39\Lib\site-packages\scipy.libs\libopenblas_v0.3.27--3aa239bc726cfb0bd8e5330d8d4c15c6.dll
        version: 0.3.27
threading_layer: pthreads
   architecture: Zen
    num_threads: 16
```

COMMENT:
That's due to some `openml.org` server issues unfortunately. You can contact them or send a message on their slack and ask.

Labels: Bug, Needs Triage
GradientBoostingClassifier feature_importances_ is all zero

### Describe the bug

I'm using GradientBoostingClassifier on a rather small dataset (n=75) for classification & feature selection.
I'm grid searching (in cross validation) the best hyper-parameters for my data and on some grids I get 0 importance for every feature (and on others, everything is ok). 
I can provide the data if needed. 
p.s. this is my first issue post, If more information / change in format is needed, please let me know

### Steps/Code to Reproduce

# code for iterative feature reduction     
    curX = X.copy()
    for feature_amount in tqdm(feature_amount_lst,desc = 'Feature Selection',leave=False):
        model.fit(curX,y)
        feature_imp = pd.Series(model.feature_importances_, index = curX.columns).sort_values(ascending = False)
        feature_imp.name = 'importances'

        feature_imp = feature_imp[:feature_amount]
        if importances_list is not None:
            importances_list.append(feature_imp)
        features_list.append(list(feature_imp.index))
        curX = curX[features_list[-1]]

### Expected Results

Importance array that sums to 1

### Actual Results

![image](https://github.com/user-attachments/assets/b2fd3459-c0dd-41cb-8c82-25025c50d31d)

To be clear - this is after fitting

### Versions

```shell
System:
    python: 3.9.6 (tags/v3.9.6:db3ff76, Jun 28 2021, 15:26:21) [MSC v.1929 64 bit (AMD64)]
executable: c:\Users\cs-lab\AppData\Local\Programs\Python\Python39\python.exe
   machine: Windows-10-10.0.19045-SP0

Python dependencies:
      sklearn: 1.5.1
          pip: 24.0
   setuptools: 56.0.0
        numpy: 1.22.4
        scipy: 1.12.0
       Cython: 0.29.37
       pandas: 1.3.2
   matplotlib: 3.8.3
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
    num_threads: 12
         prefix: vcomp
       filepath: C:\Users\cs-lab\AppData\Roaming\Python\Python39\site-packages\sklearn\.libs\vcomp140.dll
        version: None

       user_api: blas
   internal_api: openblas
    num_threads: 12
         prefix: libopenblas
       filepath: C:\Users\cs-lab\AppData\Local\Programs\Python\Python39\Lib\site-packages\numpy\.libs\libopenblas.EL2C6PLE4ZYW3ECEVIV3OXXGRN2NRFM2.gfortran-win_amd64.dll
        version: 0.3.18
threading_layer: pthreads
   architecture: Haswell

       user_api: blas
   internal_api: openblas
    num_threads: 12
         prefix: libopenblas
       filepath: C:\Users\cs-lab\AppData\Local\Programs\Python\Python39\Lib\site-packages\scipy.libs\libopenblas_v0.3.20-571-g3dec11c6-gcc_10_3_0-c2315440d6b6cef5037bad648efc8c59.dll
        version: 0.3.21.dev
threading_layer: pthreads
   architecture: Haswell
```

COMMENT:
If certain features are not used for prediction, which can happen very easily especially when there aren't enough data (which is your case), then those importance values would be 0. This is expected behavior. You can try to get a confidence value on your importances from the different cross validation folds to see how reliable those values are.

Labels: Bug, Needs Triage
GradientBoostingClassifier feature_importances_ is all zero

### Describe the bug

I'm using GradientBoostingClassifier on a rather small dataset (n=75) for classification & feature selection.
I'm grid searching (in cross validation) the best hyper-parameters for my data and on some grids I get 0 importance for every feature (and on others, everything is ok). 
I can provide the data if needed. 
p.s. this is my first issue post, If more information / change in format is needed, please let me know

### Steps/Code to Reproduce

# code for iterative feature reduction     
    curX = X.copy()
    for feature_amount in tqdm(feature_amount_lst,desc = 'Feature Selection',leave=False):
        model.fit(curX,y)
        feature_imp = pd.Series(model.feature_importances_, index = curX.columns).sort_values(ascending = False)
        feature_imp.name = 'importances'

        feature_imp = feature_imp[:feature_amount]
        if importances_list is not None:
            importances_list.append(feature_imp)
        features_list.append(list(feature_imp.index))
        curX = curX[features_list[-1]]

### Expected Results

Importance array that sums to 1

### Actual Results

![image](https://github.com/user-attachments/assets/b2fd3459-c0dd-41cb-8c82-25025c50d31d)

To be clear - this is after fitting

### Versions

```shell
System:
    python: 3.9.6 (tags/v3.9.6:db3ff76, Jun 28 2021, 15:26:21) [MSC v.1929 64 bit (AMD64)]
executable: c:\Users\cs-lab\AppData\Local\Programs\Python\Python39\python.exe
   machine: Windows-10-10.0.19045-SP0

Python dependencies:
      sklearn: 1.5.1
          pip: 24.0
   setuptools: 56.0.0
        numpy: 1.22.4
        scipy: 1.12.0
       Cython: 0.29.37
       pandas: 1.3.2
   matplotlib: 3.8.3
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
    num_threads: 12
         prefix: vcomp
       filepath: C:\Users\cs-lab\AppData\Roaming\Python\Python39\site-packages\sklearn\.libs\vcomp140.dll
        version: None

       user_api: blas
   internal_api: openblas
    num_threads: 12
         prefix: libopenblas
       filepath: C:\Users\cs-lab\AppData\Local\Programs\Python\Python39\Lib\site-packages\numpy\.libs\libopenblas.EL2C6PLE4ZYW3ECEVIV3OXXGRN2NRFM2.gfortran-win_amd64.dll
        version: 0.3.18
threading_layer: pthreads
   architecture: Haswell

       user_api: blas
   internal_api: openblas
    num_threads: 12
         prefix: libopenblas
       filepath: C:\Users\cs-lab\AppData\Local\Programs\Python\Python39\Lib\site-packages\scipy.libs\libopenblas_v0.3.20-571-g3dec11c6-gcc_10_3_0-c2315440d6b6cef5037bad648efc8c59.dll
        version: 0.3.21.dev
threading_layer: pthreads
   architecture: Haswell
```

COMMENT:
How can every feature be 0 though?
Nothing is selected for classification?

Labels: Bug, Needs Triage
GradientBoostingClassifier feature_importances_ is all zero

### Describe the bug

I'm using GradientBoostingClassifier on a rather small dataset (n=75) for classification & feature selection.
I'm grid searching (in cross validation) the best hyper-parameters for my data and on some grids I get 0 importance for every feature (and on others, everything is ok). 
I can provide the data if needed. 
p.s. this is my first issue post, If more information / change in format is needed, please let me know

### Steps/Code to Reproduce

# code for iterative feature reduction     
    curX = X.copy()
    for feature_amount in tqdm(feature_amount_lst,desc = 'Feature Selection',leave=False):
        model.fit(curX,y)
        feature_imp = pd.Series(model.feature_importances_, index = curX.columns).sort_values(ascending = False)
        feature_imp.name = 'importances'

        feature_imp = feature_imp[:feature_amount]
        if importances_list is not None:
            importances_list.append(feature_imp)
        features_list.append(list(feature_imp.index))
        curX = curX[features_list[-1]]

### Expected Results

Importance array that sums to 1

### Actual Results

![image](https://github.com/user-attachments/assets/b2fd3459-c0dd-41cb-8c82-25025c50d31d)

To be clear - this is after fitting

### Versions

```shell
System:
    python: 3.9.6 (tags/v3.9.6:db3ff76, Jun 28 2021, 15:26:21) [MSC v.1929 64 bit (AMD64)]
executable: c:\Users\cs-lab\AppData\Local\Programs\Python\Python39\python.exe
   machine: Windows-10-10.0.19045-SP0

Python dependencies:
      sklearn: 1.5.1
          pip: 24.0
   setuptools: 56.0.0
        numpy: 1.22.4
        scipy: 1.12.0
       Cython: 0.29.37
       pandas: 1.3.2
   matplotlib: 3.8.3
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
    num_threads: 12
         prefix: vcomp
       filepath: C:\Users\cs-lab\AppData\Roaming\Python\Python39\site-packages\sklearn\.libs\vcomp140.dll
        version: None

       user_api: blas
   internal_api: openblas
    num_threads: 12
         prefix: libopenblas
       filepath: C:\Users\cs-lab\AppData\Local\Programs\Python\Python39\Lib\site-packages\numpy\.libs\libopenblas.EL2C6PLE4ZYW3ECEVIV3OXXGRN2NRFM2.gfortran-win_amd64.dll
        version: 0.3.18
threading_layer: pthreads
   architecture: Haswell

       user_api: blas
   internal_api: openblas
    num_threads: 12
         prefix: libopenblas
       filepath: C:\Users\cs-lab\AppData\Local\Programs\Python\Python39\Lib\site-packages\scipy.libs\libopenblas_v0.3.20-571-g3dec11c6-gcc_10_3_0-c2315440d6b6cef5037bad648efc8c59.dll
        version: 0.3.21.dev
threading_layer: pthreads
   architecture: Haswell
```

COMMENT:
Hard to say w/o a minimal reproducer.

Labels: Bug, Needs Triage
⚠️ CI failed on Wheel builder (last failure: Aug 11, 2024) ⚠️

**CI failed on [Wheel builder](https://github.com/scikit-learn/scikit-learn/actions/runs/10336780837)** (Aug 11, 2024)

COMMENT:
## CI is no longer failing! ✅

[Successful run](https://github.com/scikit-learn/scikit-learn/actions/runs/10345565152) on Aug 12, 2024

Labels: Needs Triage
Expand build from source docs for debugging with meson

From https://github.com/scikit-learn/scikit-learn/pull/29594#issuecomment-2260154987 and https://github.com/scikit-learn/scikit-learn/pull/29594#issuecomment-2260158387:

> Could you please open a follow-up PR that expands either our "build from source" documentation or our documentation on "how to debug/profile" to explain how to switch between "release" and "debugoptimized" via by using a pip commandline flag and explains the expected impact (in terms of binary size and ability to use a debugger/profiler for native code).

> In particular it would be interesting to see the impact of this switch when using a profiler such as linux perf (see this page for Python 3.12 specific integration) on a Python script that relies heavily on native code (e.g. fitting HistGradientBosttingClassifier which is mostly Cython).

> And similarly check that it works as expected for py-spy's support for native extension profiling:

COMMENT:
cc @lesteve

Labels: Documentation, Build / CI
GaussianNB(priors=...) is useless

### Describe the bug

If I set the class priors to be very small for classes 0 and 2 and very large for class 1, I expect my predictions to be of class 1. However, I get class 0. It seems to be that `GaussianNB(priors=...)` is useless.

### Steps/Code to Reproduce

```python
from sklearn import datasets
from sklearn.naive_bayes import GaussianNB

# Load data
iris = datasets.load_iris()
features = iris.data
target = iris.target

# Create Gaussian naive Bayes object
classifer = GaussianNB()  # The prior is adjusted based on the data
# Train model
model = classifer.fit(features, target)
# Create new observation
new_observation = [[5.2, 3.6, 1.5, 0.3]]
# Predict class
model.predict(new_observation)
# array([0])

# Set prior probabilities p(y) of each class of 3 (does not work)
clf = GaussianNB(priors=[1e-12, 1-1e-11, 9e-12])
# Train model
model_priors = clf.fit(features, target)
# Create new observation
new_observation = [[5.2, 3.6, 1.5, 0.3]]
# Predict class
model_priors.predict(new_observation)
# array([0])
```

### Expected Results

- `array([0])` for `GaussianNB` without priors
- `array([1])` for `GaussianNB(priors=...)` with priors

### Actual Results

- `array([0])` for `GaussianNB` without priors
- `array([0])` for `GaussianNB(priors=...)` with priors

### Versions

```shell
System:
    python: 3.11.0rc1 (main, Aug 12 2022, 10:02:14) [GCC 11.2.0]
machine: Linux-6.5.0-44-generic-x86_64-with-glibc2.35

Python dependencies:
      sklearn: 1.4.1.post1
          pip: 24.1.2
   setuptools: 68.2.0
        numpy: 1.26.4
        scipy: 1.12.0
       Cython: None
       pandas: 2.2.1
   matplotlib: 3.8.3
       joblib: 1.3.2
threadpoolctl: 3.4.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /home/.../venv/lib/python3.11/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: Zen

       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libgomp
       filepath: /home/.../venv/lib/python3.11/site-packages/faiss_cpu.libs/libgomp-a34b3233.so.1.0.0
        version: None

       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /home/.../venv/lib/python3.11/site-packages/scipy.libs/libopenblasp-r0-23e5df77.3.21.dev.so
        version: 0.3.21.dev
threading_layer: pthreads
   architecture: Zen
```

COMMENT:
Another notice: `target` contains classes 0, 1, 2 with 50 times of each
```python
import numpy as np
from sklearn import datasets

# Load data
iris = datasets.load_iris()
features = iris.data
target = iris.target

for cls in np.unique(target):
    print(cls, target[target==cls].shape[0])
# 0 50
# 1 50
# 2 50
```

Labels: Bug, Needs Triage
GaussianNB(priors=...) is useless

### Describe the bug

If I set the class priors to be very small for classes 0 and 2 and very large for class 1, I expect my predictions to be of class 1. However, I get class 0. It seems to be that `GaussianNB(priors=...)` is useless.

### Steps/Code to Reproduce

```python
from sklearn import datasets
from sklearn.naive_bayes import GaussianNB

# Load data
iris = datasets.load_iris()
features = iris.data
target = iris.target

# Create Gaussian naive Bayes object
classifer = GaussianNB()  # The prior is adjusted based on the data
# Train model
model = classifer.fit(features, target)
# Create new observation
new_observation = [[5.2, 3.6, 1.5, 0.3]]
# Predict class
model.predict(new_observation)
# array([0])

# Set prior probabilities p(y) of each class of 3 (does not work)
clf = GaussianNB(priors=[1e-12, 1-1e-11, 9e-12])
# Train model
model_priors = clf.fit(features, target)
# Create new observation
new_observation = [[5.2, 3.6, 1.5, 0.3]]
# Predict class
model_priors.predict(new_observation)
# array([0])
```

### Expected Results

- `array([0])` for `GaussianNB` without priors
- `array([1])` for `GaussianNB(priors=...)` with priors

### Actual Results

- `array([0])` for `GaussianNB` without priors
- `array([0])` for `GaussianNB(priors=...)` with priors

### Versions

```shell
System:
    python: 3.11.0rc1 (main, Aug 12 2022, 10:02:14) [GCC 11.2.0]
machine: Linux-6.5.0-44-generic-x86_64-with-glibc2.35

Python dependencies:
      sklearn: 1.4.1.post1
          pip: 24.1.2
   setuptools: 68.2.0
        numpy: 1.26.4
        scipy: 1.12.0
       Cython: None
       pandas: 2.2.1
   matplotlib: 3.8.3
       joblib: 1.3.2
threadpoolctl: 3.4.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /home/.../venv/lib/python3.11/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: Zen

       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libgomp
       filepath: /home/.../venv/lib/python3.11/site-packages/faiss_cpu.libs/libgomp-a34b3233.so.1.0.0
        version: None

       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /home/.../venv/lib/python3.11/site-packages/scipy.libs/libopenblasp-r0-23e5df77.3.21.dev.so
        version: 0.3.21.dev
threading_layer: pthreads
   architecture: Zen
```

COMMENT:
The problem can be that the priors cannot change the extreme predicted probabilities. In the case of `GaussianNB()`, I get
```
model.predict_proba([[5.2, 3.6, 1.5, 0.3]])
# array([[1.00000000e+00, 6.06941525e-17, 3.10259941e-24]])
```
This make `GaussianNB(priors=...)` useless.

Labels: Bug, Needs Triage
GaussianNB(priors=...) is useless

### Describe the bug

If I set the class priors to be very small for classes 0 and 2 and very large for class 1, I expect my predictions to be of class 1. However, I get class 0. It seems to be that `GaussianNB(priors=...)` is useless.

### Steps/Code to Reproduce

```python
from sklearn import datasets
from sklearn.naive_bayes import GaussianNB

# Load data
iris = datasets.load_iris()
features = iris.data
target = iris.target

# Create Gaussian naive Bayes object
classifer = GaussianNB()  # The prior is adjusted based on the data
# Train model
model = classifer.fit(features, target)
# Create new observation
new_observation = [[5.2, 3.6, 1.5, 0.3]]
# Predict class
model.predict(new_observation)
# array([0])

# Set prior probabilities p(y) of each class of 3 (does not work)
clf = GaussianNB(priors=[1e-12, 1-1e-11, 9e-12])
# Train model
model_priors = clf.fit(features, target)
# Create new observation
new_observation = [[5.2, 3.6, 1.5, 0.3]]
# Predict class
model_priors.predict(new_observation)
# array([0])
```

### Expected Results

- `array([0])` for `GaussianNB` without priors
- `array([1])` for `GaussianNB(priors=...)` with priors

### Actual Results

- `array([0])` for `GaussianNB` without priors
- `array([0])` for `GaussianNB(priors=...)` with priors

### Versions

```shell
System:
    python: 3.11.0rc1 (main, Aug 12 2022, 10:02:14) [GCC 11.2.0]
machine: Linux-6.5.0-44-generic-x86_64-with-glibc2.35

Python dependencies:
      sklearn: 1.4.1.post1
          pip: 24.1.2
   setuptools: 68.2.0
        numpy: 1.26.4
        scipy: 1.12.0
       Cython: None
       pandas: 2.2.1
   matplotlib: 3.8.3
       joblib: 1.3.2
threadpoolctl: 3.4.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /home/.../venv/lib/python3.11/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: Zen

       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libgomp
       filepath: /home/.../venv/lib/python3.11/site-packages/faiss_cpu.libs/libgomp-a34b3233.so.1.0.0
        version: None

       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /home/.../venv/lib/python3.11/site-packages/scipy.libs/libopenblasp-r0-23e5df77.3.21.dev.so
        version: 0.3.21.dev
threading_layer: pthreads
   architecture: Zen
```

COMMENT:
When working with log likelihoods, those small priors, however small, allow learning from the data, which is happening here. If you set your priors to absolute zero for the other classes, then you get 1 as the output as you expect.

Labels: Bug, Needs Triage