Talip7/scikit-learn-issues-multilabel huggingface dataset:
NeighborhoodComponentsAnalysis (NCA) sets incorrect `_n_features_out` value which makes `.transform()` fail if `transform_output="pandas"`.

### Describe the bug

`NeighborhoodComponentsAnalysis.transform()` fails with the following error whenever `transform_output` is set to "pandas":
```python-traceback
ValueError: Shape of passed values is (100, 2), indices imply (100, 20)
```

### Steps/Code to Reproduce

```python
import numpy as np
from sklearn.datasets import make_classification
from sklearn.neighbors import NeighborhoodComponentsAnalysis

random_state = np.random.RandomState(42)
X, y = make_classification(random_state=random_state)
nca = NeighborhoodComponentsAnalysis(n_components=2, random_state=random_state)
nca.set_output(transform="pandas")
nca.fit_transform(X, y)
```

### Expected Results

`NeighborhoodComponentsAnalysis.transform()` is expected to return a dataframe with `n_components` columns.

### Actual Results

The error lies in this line here:

https://github.com/scikit-learn/scikit-learn/blob/cb836be0ff8347ccb0ab722760df68d07485101e/sklearn/neighbors/_nca.py#L326

since `.components_` has the shape `(n_components, n_features)`, so this line should be, for example

    self._n_features_out = self.components_.shape[0]

because `._n_features_out` ultimately needs to correspond to `n_components` (only then the correct number of column labels will be produced, that is 2 instead of 20).

Instead, the bug could be fixed by copying the pattern found in `PCA` where [this is a property](https://github.com/scikit-learn/scikit-learn/blob/cb836be0ff8347ccb0ab722760df68d07485101e/sklearn/decomposition/_base.py#L190C5-L193C41) which would replace the faulty line of code:

    @property
    def _n_features_out(self):
        """Number of transformed output features."""
        return self.components_.shape[0]

### Versions

```shell
sklearn: 1.4.0
```

COMMENT:
It would be straightforward for me to change the number but I have never contributed to this project: at this point in time, unfortunately, I cannot afford the overhead of familiarizing myself with community guidelines, find out how tests are run etc. Sorry for that.

Labels: Bug
NeighborhoodComponentsAnalysis (NCA) sets incorrect `_n_features_out` value which makes `.transform()` fail if `transform_output="pandas"`.

### Describe the bug

`NeighborhoodComponentsAnalysis.transform()` fails with the following error whenever `transform_output` is set to "pandas":
```python-traceback
ValueError: Shape of passed values is (100, 2), indices imply (100, 20)
```

### Steps/Code to Reproduce

```python
import numpy as np
from sklearn.datasets import make_classification
from sklearn.neighbors import NeighborhoodComponentsAnalysis

random_state = np.random.RandomState(42)
X, y = make_classification(random_state=random_state)
nca = NeighborhoodComponentsAnalysis(n_components=2, random_state=random_state)
nca.set_output(transform="pandas")
nca.fit_transform(X, y)
```

### Expected Results

`NeighborhoodComponentsAnalysis.transform()` is expected to return a dataframe with `n_components` columns.

### Actual Results

The error lies in this line here:

https://github.com/scikit-learn/scikit-learn/blob/cb836be0ff8347ccb0ab722760df68d07485101e/sklearn/neighbors/_nca.py#L326

since `.components_` has the shape `(n_components, n_features)`, so this line should be, for example

    self._n_features_out = self.components_.shape[0]

because `._n_features_out` ultimately needs to correspond to `n_components` (only then the correct number of column labels will be produced, that is 2 instead of 20).

Instead, the bug could be fixed by copying the pattern found in `PCA` where [this is a property](https://github.com/scikit-learn/scikit-learn/blob/cb836be0ff8347ccb0ab722760df68d07485101e/sklearn/decomposition/_base.py#L190C5-L193C41) which would replace the faulty line of code:

    @property
    def _n_features_out(self):
        """Number of transformed output features."""
        return self.components_.shape[0]

### Versions

```shell
sklearn: 1.4.0
```

COMMENT:
@glemaitre Do I can look into it?

Labels: Bug
NeighborhoodComponentsAnalysis (NCA) sets incorrect `_n_features_out` value which makes `.transform()` fail if `transform_output="pandas"`.

### Describe the bug

`NeighborhoodComponentsAnalysis.transform()` fails with the following error whenever `transform_output` is set to "pandas":
```python-traceback
ValueError: Shape of passed values is (100, 2), indices imply (100, 20)
```

### Steps/Code to Reproduce

```python
import numpy as np
from sklearn.datasets import make_classification
from sklearn.neighbors import NeighborhoodComponentsAnalysis

random_state = np.random.RandomState(42)
X, y = make_classification(random_state=random_state)
nca = NeighborhoodComponentsAnalysis(n_components=2, random_state=random_state)
nca.set_output(transform="pandas")
nca.fit_transform(X, y)
```

### Expected Results

`NeighborhoodComponentsAnalysis.transform()` is expected to return a dataframe with `n_components` columns.

### Actual Results

The error lies in this line here:

https://github.com/scikit-learn/scikit-learn/blob/cb836be0ff8347ccb0ab722760df68d07485101e/sklearn/neighbors/_nca.py#L326

since `.components_` has the shape `(n_components, n_features)`, so this line should be, for example

    self._n_features_out = self.components_.shape[0]

because `._n_features_out` ultimately needs to correspond to `n_components` (only then the correct number of column labels will be produced, that is 2 instead of 20).

Instead, the bug could be fixed by copying the pattern found in `PCA` where [this is a property](https://github.com/scikit-learn/scikit-learn/blob/cb836be0ff8347ccb0ab722760df68d07485101e/sklearn/decomposition/_base.py#L190C5-L193C41) which would replace the faulty line of code:

    @property
    def _n_features_out(self):
        """Number of transformed output features."""
        return self.components_.shape[0]

### Versions

```shell
sklearn: 1.4.0
```

COMMENT:
@ictorv go for it.

Labels: Bug
NeighborhoodComponentsAnalysis (NCA) sets incorrect `_n_features_out` value which makes `.transform()` fail if `transform_output="pandas"`.

### Describe the bug

`NeighborhoodComponentsAnalysis.transform()` fails with the following error whenever `transform_output` is set to "pandas":
```python-traceback
ValueError: Shape of passed values is (100, 2), indices imply (100, 20)
```

### Steps/Code to Reproduce

```python
import numpy as np
from sklearn.datasets import make_classification
from sklearn.neighbors import NeighborhoodComponentsAnalysis

random_state = np.random.RandomState(42)
X, y = make_classification(random_state=random_state)
nca = NeighborhoodComponentsAnalysis(n_components=2, random_state=random_state)
nca.set_output(transform="pandas")
nca.fit_transform(X, y)
```

### Expected Results

`NeighborhoodComponentsAnalysis.transform()` is expected to return a dataframe with `n_components` columns.

### Actual Results

The error lies in this line here:

https://github.com/scikit-learn/scikit-learn/blob/cb836be0ff8347ccb0ab722760df68d07485101e/sklearn/neighbors/_nca.py#L326

since `.components_` has the shape `(n_components, n_features)`, so this line should be, for example

    self._n_features_out = self.components_.shape[0]

because `._n_features_out` ultimately needs to correspond to `n_components` (only then the correct number of column labels will be produced, that is 2 instead of 20).

Instead, the bug could be fixed by copying the pattern found in `PCA` where [this is a property](https://github.com/scikit-learn/scikit-learn/blob/cb836be0ff8347ccb0ab722760df68d07485101e/sklearn/decomposition/_base.py#L190C5-L193C41) which would replace the faulty line of code:

    @property
    def _n_features_out(self):
        """Number of transformed output features."""
        return self.components_.shape[0]

### Versions

```shell
sklearn: 1.4.0
```

COMMENT:
@ictorv Sorry, I hope I have not doubled up on work. Did not see the comment here - just opened PR with some rough changes.

Labels: Bug
NeighborhoodComponentsAnalysis (NCA) sets incorrect `_n_features_out` value which makes `.transform()` fail if `transform_output="pandas"`.

### Describe the bug

`NeighborhoodComponentsAnalysis.transform()` fails with the following error whenever `transform_output` is set to "pandas":
```python-traceback
ValueError: Shape of passed values is (100, 2), indices imply (100, 20)
```

### Steps/Code to Reproduce

```python
import numpy as np
from sklearn.datasets import make_classification
from sklearn.neighbors import NeighborhoodComponentsAnalysis

random_state = np.random.RandomState(42)
X, y = make_classification(random_state=random_state)
nca = NeighborhoodComponentsAnalysis(n_components=2, random_state=random_state)
nca.set_output(transform="pandas")
nca.fit_transform(X, y)
```

### Expected Results

`NeighborhoodComponentsAnalysis.transform()` is expected to return a dataframe with `n_components` columns.

### Actual Results

The error lies in this line here:

https://github.com/scikit-learn/scikit-learn/blob/cb836be0ff8347ccb0ab722760df68d07485101e/sklearn/neighbors/_nca.py#L326

since `.components_` has the shape `(n_components, n_features)`, so this line should be, for example

    self._n_features_out = self.components_.shape[0]

because `._n_features_out` ultimately needs to correspond to `n_components` (only then the correct number of column labels will be produced, that is 2 instead of 20).

Instead, the bug could be fixed by copying the pattern found in `PCA` where [this is a property](https://github.com/scikit-learn/scikit-learn/blob/cb836be0ff8347ccb0ab722760df68d07485101e/sklearn/decomposition/_base.py#L190C5-L193C41) which would replace the faulty line of code:

    @property
    def _n_features_out(self):
        """Number of transformed output features."""
        return self.components_.shape[0]

### Versions

```shell
sklearn: 1.4.0
```

COMMENT:
@brendanlu it's okay, I was going thorough it. Can you explain what changes you made and what is status now?

Labels: Bug
NeighborhoodComponentsAnalysis (NCA) sets incorrect `_n_features_out` value which makes `.transform()` fail if `transform_output="pandas"`.

### Describe the bug

`NeighborhoodComponentsAnalysis.transform()` fails with the following error whenever `transform_output` is set to "pandas":
```python-traceback
ValueError: Shape of passed values is (100, 2), indices imply (100, 20)
```

### Steps/Code to Reproduce

```python
import numpy as np
from sklearn.datasets import make_classification
from sklearn.neighbors import NeighborhoodComponentsAnalysis

random_state = np.random.RandomState(42)
X, y = make_classification(random_state=random_state)
nca = NeighborhoodComponentsAnalysis(n_components=2, random_state=random_state)
nca.set_output(transform="pandas")
nca.fit_transform(X, y)
```

### Expected Results

`NeighborhoodComponentsAnalysis.transform()` is expected to return a dataframe with `n_components` columns.

### Actual Results

The error lies in this line here:

https://github.com/scikit-learn/scikit-learn/blob/cb836be0ff8347ccb0ab722760df68d07485101e/sklearn/neighbors/_nca.py#L326

since `.components_` has the shape `(n_components, n_features)`, so this line should be, for example

    self._n_features_out = self.components_.shape[0]

because `._n_features_out` ultimately needs to correspond to `n_components` (only then the correct number of column labels will be produced, that is 2 instead of 20).

Instead, the bug could be fixed by copying the pattern found in `PCA` where [this is a property](https://github.com/scikit-learn/scikit-learn/blob/cb836be0ff8347ccb0ab722760df68d07485101e/sklearn/decomposition/_base.py#L190C5-L193C41) which would replace the faulty line of code:

    @property
    def _n_features_out(self):
        """Number of transformed output features."""
        return self.components_.shape[0]

### Versions

```shell
sklearn: 1.4.0
```

COMMENT:
@ictorv I copied the pattern found in the base pca class for defining `_n_features_out`, and adjusted the test file slightly. I think this bug slipped through the existing unit test because `n_features == n_components` when `n_features` is not set in the estimator, so I slightly adjusted it to ensure it would catch this underlying issue.

Labels: Bug
NeighborhoodComponentsAnalysis (NCA) sets incorrect `_n_features_out` value which makes `.transform()` fail if `transform_output="pandas"`.

### Describe the bug

`NeighborhoodComponentsAnalysis.transform()` fails with the following error whenever `transform_output` is set to "pandas":
```python-traceback
ValueError: Shape of passed values is (100, 2), indices imply (100, 20)
```

### Steps/Code to Reproduce

```python
import numpy as np
from sklearn.datasets import make_classification
from sklearn.neighbors import NeighborhoodComponentsAnalysis

random_state = np.random.RandomState(42)
X, y = make_classification(random_state=random_state)
nca = NeighborhoodComponentsAnalysis(n_components=2, random_state=random_state)
nca.set_output(transform="pandas")
nca.fit_transform(X, y)
```

### Expected Results

`NeighborhoodComponentsAnalysis.transform()` is expected to return a dataframe with `n_components` columns.

### Actual Results

The error lies in this line here:

https://github.com/scikit-learn/scikit-learn/blob/cb836be0ff8347ccb0ab722760df68d07485101e/sklearn/neighbors/_nca.py#L326

since `.components_` has the shape `(n_components, n_features)`, so this line should be, for example

    self._n_features_out = self.components_.shape[0]

because `._n_features_out` ultimately needs to correspond to `n_components` (only then the correct number of column labels will be produced, that is 2 instead of 20).

Instead, the bug could be fixed by copying the pattern found in `PCA` where [this is a property](https://github.com/scikit-learn/scikit-learn/blob/cb836be0ff8347ccb0ab722760df68d07485101e/sklearn/decomposition/_base.py#L190C5-L193C41) which would replace the faulty line of code:

    @property
    def _n_features_out(self):
        """Number of transformed output features."""
        return self.components_.shape[0]

### Versions

```shell
sklearn: 1.4.0
```

COMMENT:
@glemaitre PR linked here ready for review if you get a moment sometime.

Labels: Bug
Tests failing when cuda installed but no GPU is present

after doing `conda install pytorch cupy`, my tests fail with:

```
FAILED sklearn/metrics/tests/test_common.py::test_array_api_compliance[
accuracy_score-check_array_api_binary_classification_metric-cupy-None-None] 
- cupy_backends.cuda.api.runtime.CUDARuntimeError: cudaErrorNoDevice: 
- no CUDA-capable device is detected
```

I don't think tests should ever fail for this, should they?

cc @ogrisel @betatim

COMMENT:
Indeed we can tried to allocate a test torch / cupy array on the device and catch any such low level exception before yielding this test configuration in `yield_namespace_device_dtype_combinations`.

Labels: Bug
Tests failing when cuda installed but no GPU is present

after doing `conda install pytorch cupy`, my tests fail with:

```
FAILED sklearn/metrics/tests/test_common.py::test_array_api_compliance[
accuracy_score-check_array_api_binary_classification_metric-cupy-None-None] 
- cupy_backends.cuda.api.runtime.CUDARuntimeError: cudaErrorNoDevice: 
- no CUDA-capable device is detected
```

I don't think tests should ever fail for this, should they?

cc @ogrisel @betatim

COMMENT:
I agree we should have a nicer error/find a way not to error.

---

For my education: why is cupy installed if the system doesn't have a GPU? Cupy is special compared to pytorch and such in that it does not support CPU execution.

Labels: Bug
Tests failing when cuda installed but no GPU is present

after doing `conda install pytorch cupy`, my tests fail with:

```
FAILED sklearn/metrics/tests/test_common.py::test_array_api_compliance[
accuracy_score-check_array_api_binary_classification_metric-cupy-None-None] 
- cupy_backends.cuda.api.runtime.CUDARuntimeError: cudaErrorNoDevice: 
- no CUDA-capable device is detected
```

I don't think tests should ever fail for this, should they?

cc @ogrisel @betatim

COMMENT:
This happens on laptops where they have two GPUs, one integrated and one nvidia dedicated. You have the device, so it's detected, the drivers are installed, but the devices is disabled (due to power consumption).

Labels: Bug
Tests failing when cuda installed but no GPU is present

after doing `conda install pytorch cupy`, my tests fail with:

```
FAILED sklearn/metrics/tests/test_common.py::test_array_api_compliance[
accuracy_score-check_array_api_binary_classification_metric-cupy-None-None] 
- cupy_backends.cuda.api.runtime.CUDARuntimeError: cudaErrorNoDevice: 
- no CUDA-capable device is detected
```

I don't think tests should ever fail for this, should they?

cc @ogrisel @betatim

COMMENT:
One issue is that in this setting, cuda fails to detect unavailability of the card in a reasonably fast manner: https://github.com/cupy/cupy/issues/8152

Labels: Bug
Trigger lockfile update with a comment

I've never been able to run the script to update the lock files w/o errors. My last attempt resulted in https://github.com/scikit-learn/scikit-learn/pull/28258#issuecomment-1910627538 which also didn't work, and that's after I had to install conda on my env, which I don't usually have since I use micro mamba.

It would be nice to be able to trigger a bot to update lock files the same way you can re-render on conda-forge. A comment with value similar to `@scikit-learn-bot please re-render` could update the lock files on the PR.

WDYT @lesteve ?

COMMENT:
Yeah I agree, it would be nice in a PR kind of like conda-forge please rerender to put a comment saying "please update lock-file". Or maybe something even more precise, "please update main-ci lock-files", "please update scipy-dev lock-files" i.e. the tag that were recently added to the update environment and lock file script.

You would also need to think about adding markers in the commit message like `[doc build]` `[scipy-dev]` in some cases see https://github.com/scikit-learn/scikit-learn/blob/main/.github/workflows/update-lock-files.yml

Labels: Build / CI
Trigger lockfile update with a comment

I've never been able to run the script to update the lock files w/o errors. My last attempt resulted in https://github.com/scikit-learn/scikit-learn/pull/28258#issuecomment-1910627538 which also didn't work, and that's after I had to install conda on my env, which I don't usually have since I use micro mamba.

It would be nice to be able to trigger a bot to update lock files the same way you can re-render on conda-forge. A comment with value similar to `@scikit-learn-bot please re-render` could update the lock files on the PR.

WDYT @lesteve ?

COMMENT:
@lesteve now that we have the bot infrastructure, do you think implementing this is easier?

Labels: Build / CI
Trigger lockfile update with a comment

I've never been able to run the script to update the lock files w/o errors. My last attempt resulted in https://github.com/scikit-learn/scikit-learn/pull/28258#issuecomment-1910627538 which also didn't work, and that's after I had to install conda on my env, which I don't usually have since I use micro mamba.

It would be nice to be able to trigger a bot to update lock files the same way you can re-render on conda-forge. A comment with value similar to `@scikit-learn-bot please re-render` could update the lock files on the PR.

WDYT @lesteve ?

COMMENT:
This is probably doable although I don't think much has changed since this issue was created (January 2024), so not sure what makes you think this would be easier now ...

Full-disclosure: moving this forward is unlikely to be high enough in my personal priority list but if someone finds this is a fun project to work on, I am willing to review the PR :wink:.

Labels: Build / CI
Trigger lockfile update with a comment

I've never been able to run the script to update the lock files w/o errors. My last attempt resulted in https://github.com/scikit-learn/scikit-learn/pull/28258#issuecomment-1910627538 which also didn't work, and that's after I had to install conda on my env, which I don't usually have since I use micro mamba.

It would be nice to be able to trigger a bot to update lock files the same way you can re-render on conda-forge. A comment with value similar to `@scikit-learn-bot please re-render` could update the lock files on the PR.

WDYT @lesteve ?

COMMENT:
@adrinjalali @lesteve I opened #29505 to give this a try, though I'm not sure if it's the correct way (since I'm no expert in this). My Windows machine always fails with stupid reasons running the lock file update script which pushed me to take a look at this 🤣 Would you like to give a review?

Labels: Build / CI
Trigger lockfile update with a comment

I've never been able to run the script to update the lock files w/o errors. My last attempt resulted in https://github.com/scikit-learn/scikit-learn/pull/28258#issuecomment-1910627538 which also didn't work, and that's after I had to install conda on my env, which I don't usually have since I use micro mamba.

It would be nice to be able to trigger a bot to update lock files the same way you can re-render on conda-forge. A comment with value similar to `@scikit-learn-bot please re-render` could update the lock files on the PR.

WDYT @lesteve ?

COMMENT:
Congrats for taking the plunge, and good luck with this! I'll try to take a look at #29505, the first recommendation I have for this kind of CI work is to test it on your fork or on a separate scikit-learn-like repo where you can iterate faster like Tim did for his CUDA CI work: https://github.com/betatim/scikit-learn-ci.

Out of curiosity could you open a separate issue with the kind of errors you get when trying to update the lock-files on Windows. Some of it may be improvable you know :wink:. Off the top of my head, it could be linked to limitations of conda-lock on Windows, something that the update script does not deal with on Windows, or plenty of other reasons.

Labels: Build / CI