Talip7/scikit-learn-issues-multilabel huggingface dataset:
Add a version of `GenericUnivariateSelect`/`SelectPercentile`/`SelectKBest` that allows input of X with missing values and `y=None`.

### Describe the workflow you want to enable

- Select features by the percentage of missing values of X
- Select features only by statistical properties of X before y is available

### Describe your proposed solution

Create a version with weaker X, y checks.

### Describe alternatives you've considered, if relevant

1.
```python
class LooseGenericUnivariateSelect(GenericUnivariateSelect):
    def fit(self, X: Any, y: Any = None) -> Self:
        y = np.ones(X.shape[0])
        return super().fit(X, y)
    
    def _validate_data(self, X, y=None, *args, **kwargs):
        kwargs["force_all_finite"] = False
        kwargs.pop("multi_output", None)
        return super()._validate_data(X, y, *args, **kwargs)
    
    def _more_tags(self):
        return {"requires_y": False, "allow_nan": True}
```
Not sure if this code works properly.
2.
Bother to create an estimator that outputs the desired function in `feature_importance_` and use `SelectFromModel`

### Additional context

_No response_

COMMENT:
> Select features by the percentage of missing values of X

For example, if one wants to use both an estimator that allows NaNs and one that does not allow NaNs in a single complex `Pipeline`,  NaNs need to be removed at some point; in certain fields where feature imputation is not prefered, something like [feature_engine.DropMissingData]( https://feature-engine.trainindata.com/en/1.3.x/api_doc/imputation/DropMissingData.html#feature_engine.imputation.DropMissingData) can be used to remove rows that contain NaNs, but if for some reason a column is included where all values are NaN in X, all rows will be removed, so one may want to preliminarily remove some columns with such a transformer within a `Pipeline`.

> Select features only by statistical properties of X before y is available

This could be used when you want to use a `Pipeline` for feature transformation, and then experiment with multiple estimators. 
When reducing features by their monotonicity or periodicity, y is unnecessary.

Labels: New Feature
Add a version of `GenericUnivariateSelect`/`SelectPercentile`/`SelectKBest` that allows input of X with missing values and `y=None`.

### Describe the workflow you want to enable

- Select features by the percentage of missing values of X
- Select features only by statistical properties of X before y is available

### Describe your proposed solution

Create a version with weaker X, y checks.

### Describe alternatives you've considered, if relevant

1.
```python
class LooseGenericUnivariateSelect(GenericUnivariateSelect):
    def fit(self, X: Any, y: Any = None) -> Self:
        y = np.ones(X.shape[0])
        return super().fit(X, y)
    
    def _validate_data(self, X, y=None, *args, **kwargs):
        kwargs["force_all_finite"] = False
        kwargs.pop("multi_output", None)
        return super()._validate_data(X, y, *args, **kwargs)
    
    def _more_tags(self):
        return {"requires_y": False, "allow_nan": True}
```
Not sure if this code works properly.
2.
Bother to create an estimator that outputs the desired function in `feature_importance_` and use `SelectFromModel`

### Additional context

_No response_

COMMENT:
> Select features by the percentage of missing values of X

I am not sure that we will even advocate for dropping missing values. I am not convinced about this one.

Labels: New Feature
Add a version of `GenericUnivariateSelect`/`SelectPercentile`/`SelectKBest` that allows input of X with missing values and `y=None`.

### Describe the workflow you want to enable

- Select features by the percentage of missing values of X
- Select features only by statistical properties of X before y is available

### Describe your proposed solution

Create a version with weaker X, y checks.

### Describe alternatives you've considered, if relevant

1.
```python
class LooseGenericUnivariateSelect(GenericUnivariateSelect):
    def fit(self, X: Any, y: Any = None) -> Self:
        y = np.ones(X.shape[0])
        return super().fit(X, y)
    
    def _validate_data(self, X, y=None, *args, **kwargs):
        kwargs["force_all_finite"] = False
        kwargs.pop("multi_output", None)
        return super()._validate_data(X, y, *args, **kwargs)
    
    def _more_tags(self):
        return {"requires_y": False, "allow_nan": True}
```
Not sure if this code works properly.
2.
Bother to create an estimator that outputs the desired function in `feature_importance_` and use `SelectFromModel`

### Additional context

_No response_

COMMENT:
> Select features only by statistical properties of X before y is available

Nothing prevent you to provide a function that does not use internally the `y`, ins't it?

Labels: New Feature
Add a version of `GenericUnivariateSelect`/`SelectPercentile`/`SelectKBest` that allows input of X with missing values and `y=None`.

### Describe the workflow you want to enable

- Select features by the percentage of missing values of X
- Select features only by statistical properties of X before y is available

### Describe your proposed solution

Create a version with weaker X, y checks.

### Describe alternatives you've considered, if relevant

1.
```python
class LooseGenericUnivariateSelect(GenericUnivariateSelect):
    def fit(self, X: Any, y: Any = None) -> Self:
        y = np.ones(X.shape[0])
        return super().fit(X, y)
    
    def _validate_data(self, X, y=None, *args, **kwargs):
        kwargs["force_all_finite"] = False
        kwargs.pop("multi_output", None)
        return super()._validate_data(X, y, *args, **kwargs)
    
    def _more_tags(self):
        return {"requires_y": False, "allow_nan": True}
```
Not sure if this code works properly.
2.
Bother to create an estimator that outputs the desired function in `feature_importance_` and use `SelectFromModel`

### Additional context

_No response_

COMMENT:
Isn't it a matter for the function, not these estimators, to determine if y is necessary in the first place? I don't think it is a good idea to reduce the availability of estimators by making such unnecessary checks. This shouldn't be such a big change to refuse.

> > Select features only by statistical properties of X before y is available
> 
> Nothing prevent you to provide a function that does not use internally the `y`, ins't it?

This does not address the problem. Please read my statement again.

Labels: New Feature
Add a version of `GenericUnivariateSelect`/`SelectPercentile`/`SelectKBest` that allows input of X with missing values and `y=None`.

### Describe the workflow you want to enable

- Select features by the percentage of missing values of X
- Select features only by statistical properties of X before y is available

### Describe your proposed solution

Create a version with weaker X, y checks.

### Describe alternatives you've considered, if relevant

1.
```python
class LooseGenericUnivariateSelect(GenericUnivariateSelect):
    def fit(self, X: Any, y: Any = None) -> Self:
        y = np.ones(X.shape[0])
        return super().fit(X, y)
    
    def _validate_data(self, X, y=None, *args, **kwargs):
        kwargs["force_all_finite"] = False
        kwargs.pop("multi_output", None)
        return super()._validate_data(X, y, *args, **kwargs)
    
    def _more_tags(self):
        return {"requires_y": False, "allow_nan": True}
```
Not sure if this code works properly.
2.
Bother to create an estimator that outputs the desired function in `feature_importance_` and use `SelectFromModel`

### Additional context

_No response_

COMMENT:
> > Select features by the percentage of missing values of X
> 
> I am not sure that we will even advocate for dropping missing values. I am not convinced about this one.

When dealing with data that has a complex arrangement of missing values, the removal of missing values can be more difficult than one might imagine.

Labels: New Feature
Add a version of `GenericUnivariateSelect`/`SelectPercentile`/`SelectKBest` that allows input of X with missing values and `y=None`.

### Describe the workflow you want to enable

- Select features by the percentage of missing values of X
- Select features only by statistical properties of X before y is available

### Describe your proposed solution

Create a version with weaker X, y checks.

### Describe alternatives you've considered, if relevant

1.
```python
class LooseGenericUnivariateSelect(GenericUnivariateSelect):
    def fit(self, X: Any, y: Any = None) -> Self:
        y = np.ones(X.shape[0])
        return super().fit(X, y)
    
    def _validate_data(self, X, y=None, *args, **kwargs):
        kwargs["force_all_finite"] = False
        kwargs.pop("multi_output", None)
        return super()._validate_data(X, y, *args, **kwargs)
    
    def _more_tags(self):
        return {"requires_y": False, "allow_nan": True}
```
Not sure if this code works properly.
2.
Bother to create an estimator that outputs the desired function in `feature_importance_` and use `SelectFromModel`

### Additional context

_No response_

COMMENT:
> When dealing with data that has a complex arrangement of missing values, the removal of missing values can be more difficult than one might imagine.

To be more explicit, I think that we should leverage a bit more the following work in the documentation on the topic:

- https://proceedings.neurips.cc/paper/2021/file/5fe8fdc79ce292c39c5f209d734b7206-Paper.pdf
- https://academic.oup.com/gigascience/article/doi/10.1093/gigascience/giac013/6568998

In short, if you are using a strong learner, then the imputation techniques does not matter and leveraging the missing handling of random-forest or gradient boosting would be best. More advanced imputation techniques could be useful with simpler learner with a much higher cost.

Regarding dropping the missing values does not seem wise because in real use-case, the missingness is usually MNAR (missing not at random) and potentially have a predictive power for your final task.

Labels: New Feature
Add a version of `GenericUnivariateSelect`/`SelectPercentile`/`SelectKBest` that allows input of X with missing values and `y=None`.

### Describe the workflow you want to enable

- Select features by the percentage of missing values of X
- Select features only by statistical properties of X before y is available

### Describe your proposed solution

Create a version with weaker X, y checks.

### Describe alternatives you've considered, if relevant

1.
```python
class LooseGenericUnivariateSelect(GenericUnivariateSelect):
    def fit(self, X: Any, y: Any = None) -> Self:
        y = np.ones(X.shape[0])
        return super().fit(X, y)
    
    def _validate_data(self, X, y=None, *args, **kwargs):
        kwargs["force_all_finite"] = False
        kwargs.pop("multi_output", None)
        return super()._validate_data(X, y, *args, **kwargs)
    
    def _more_tags(self):
        return {"requires_y": False, "allow_nan": True}
```
Not sure if this code works properly.
2.
Bother to create an estimator that outputs the desired function in `feature_importance_` and use `SelectFromModel`

### Additional context

_No response_

COMMENT:
Something related to this feature request is the desired to drop constant feature as requested in `OneHotEncoder`: https://github.com/scikit-learn/scikit-learn/issues/27435

Making these selectors unsupervised allow to not to have to create a new estimator. I still think that it could be useful to get a `SelectThreshold` filter in this case.

I am going to implement those.

/take

Labels: New Feature
Add a version of `GenericUnivariateSelect`/`SelectPercentile`/`SelectKBest` that allows input of X with missing values and `y=None`.

### Describe the workflow you want to enable

- Select features by the percentage of missing values of X
- Select features only by statistical properties of X before y is available

### Describe your proposed solution

Create a version with weaker X, y checks.

### Describe alternatives you've considered, if relevant

1.
```python
class LooseGenericUnivariateSelect(GenericUnivariateSelect):
    def fit(self, X: Any, y: Any = None) -> Self:
        y = np.ones(X.shape[0])
        return super().fit(X, y)
    
    def _validate_data(self, X, y=None, *args, **kwargs):
        kwargs["force_all_finite"] = False
        kwargs.pop("multi_output", None)
        return super()._validate_data(X, y, *args, **kwargs)
    
    def _more_tags(self):
        return {"requires_y": False, "allow_nan": True}
```
Not sure if this code works properly.
2.
Bother to create an estimator that outputs the desired function in `feature_importance_` and use `SelectFromModel`

### Additional context

_No response_

COMMENT:
I open https://github.com/scikit-learn/scikit-learn/pull/27721

Labels: New Feature
Add a version of `GenericUnivariateSelect`/`SelectPercentile`/`SelectKBest` that allows input of X with missing values and `y=None`.

### Describe the workflow you want to enable

- Select features by the percentage of missing values of X
- Select features only by statistical properties of X before y is available

### Describe your proposed solution

Create a version with weaker X, y checks.

### Describe alternatives you've considered, if relevant

1.
```python
class LooseGenericUnivariateSelect(GenericUnivariateSelect):
    def fit(self, X: Any, y: Any = None) -> Self:
        y = np.ones(X.shape[0])
        return super().fit(X, y)
    
    def _validate_data(self, X, y=None, *args, **kwargs):
        kwargs["force_all_finite"] = False
        kwargs.pop("multi_output", None)
        return super()._validate_data(X, y, *args, **kwargs)
    
    def _more_tags(self):
        return {"requires_y": False, "allow_nan": True}
```
Not sure if this code works properly.
2.
Bother to create an estimator that outputs the desired function in `feature_importance_` and use `SelectFromModel`

### Additional context

_No response_

COMMENT:
I subsequently open https://github.com/scikit-learn/scikit-learn/pull/27722 that build upon #27721

Labels: New Feature
Missing assert in test_kernel_approximation.py

### Describe the bug

[scikit-learn/sklearn/tests/test\_kernel\_approximation.py at main · scikit-learn/scikit-learn](https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/tests/test_kernel_approximation.py#L144)

```python
@pytest.mark.parametrize("method", ["fit", "fit_transform", "transform"])
@pytest.mark.parametrize("sample_steps", range(1, 4))
def test_additive_chi2_sampler_sample_steps(method, sample_steps):
    """Check that the input sample step doesn't raise an error
    and that sample interval doesn't change after fit.
    """
    transformer = AdditiveChi2Sampler(sample_steps=sample_steps)
    getattr(transformer, method)(X)

    sample_interval = 0.5
    transformer = AdditiveChi2Sampler(
        sample_steps=sample_steps,
        sample_interval=sample_interval,
    )
    getattr(transformer, method)(X)
    transformer.sample_interval == sample_interval
```

The last line `transformer.sample_interval == sample_interval` should probably be `assert transformer.sample_interval == sample_interval`.


### Steps/Code to Reproduce

I observed this by statically analyzing the code, but I suspect the `test_additive_chi2_sampler_sample_steps` never fails.

### Expected Results

An assertion in the unittest.

### Actual Results

No assertion in the unittest

### Versions

```shell
main branch
```

COMMENT:
Indeed. @Sjord do you want to make a pull-request do add the `assert`.

Labels: Bug
Missing assert in test_kernel_approximation.py

### Describe the bug

[scikit-learn/sklearn/tests/test\_kernel\_approximation.py at main · scikit-learn/scikit-learn](https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/tests/test_kernel_approximation.py#L144)

```python
@pytest.mark.parametrize("method", ["fit", "fit_transform", "transform"])
@pytest.mark.parametrize("sample_steps", range(1, 4))
def test_additive_chi2_sampler_sample_steps(method, sample_steps):
    """Check that the input sample step doesn't raise an error
    and that sample interval doesn't change after fit.
    """
    transformer = AdditiveChi2Sampler(sample_steps=sample_steps)
    getattr(transformer, method)(X)

    sample_interval = 0.5
    transformer = AdditiveChi2Sampler(
        sample_steps=sample_steps,
        sample_interval=sample_interval,
    )
    getattr(transformer, method)(X)
    transformer.sample_interval == sample_interval
```

The last line `transformer.sample_interval == sample_interval` should probably be `assert transformer.sample_interval == sample_interval`.


### Steps/Code to Reproduce

I observed this by statically analyzing the code, but I suspect the `test_additive_chi2_sampler_sample_steps` never fails.

### Expected Results

An assertion in the unittest.

### Actual Results

No assertion in the unittest

### Versions

```shell
main branch
```

COMMENT:
Hi. If no one is picking this up, I'd like to work on this

Labels: Bug
Missing assert in test_kernel_approximation.py

### Describe the bug

[scikit-learn/sklearn/tests/test\_kernel\_approximation.py at main · scikit-learn/scikit-learn](https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/tests/test_kernel_approximation.py#L144)

```python
@pytest.mark.parametrize("method", ["fit", "fit_transform", "transform"])
@pytest.mark.parametrize("sample_steps", range(1, 4))
def test_additive_chi2_sampler_sample_steps(method, sample_steps):
    """Check that the input sample step doesn't raise an error
    and that sample interval doesn't change after fit.
    """
    transformer = AdditiveChi2Sampler(sample_steps=sample_steps)
    getattr(transformer, method)(X)

    sample_interval = 0.5
    transformer = AdditiveChi2Sampler(
        sample_steps=sample_steps,
        sample_interval=sample_interval,
    )
    getattr(transformer, method)(X)
    transformer.sample_interval == sample_interval
```

The last line `transformer.sample_interval == sample_interval` should probably be `assert transformer.sample_interval == sample_interval`.


### Steps/Code to Reproduce

I observed this by statically analyzing the code, but I suspect the `test_additive_chi2_sampler_sample_steps` never fails.

### Expected Results

An assertion in the unittest.

### Actual Results

No assertion in the unittest

### Versions

```shell
main branch
```

COMMENT:
@punndcoder28 you can go ahead

Labels: Bug
Deprecate murmurhash3_32

`sklearn.utils.murmurhash3_32` is part of our API, but we don't use it anywhere internally.
I propose to deprecate and finally remove it. The standard Python [`hash`](https://docs.python.org/3/library/functions.html#hash) function using SipHash per [PEP0456](https://peps.python.org/pep-0456/) might serve as a good replacement depending on the use case.

COMMENT:
While the Python function `sklearn.utils.murmurhash3_32` is not used in scikit-learn, the underlying Cython function `murmurhash3_bytes_s32` is used by our feature hasher named `HashingVectorizer`.

I think the purpose of the Python wrapper `sklearn.utils.murmurhash3_32` is to to be able to write non-regression unit tests.

I agree we could deprecate it to make it private, but I would still keep the Python wrapper (but private) for testing.

Labels: API, Breaking Change, cython
Deprecate murmurhash3_32

`sklearn.utils.murmurhash3_32` is part of our API, but we don't use it anywhere internally.
I propose to deprecate and finally remove it. The standard Python [`hash`](https://docs.python.org/3/library/functions.html#hash) function using SipHash per [PEP0456](https://peps.python.org/pep-0456/) might serve as a good replacement depending on the use case.

COMMENT:
For me an interesting question is if somewhere in Python there is a hash function that we can use for `HashingVectorizer`. Not having to have an "extra thing" (yes murmurhash is small, etc but still, less is less) would be attractive. Unfortunately I've not thought about hash functions in a few years so I don't know what the state of the art is. Last time I did investigate murmurhash (or cousins of it) were _the_ choice for applications where you wanted speed and did not need a cryptographically strong hash function. A downside of switching hash functions is that the results of `HashingVectorizer` would change (I guess), which means it becomes a bit of a faff to do in practice.

AFAIK the builtin `hash` of Python does "weird things" (`hash(42) == 4`) which makes it unsuitable for most applications. It is massively optimise/purpose made for use inside CPython.

I have no opinion on make private/keep as is for the Python function.

Labels: API, Breaking Change, cython
Deprecate murmurhash3_32

`sklearn.utils.murmurhash3_32` is part of our API, but we don't use it anywhere internally.
I propose to deprecate and finally remove it. The standard Python [`hash`](https://docs.python.org/3/library/functions.html#hash) function using SipHash per [PEP0456](https://peps.python.org/pep-0456/) might serve as a good replacement depending on the use case.

COMMENT:
At least with Python 3.10, I get `hash(42) == 42`, which is expected according to [hash](https://docs.python.org/3/library/functions.html#hash):
> Numeric values that compare equal have the same hash value

Labels: API, Breaking Change, cython
Deprecate murmurhash3_32

`sklearn.utils.murmurhash3_32` is part of our API, but we don't use it anywhere internally.
I propose to deprecate and finally remove it. The standard Python [`hash`](https://docs.python.org/3/library/functions.html#hash) function using SipHash per [PEP0456](https://peps.python.org/pep-0456/) might serve as a good replacement depending on the use case.

COMMENT:
Yes, it is expected that `hash(42) == 42`, I think its been like this for a long time. The point I was trying to make is that `hash` (which you might think is the most natural choice for "I need a hash function in my Python program") is a "hash function with many little tweaks that turn out to be useful if you use it to build the CPython interpreter, but less useful if you need a real hash function".

Labels: API, Breaking Change, cython
Deprecate murmurhash3_32

`sklearn.utils.murmurhash3_32` is part of our API, but we don't use it anywhere internally.
I propose to deprecate and finally remove it. The standard Python [`hash`](https://docs.python.org/3/library/functions.html#hash) function using SipHash per [PEP0456](https://peps.python.org/pep-0456/) might serve as a good replacement depending on the use case.

COMMENT:
Wouldn’t `hash` work for FeatureHasher?

Labels: API, Breaking Change, cython
Deprecate murmurhash3_32

`sklearn.utils.murmurhash3_32` is part of our API, but we don't use it anywhere internally.
I propose to deprecate and finally remove it. The standard Python [`hash`](https://docs.python.org/3/library/functions.html#hash) function using SipHash per [PEP0456](https://peps.python.org/pep-0456/) might serve as a good replacement depending on the use case.

COMMENT:
From googling a bit and trying to update my knowledge I realised that I can't find a good source describing how exactly `hash` works and that most likely my knowledge is out of date. Because for example things like https://peps.python.org/pep-0456/ have been adopted in the mean time.

---

I always believed that `hash` isn't good because (for some inputs) its output doesn't change a lot for similar inputs. `hash(42) + 1 == hash(43)`. Where for a "real" hash function you want there to be as much difference between the output of similar inputs as between totally different inputs.

Labels: API, Breaking Change, cython