Talip7/scikit-learn-issues-multilabel huggingface dataset:
Add an option to OrdinalEncoder to sort encoding by decreasing frequencies

### Describe the workflow you want to enable

At the moment, when no categories are provided, the default ordering of categories for a given categorical column is based on the lexicographical ordering of the categories observed in the training set.

However, this is quite arbitrary and one could instead provide an option to encode such values based on their observed frequency in the training set.

The motivation would be to nudge the inductive bias of tree-based models (and models that favor axis aligned decision functions) into separating nominal from rare values more easily (e.g. fewer splits in a tree). This might be especially useful for outlier detection models such as `IsolationForest`.

Related to #15796.

### Describe your proposed solution


Extend the `categories` option to have:

- `categories="lexigraphical"` (default for backward compat);
- `categories=user_provided_list` (same as today);
- `categories="frequency"` (the new option).

If `categories="frequency"`, then the generated category encodings would be:

- 0 would encode the most frequent category observed in the training set,
- 1 the second most frequent,
- ...
- and so on until the least frequent categories.

Tie breaking could be based on the lexicographical order to ensure that the behavior of `OrdinalEncoder` remains invariant under a shuffling of the rows of the training set.

EDIT: we could name this option `"decreasing_frequency"` instead of `"frequency"` to be more explicit that values closest to 0 mean higher frequency categories. And this would allow to also add `"increasing_frequency"` if we want, but I am not sure if it would be needed/useful to have both options.

### Describe alternatives you've considered, if relevant

- Use `TargetEncoder` that leverages some frequency info (mixed with the mean value of the target variable), but this is a supervised method and would therefore not be suitable for unsupervised anomaly detection tasks, for instance.

- Introduce a new dedicated class, e.g. `FrequencyEncoder`.
  - pro: allow using the observed relative frequency (between 0 and 1) as value (but would collapse equally frequent categories into the same numerical value).
  - con: introduces yet another estimator class in our public API.

### Additional context

_No response_

COMMENT:
Note to contributors: please wait for maintainers to reach a consensus on whether we want this new feature (I do) and how to implement it (if we decide its inclusion) before asking for guidance or contributing a PR.

Labels: New Feature, Needs Decision - Include Feature
Add an option to OrdinalEncoder to sort encoding by decreasing frequencies

### Describe the workflow you want to enable

At the moment, when no categories are provided, the default ordering of categories for a given categorical column is based on the lexicographical ordering of the categories observed in the training set.

However, this is quite arbitrary and one could instead provide an option to encode such values based on their observed frequency in the training set.

The motivation would be to nudge the inductive bias of tree-based models (and models that favor axis aligned decision functions) into separating nominal from rare values more easily (e.g. fewer splits in a tree). This might be especially useful for outlier detection models such as `IsolationForest`.

Related to #15796.

### Describe your proposed solution


Extend the `categories` option to have:

- `categories="lexigraphical"` (default for backward compat);
- `categories=user_provided_list` (same as today);
- `categories="frequency"` (the new option).

If `categories="frequency"`, then the generated category encodings would be:

- 0 would encode the most frequent category observed in the training set,
- 1 the second most frequent,
- ...
- and so on until the least frequent categories.

Tie breaking could be based on the lexicographical order to ensure that the behavior of `OrdinalEncoder` remains invariant under a shuffling of the rows of the training set.

EDIT: we could name this option `"decreasing_frequency"` instead of `"frequency"` to be more explicit that values closest to 0 mean higher frequency categories. And this would allow to also add `"increasing_frequency"` if we want, but I am not sure if it would be needed/useful to have both options.

### Describe alternatives you've considered, if relevant

- Use `TargetEncoder` that leverages some frequency info (mixed with the mean value of the target variable), but this is a supervised method and would therefore not be suitable for unsupervised anomaly detection tasks, for instance.

- Introduce a new dedicated class, e.g. `FrequencyEncoder`.
  - pro: allow using the observed relative frequency (between 0 and 1) as value (but would collapse equally frequent categories into the same numerical value).
  - con: introduces yet another estimator class in our public API.

### Additional context

_No response_

COMMENT:
would like to work on this if this goes you decide to go ahead on this.

Labels: New Feature, Needs Decision - Include Feature
Add an option to OrdinalEncoder to sort encoding by decreasing frequencies

### Describe the workflow you want to enable

At the moment, when no categories are provided, the default ordering of categories for a given categorical column is based on the lexicographical ordering of the categories observed in the training set.

However, this is quite arbitrary and one could instead provide an option to encode such values based on their observed frequency in the training set.

The motivation would be to nudge the inductive bias of tree-based models (and models that favor axis aligned decision functions) into separating nominal from rare values more easily (e.g. fewer splits in a tree). This might be especially useful for outlier detection models such as `IsolationForest`.

Related to #15796.

### Describe your proposed solution


Extend the `categories` option to have:

- `categories="lexigraphical"` (default for backward compat);
- `categories=user_provided_list` (same as today);
- `categories="frequency"` (the new option).

If `categories="frequency"`, then the generated category encodings would be:

- 0 would encode the most frequent category observed in the training set,
- 1 the second most frequent,
- ...
- and so on until the least frequent categories.

Tie breaking could be based on the lexicographical order to ensure that the behavior of `OrdinalEncoder` remains invariant under a shuffling of the rows of the training set.

EDIT: we could name this option `"decreasing_frequency"` instead of `"frequency"` to be more explicit that values closest to 0 mean higher frequency categories. And this would allow to also add `"increasing_frequency"` if we want, but I am not sure if it would be needed/useful to have both options.

### Describe alternatives you've considered, if relevant

- Use `TargetEncoder` that leverages some frequency info (mixed with the mean value of the target variable), but this is a supervised method and would therefore not be suitable for unsupervised anomaly detection tasks, for instance.

- Introduce a new dedicated class, e.g. `FrequencyEncoder`.
  - pro: allow using the observed relative frequency (between 0 and 1) as value (but would collapse equally frequent categories into the same numerical value).
  - con: introduces yet another estimator class in our public API.

### Additional context

_No response_

COMMENT:
This looks like a great addition! I don't think I ever used the `"auto"` option TBH.

I was thinking if these options could be used with #32179 in the future. That would allow combining explicit categories for some columns with frequency-based (or other) ordering for other columns in the same encoder.

Here is an example:
 
```python
enc = OrdinalEncoder(categories={
    "color": "frequency",
    "size": ["S", "M", "L"],
})
```

Labels: New Feature, Needs Decision - Include Feature
Add an option to OrdinalEncoder to sort encoding by decreasing frequencies

### Describe the workflow you want to enable

At the moment, when no categories are provided, the default ordering of categories for a given categorical column is based on the lexicographical ordering of the categories observed in the training set.

However, this is quite arbitrary and one could instead provide an option to encode such values based on their observed frequency in the training set.

The motivation would be to nudge the inductive bias of tree-based models (and models that favor axis aligned decision functions) into separating nominal from rare values more easily (e.g. fewer splits in a tree). This might be especially useful for outlier detection models such as `IsolationForest`.

Related to #15796.

### Describe your proposed solution


Extend the `categories` option to have:

- `categories="lexigraphical"` (default for backward compat);
- `categories=user_provided_list` (same as today);
- `categories="frequency"` (the new option).

If `categories="frequency"`, then the generated category encodings would be:

- 0 would encode the most frequent category observed in the training set,
- 1 the second most frequent,
- ...
- and so on until the least frequent categories.

Tie breaking could be based on the lexicographical order to ensure that the behavior of `OrdinalEncoder` remains invariant under a shuffling of the rows of the training set.

EDIT: we could name this option `"decreasing_frequency"` instead of `"frequency"` to be more explicit that values closest to 0 mean higher frequency categories. And this would allow to also add `"increasing_frequency"` if we want, but I am not sure if it would be needed/useful to have both options.

### Describe alternatives you've considered, if relevant

- Use `TargetEncoder` that leverages some frequency info (mixed with the mean value of the target variable), but this is a supervised method and would therefore not be suitable for unsupervised anomaly detection tasks, for instance.

- Introduce a new dedicated class, e.g. `FrequencyEncoder`.
  - pro: allow using the observed relative frequency (between 0 and 1) as value (but would collapse equally frequent categories into the same numerical value).
  - con: introduces yet another estimator class in our public API.

### Additional context

_No response_

COMMENT:
Anyone working on this feature else i like to work on this

Labels: New Feature, Needs Decision - Include Feature
Add an option to OrdinalEncoder to sort encoding by decreasing frequencies

### Describe the workflow you want to enable

At the moment, when no categories are provided, the default ordering of categories for a given categorical column is based on the lexicographical ordering of the categories observed in the training set.

However, this is quite arbitrary and one could instead provide an option to encode such values based on their observed frequency in the training set.

The motivation would be to nudge the inductive bias of tree-based models (and models that favor axis aligned decision functions) into separating nominal from rare values more easily (e.g. fewer splits in a tree). This might be especially useful for outlier detection models such as `IsolationForest`.

Related to #15796.

### Describe your proposed solution


Extend the `categories` option to have:

- `categories="lexigraphical"` (default for backward compat);
- `categories=user_provided_list` (same as today);
- `categories="frequency"` (the new option).

If `categories="frequency"`, then the generated category encodings would be:

- 0 would encode the most frequent category observed in the training set,
- 1 the second most frequent,
- ...
- and so on until the least frequent categories.

Tie breaking could be based on the lexicographical order to ensure that the behavior of `OrdinalEncoder` remains invariant under a shuffling of the rows of the training set.

EDIT: we could name this option `"decreasing_frequency"` instead of `"frequency"` to be more explicit that values closest to 0 mean higher frequency categories. And this would allow to also add `"increasing_frequency"` if we want, but I am not sure if it would be needed/useful to have both options.

### Describe alternatives you've considered, if relevant

- Use `TargetEncoder` that leverages some frequency info (mixed with the mean value of the target variable), but this is a supervised method and would therefore not be suitable for unsupervised anomaly detection tasks, for instance.

- Introduce a new dedicated class, e.g. `FrequencyEncoder`.
  - pro: allow using the observed relative frequency (between 0 and 1) as value (but would collapse equally frequent categories into the same numerical value).
  - con: introduces yet another estimator class in our public API.

### Additional context

_No response_

COMMENT:
I would like too do it !

Labels: New Feature, Needs Decision - Include Feature
Add an option to OrdinalEncoder to sort encoding by decreasing frequencies

### Describe the workflow you want to enable

At the moment, when no categories are provided, the default ordering of categories for a given categorical column is based on the lexicographical ordering of the categories observed in the training set.

However, this is quite arbitrary and one could instead provide an option to encode such values based on their observed frequency in the training set.

The motivation would be to nudge the inductive bias of tree-based models (and models that favor axis aligned decision functions) into separating nominal from rare values more easily (e.g. fewer splits in a tree). This might be especially useful for outlier detection models such as `IsolationForest`.

Related to #15796.

### Describe your proposed solution


Extend the `categories` option to have:

- `categories="lexigraphical"` (default for backward compat);
- `categories=user_provided_list` (same as today);
- `categories="frequency"` (the new option).

If `categories="frequency"`, then the generated category encodings would be:

- 0 would encode the most frequent category observed in the training set,
- 1 the second most frequent,
- ...
- and so on until the least frequent categories.

Tie breaking could be based on the lexicographical order to ensure that the behavior of `OrdinalEncoder` remains invariant under a shuffling of the rows of the training set.

EDIT: we could name this option `"decreasing_frequency"` instead of `"frequency"` to be more explicit that values closest to 0 mean higher frequency categories. And this would allow to also add `"increasing_frequency"` if we want, but I am not sure if it would be needed/useful to have both options.

### Describe alternatives you've considered, if relevant

- Use `TargetEncoder` that leverages some frequency info (mixed with the mean value of the target variable), but this is a supervised method and would therefore not be suitable for unsupervised anomaly detection tasks, for instance.

- Introduce a new dedicated class, e.g. `FrequencyEncoder`.
  - pro: allow using the observed relative frequency (between 0 and 1) as value (but would collapse equally frequent categories into the same numerical value).
  - con: introduces yet another estimator class in our public API.

### Additional context

_No response_

COMMENT:
If you give a thumbs up, I can start working on it

Labels: New Feature, Needs Decision - Include Feature
GridSearchCV cannot obtain the optimal parameter results.

### Describe the bug

When I used GridSearchCV to obtain the optimal parameters, I found that different cross-validation folds produced inconsistent results.

### Steps/Code to Reproduce

```python3
X_train, X_test, y_train, y_test = train_test_split(x, y, test_size=0.7, random_state=0)
param_grid_rf = {'n_estimators': [5, 10, 15, 35, 50, 70]}
grid_search = GridSearchCV(RandomForestClassifier(random_state=42), param_grid_rf, cv=3, scoring='roc_auc')
grid_search.fit(X_train, y_train)
best_params = grid_search.best_params_
print(f'{roc_auc_score(y_test, y_proba):.4f}')
```
> auc: 0.8859
best_params is 50

```python3
X_train, X_test, y_train, y_test = train_test_split(x, y, test_size=0.7, random_state=0)
param_grid_rf = {'n_estimators': [50, 70, 80, 90]}
grid_search = GridSearchCV(RandomForestClassifier(random_state=42), param_grid_rf, cv=3, scoring='roc_auc')
grid_search.fit(X_train, y_train)
best_params = grid_search.best_params_
print(f'{roc_auc_score(y_test, y_proba):.4f}')
```
> auc: 0.8725
best_params is 80

### Expected Results

Both results above should be identical.
>auc: 0.8859
best_params is 50

### Actual Results

Between the two parameter lists, we should obtain the parameters with the best AUC, not inconsistent ones.

### Versions

```shell
System:
    python: 3.10.12 (main, May 27 2025, 17:12:29) [GCC 11.4.0]
executable: /work/users/suny/mm/.venv/bin/python
   machine: Linux-5.15.0-143-generic-x86_64-with-glibc2.35

Python dependencies:
      sklearn: 1.7.2
          pip: 25.2
   setuptools: 80.9.0
        numpy: 2.2.6
        scipy: 1.15.3
       Cython: None
       pandas: 2.3.2
   matplotlib: 3.10.6
       joblib: 1.5.2
threadpoolctl: 3.6.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 64
         prefix: libscipy_openblas
       filepath: /work/users/suny/mm/.venv/lib/python3.10/site-packages/numpy.libs/libscipy_openblas64_-56d6093b.so
        version: 0.3.29
threading_layer: pthreads
   architecture: SkylakeX

       user_api: blas
   internal_api: openblas
    num_threads: 64
         prefix: libscipy_openblas
       filepath: /work/users/suny/mm/.venv/lib/python3.10/site-packages/scipy.libs/libscipy_openblas-68440149.so
        version: 0.3.28
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 144
         prefix: libgomp
       filepath: /work/users/suny/mm/.venv/lib/python3.10/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None

       user_api: openmp
   internal_api: openmp
    num_threads: 144
         prefix: libgomp
       filepath: /usr/lib/x86_64-linux-gnu/libgomp.so.1.0.0
        version: None
```

COMMENT:
Closing this due to inactivity, but happy to re-open if required info provided.

Labels: Bug, Needs Info, Needs Reproducible Code
⚠️ CI failed on Linux_Runs.pylatest_conda_forge_mkl (last failure: Sep 11, 2025) ⚠️

**CI failed on [Linux_Runs.pylatest_conda_forge_mkl](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=79876&view=logs&j=dde5042c-7464-5d47-9507-31bdd2ee0a3a)** (Sep 11, 2025)
- Test Collection Failure

COMMENT:
## CI is no longer failing! ✅

[Successful run](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=79876&view=logs&j=de937867-5818-5907-5645-0b22541236da) on Sep 11, 2025

Labels: Needs Triage
Allow `categories` parameter in `OrdinalEncoder` to accept a dict of column names → categories

### Describe the workflow you want to enable

Currently, the `categories` parameter in `OrdinalEncoder` only accepts:

- `"auto"`, or
- a list of lists, where the position of each list corresponds to the order of columns in the input.

This makes it _somewhat inconvenient_ when working with pandas DataFrames, since one must manually align the category lists with the column order.

### Describe your proposed solution

Allow `categories` to also accept a dictionary mapping column names to category lists. For example:

```python
encoder = OrdinalEncoder(categories={
    "size": ["small", "medium", "large"],
    "priority": ["low", "medium", "high"]
})
```

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

#### Motivation

- Improves ergonomics when working with pandas (very common in scikit-learn pipelines).
- Reduces potential bugs from mismatched column ordering.
- Makes the API consistent with the way many users already think about preprocessing (column → transformation).

COMMENT:
Hey there.
This sounds like a useful addition, for pandas workflows. If the proposal is accepted, I’d be happy to take on the implementation and open a PR.

Labels: New Feature
Allow `categories` parameter in `OrdinalEncoder` to accept a dict of column names → categories

### Describe the workflow you want to enable

Currently, the `categories` parameter in `OrdinalEncoder` only accepts:

- `"auto"`, or
- a list of lists, where the position of each list corresponds to the order of columns in the input.

This makes it _somewhat inconvenient_ when working with pandas DataFrames, since one must manually align the category lists with the column order.

### Describe your proposed solution

Allow `categories` to also accept a dictionary mapping column names to category lists. For example:

```python
encoder = OrdinalEncoder(categories={
    "size": ["small", "medium", "large"],
    "priority": ["low", "medium", "high"]
})
```

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

#### Motivation

- Improves ergonomics when working with pandas (very common in scikit-learn pipelines).
- Reduces potential bugs from mismatched column ordering.
- Makes the API consistent with the way many users already think about preprocessing (column → transformation).

COMMENT:
I agree, this sounds like a very useful addition. @joaosferreira did you intend to open a PR yourself for it? Otherwise, @Yousinator could give it a shot.

Labels: New Feature
Allow `categories` parameter in `OrdinalEncoder` to accept a dict of column names → categories

### Describe the workflow you want to enable

Currently, the `categories` parameter in `OrdinalEncoder` only accepts:

- `"auto"`, or
- a list of lists, where the position of each list corresponds to the order of columns in the input.

This makes it _somewhat inconvenient_ when working with pandas DataFrames, since one must manually align the category lists with the column order.

### Describe your proposed solution

Allow `categories` to also accept a dictionary mapping column names to category lists. For example:

```python
encoder = OrdinalEncoder(categories={
    "size": ["small", "medium", "large"],
    "priority": ["low", "medium", "high"]
})
```

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

#### Motivation

- Improves ergonomics when working with pandas (very common in scikit-learn pipelines).
- Reduces potential bugs from mismatched column ordering.
- Makes the API consistent with the way many users already think about preprocessing (column → transformation).

COMMENT:
Yes, I do. Already working on a draft. Is that okay @ogrisel and @Yousinator?

Labels: New Feature
Latex not correctly rendered for ridge

### Describe the issue linked to the documentation

Description
In the online API reference, formula is displayed as:
`||y - Xw||^2_2 + alpha * ||w||^2_2`

it should instead look like

<img width="299" height="77" alt="Image" src="https://github.com/user-attachments/assets/f2f77288-d7a6-4ece-8f06-4e2b076c032d" />

Steps to Reproduce the Issue:
Please see [https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Ridge.html](url)


### Suggest a potential alternative/fix

_No response_

COMMENT:
I'm closing this issue because it duplicates #31593.

Labels: Documentation, Needs Triage