Talip7/scikit-learn-issues-multilabel huggingface dataset:
TargetEncoder has no way to select the columns being processed

### Describe the workflow you want to enable

`TargetEncoder` currently appears to try and look at the entire X dataframe. That's usually not what I want.

What I want to do is: just target encode one feature, or perhaps a small number of features in a dataset with many features.

Additionally, I want to be able to choose whether to keep the original columns that were target encoded, or drop them.

### Describe your proposed solution

`TargetEncoder` should have an argument indicating which columns (features) it should process. The default there could simply be all columns. The argument should allow the user to specify the features by column index / array index, or by column name.

It should have another option, allowing the user to choose whether they want to keep the original (untransformed) features, or drop them.

There should be a third option there, indicating an optional suffix to be added to the names of the transformed columns. If the original columns are dropped, the suffix could be the null string (no suffix). If the original columns are kept, the default suffix could be some reasonable string such as `_target_encoded`, and the user should be able to override it.

### Describe alternatives you've considered, if relevant

I can probably conjure the right combination of column selection in `ColumnTransformer` plus a bunch of other transformers, but it's unnecessarily complex.

### Additional context

_No response_

COMMENT:
`ColumnTransformer` is there to do exactly this. I don't think the convenience is worth adding the complexity to the code base. You  an simply do:

```py
make_column_transformer(
    (TargetEncoder(), ['col1', 'col2']),
    remainder="passthrough",
)
```

And if you have a lot of these cases, a simple function in your code base returning the right `ColumnTransformer` would solve the issue.

Labels: New Feature, Needs Triage
Transform output to xarray objects

### Describe the workflow you want to enable

I just watched @thomasjpfan's SciPy lightning talk where he showed using `.set_output(transform='pandas')` to return results wrapped in pandas dataframes. **I want the same but for `xarray.Dataset`**.

### Describe your proposed solution

However it works for pandas but for xarray. (Which PR was it implemented in?) There is even an [`xarray.Dataset.from_dataframe`](https://docs.xarray.dev/en/latest/generated/xarray.Dataset.from_dataframe.html) method.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

I think a fully N-dimensional object would make a lot of sense to wrap sklearn output in. I recently found myself wishing this existed whilst doing PCA for example.

COMMENT:
Thanks for opening this issue. I'm going to cross-link it with #25896 which is related as it is about supporting dataframes other than pandas. I think it makes sense to try and have an overview of "all the demands" when trying to come up with a solution. At least I hope we can come up with a general abstraction that makes it simpler to add new in-/outputs.

I'll leave it to you to decide if you want to keep this issue open or close it&post a short comment about xarray in the linked issue.

Labels: New Feature
Transform output to xarray objects

### Describe the workflow you want to enable

I just watched @thomasjpfan's SciPy lightning talk where he showed using `.set_output(transform='pandas')` to return results wrapped in pandas dataframes. **I want the same but for `xarray.Dataset`**.

### Describe your proposed solution

However it works for pandas but for xarray. (Which PR was it implemented in?) There is even an [`xarray.Dataset.from_dataframe`](https://docs.xarray.dev/en/latest/generated/xarray.Dataset.from_dataframe.html) method.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

I think a fully N-dimensional object would make a lot of sense to wrap sklearn output in. I recently found myself wishing this existed whilst doing PCA for example.

COMMENT:
I think this in particular is also very much related to the discussion we had during the sprint in Paris, where we talked about a heterogeneous container. 

I don't remember all the details anymore, but I think @thomasjpfan has a good overview.

Labels: New Feature
Transform output to xarray objects

### Describe the workflow you want to enable

I just watched @thomasjpfan's SciPy lightning talk where he showed using `.set_output(transform='pandas')` to return results wrapped in pandas dataframes. **I want the same but for `xarray.Dataset`**.

### Describe your proposed solution

However it works for pandas but for xarray. (Which PR was it implemented in?) There is even an [`xarray.Dataset.from_dataframe`](https://docs.xarray.dev/en/latest/generated/xarray.Dataset.from_dataframe.html) method.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

I think a fully N-dimensional object would make a lot of sense to wrap sklearn output in. I recently found myself wishing this existed whilst doing PCA for example.

COMMENT:
At SciPy 2023, I spoke with Tom about supporting XArray directly in scikit-learn as another "output" option. Compared to Panda's DataFrames, XArray has a bit more metadata that is "sample aligned" to be passed to the output. I'm planning to open a PR proposing a general abstraction to easily support other output containers.

Labels: New Feature
Are there some format limitation of the training batches?

For mmc4 training, I noticed that the loss before the first <image> token is removed in https://github.com/mlfoundations/open_flamingo/blob/main/open_flamingo/train/train_utils.py#L132. I wonder why we need an operation like this.

Thanks!

COMMENT:
Being directed to a wrong repo.

Labels: Needs Triage
Documentation on how DTs deal with nans

### Describe the issue linked to the documentation

This webpage suggests that DTs can handle nans in a way that is not just imputation: https://scikit-learn.org/stable/modules/impute.html#estimators-that-handle-nan-values

However, I cannot find any documentation on how this is accomplished.

### Suggest a potential alternative/fix

_No response_

COMMENT:
https://scikit-learn.org/stable/modules/tree.html#missing-values-support

Labels: Documentation, Needs Triage
Add an example showcasing the HistGradientBoosting...

### Describe the issue linked to the documentation

The documentation is not putting forward the HGBT very much. People are not guided to it, while it is arguably the most useful model in scikit-learn

One paradigmatic problem is that on the front-page we have a nice visual on regression, but it point to an AdaBoost example :crying_cat_face: 
![image](https://github.com/scikit-learn/scikit-learn/assets/208217/b1bebf8c-168c-4da5-884d-8e3b8ab6c0e6)
I tried having it point to a good didactic example on the HGBT but I couldn't find one.

### Suggest a potential alternative/fix

We should create an example dedicating to showcasing the HBGT.

I think that it should start with a simple didactic example + plot (a plot a bit like the one above would be nice). It should then (in separate section) showcase a few other nice features, for instance:

- Quantile regression
- Support of missing values
- Support of categorical values
- Monotonicity constraints

There are already examples on the various features that I listed above. Ideally, I like not to add examples (we have too many). I'm not sure whether we should merge these in (not convinced, as the corresponding examples are probably too detailed) or point too them.

I would also make sure that this example is linked to from a bunch of places (for instance from the various examples on gradient boosting, just to make sure that people do find the HGBT)

Also, we should link from the HGBT docs to this example and vice-versa

- [ ] Do the example
- [ ] Replace the landing-page figure and link to example
- [ ] Cross-link in the documentation
- [ ] Cross-link in other examples

cc @ArturoAmorQ

COMMENT:
related: https://github.com/scikit-learn/scikit-learn/issues/26220

Labels: Documentation
Add an example showcasing the HistGradientBoosting...

### Describe the issue linked to the documentation

The documentation is not putting forward the HGBT very much. People are not guided to it, while it is arguably the most useful model in scikit-learn

One paradigmatic problem is that on the front-page we have a nice visual on regression, but it point to an AdaBoost example :crying_cat_face: 
![image](https://github.com/scikit-learn/scikit-learn/assets/208217/b1bebf8c-168c-4da5-884d-8e3b8ab6c0e6)
I tried having it point to a good didactic example on the HGBT but I couldn't find one.

### Suggest a potential alternative/fix

We should create an example dedicating to showcasing the HBGT.

I think that it should start with a simple didactic example + plot (a plot a bit like the one above would be nice). It should then (in separate section) showcase a few other nice features, for instance:

- Quantile regression
- Support of missing values
- Support of categorical values
- Monotonicity constraints

There are already examples on the various features that I listed above. Ideally, I like not to add examples (we have too many). I'm not sure whether we should merge these in (not convinced, as the corresponding examples are probably too detailed) or point too them.

I would also make sure that this example is linked to from a bunch of places (for instance from the various examples on gradient boosting, just to make sure that people do find the HGBT)

Also, we should link from the HGBT docs to this example and vice-versa

- [ ] Do the example
- [ ] Replace the landing-page figure and link to example
- [ ] Cross-link in the documentation
- [ ] Cross-link in other examples

cc @ArturoAmorQ

COMMENT:
I will take care of this.

Labels: Documentation
Add an example showcasing the HistGradientBoosting...

### Describe the issue linked to the documentation

The documentation is not putting forward the HGBT very much. People are not guided to it, while it is arguably the most useful model in scikit-learn

One paradigmatic problem is that on the front-page we have a nice visual on regression, but it point to an AdaBoost example :crying_cat_face: 
![image](https://github.com/scikit-learn/scikit-learn/assets/208217/b1bebf8c-168c-4da5-884d-8e3b8ab6c0e6)
I tried having it point to a good didactic example on the HGBT but I couldn't find one.

### Suggest a potential alternative/fix

We should create an example dedicating to showcasing the HBGT.

I think that it should start with a simple didactic example + plot (a plot a bit like the one above would be nice). It should then (in separate section) showcase a few other nice features, for instance:

- Quantile regression
- Support of missing values
- Support of categorical values
- Monotonicity constraints

There are already examples on the various features that I listed above. Ideally, I like not to add examples (we have too many). I'm not sure whether we should merge these in (not convinced, as the corresponding examples are probably too detailed) or point too them.

I would also make sure that this example is linked to from a bunch of places (for instance from the various examples on gradient boosting, just to make sure that people do find the HGBT)

Also, we should link from the HGBT docs to this example and vice-versa

- [ ] Do the example
- [ ] Replace the landing-page figure and link to example
- [ ] Cross-link in the documentation
- [ ] Cross-link in other examples

cc @ArturoAmorQ

COMMENT:
Can we re-write the Adaboost example to use HistGradBoost and conserve the number of examples by having one less example for Adaboost?

Labels: Documentation
Add an example showcasing the HistGradientBoosting...

### Describe the issue linked to the documentation

The documentation is not putting forward the HGBT very much. People are not guided to it, while it is arguably the most useful model in scikit-learn

One paradigmatic problem is that on the front-page we have a nice visual on regression, but it point to an AdaBoost example :crying_cat_face: 
![image](https://github.com/scikit-learn/scikit-learn/assets/208217/b1bebf8c-168c-4da5-884d-8e3b8ab6c0e6)
I tried having it point to a good didactic example on the HGBT but I couldn't find one.

### Suggest a potential alternative/fix

We should create an example dedicating to showcasing the HBGT.

I think that it should start with a simple didactic example + plot (a plot a bit like the one above would be nice). It should then (in separate section) showcase a few other nice features, for instance:

- Quantile regression
- Support of missing values
- Support of categorical values
- Monotonicity constraints

There are already examples on the various features that I listed above. Ideally, I like not to add examples (we have too many). I'm not sure whether we should merge these in (not convinced, as the corresponding examples are probably too detailed) or point too them.

I would also make sure that this example is linked to from a bunch of places (for instance from the various examples on gradient boosting, just to make sure that people do find the HGBT)

Also, we should link from the HGBT docs to this example and vice-versa

- [ ] Do the example
- [ ] Replace the landing-page figure and link to example
- [ ] Cross-link in the documentation
- [ ] Cross-link in other examples

cc @ArturoAmorQ

COMMENT:
> Can we re-write the Adaboost example to use HistGradBoost and conserve the number of examples by having one less example for Adaboost?

I think that it's the only AdaBoost example

Labels: Documentation
Add an example showcasing the HistGradientBoosting...

### Describe the issue linked to the documentation

The documentation is not putting forward the HGBT very much. People are not guided to it, while it is arguably the most useful model in scikit-learn

One paradigmatic problem is that on the front-page we have a nice visual on regression, but it point to an AdaBoost example :crying_cat_face: 
![image](https://github.com/scikit-learn/scikit-learn/assets/208217/b1bebf8c-168c-4da5-884d-8e3b8ab6c0e6)
I tried having it point to a good didactic example on the HGBT but I couldn't find one.

### Suggest a potential alternative/fix

We should create an example dedicating to showcasing the HBGT.

I think that it should start with a simple didactic example + plot (a plot a bit like the one above would be nice). It should then (in separate section) showcase a few other nice features, for instance:

- Quantile regression
- Support of missing values
- Support of categorical values
- Monotonicity constraints

There are already examples on the various features that I listed above. Ideally, I like not to add examples (we have too many). I'm not sure whether we should merge these in (not convinced, as the corresponding examples are probably too detailed) or point too them.

I would also make sure that this example is linked to from a bunch of places (for instance from the various examples on gradient boosting, just to make sure that people do find the HGBT)

Also, we should link from the HGBT docs to this example and vice-versa

- [ ] Do the example
- [ ] Replace the landing-page figure and link to example
- [ ] Cross-link in the documentation
- [ ] Cross-link in other examples

cc @ArturoAmorQ

COMMENT:
We can still re-write the Adaboost example to add a similar figure for HGBT but using the parameter `max_iter` instead of `n_estimators`.

This would conserve the number of examples and Adaboost examples at the same time.

Labels: Documentation
Add an example showcasing the HistGradientBoosting...

### Describe the issue linked to the documentation

The documentation is not putting forward the HGBT very much. People are not guided to it, while it is arguably the most useful model in scikit-learn

One paradigmatic problem is that on the front-page we have a nice visual on regression, but it point to an AdaBoost example :crying_cat_face: 
![image](https://github.com/scikit-learn/scikit-learn/assets/208217/b1bebf8c-168c-4da5-884d-8e3b8ab6c0e6)
I tried having it point to a good didactic example on the HGBT but I couldn't find one.

### Suggest a potential alternative/fix

We should create an example dedicating to showcasing the HBGT.

I think that it should start with a simple didactic example + plot (a plot a bit like the one above would be nice). It should then (in separate section) showcase a few other nice features, for instance:

- Quantile regression
- Support of missing values
- Support of categorical values
- Monotonicity constraints

There are already examples on the various features that I listed above. Ideally, I like not to add examples (we have too many). I'm not sure whether we should merge these in (not convinced, as the corresponding examples are probably too detailed) or point too them.

I would also make sure that this example is linked to from a bunch of places (for instance from the various examples on gradient boosting, just to make sure that people do find the HGBT)

Also, we should link from the HGBT docs to this example and vice-versa

- [ ] Do the example
- [ ] Replace the landing-page figure and link to example
- [ ] Cross-link in the documentation
- [ ] Cross-link in other examples

cc @ArturoAmorQ

COMMENT:
> We can still re-write the Adaboost example to add a similar figure for HGBT but using the parameter max_iter instead of n_estimators.

I would rather not have AdaBoost and HGBT on the same example. AdaBoost is minor and should not get much attention, while HGBT really deserves a full example of it's own.

In terms of removing examples, we have many better candidates (for instance merging the examples "Recursive feature elimination" and "Recursive feature elimination with cross-validation")

Labels: Documentation
Add an example showcasing the HistGradientBoosting...

### Describe the issue linked to the documentation

The documentation is not putting forward the HGBT very much. People are not guided to it, while it is arguably the most useful model in scikit-learn

One paradigmatic problem is that on the front-page we have a nice visual on regression, but it point to an AdaBoost example :crying_cat_face: 
![image](https://github.com/scikit-learn/scikit-learn/assets/208217/b1bebf8c-168c-4da5-884d-8e3b8ab6c0e6)
I tried having it point to a good didactic example on the HGBT but I couldn't find one.

### Suggest a potential alternative/fix

We should create an example dedicating to showcasing the HBGT.

I think that it should start with a simple didactic example + plot (a plot a bit like the one above would be nice). It should then (in separate section) showcase a few other nice features, for instance:

- Quantile regression
- Support of missing values
- Support of categorical values
- Monotonicity constraints

There are already examples on the various features that I listed above. Ideally, I like not to add examples (we have too many). I'm not sure whether we should merge these in (not convinced, as the corresponding examples are probably too detailed) or point too them.

I would also make sure that this example is linked to from a bunch of places (for instance from the various examples on gradient boosting, just to make sure that people do find the HGBT)

Also, we should link from the HGBT docs to this example and vice-versa

- [ ] Do the example
- [ ] Replace the landing-page figure and link to example
- [ ] Cross-link in the documentation
- [ ] Cross-link in other examples

cc @ArturoAmorQ

COMMENT:
Also, advertising stock price modeling with simple ML model is not helpful at managing expectations of our users. We should probably highlight more successful applications of regression. Here are candidate suggestions:

- drug response,
- demand forecasting in retail,
- real estate prices,
- machine failure rates,
- energy consumption.

Labels: Documentation
Add an example showcasing the HistGradientBoosting...

### Describe the issue linked to the documentation

The documentation is not putting forward the HGBT very much. People are not guided to it, while it is arguably the most useful model in scikit-learn

One paradigmatic problem is that on the front-page we have a nice visual on regression, but it point to an AdaBoost example :crying_cat_face: 
![image](https://github.com/scikit-learn/scikit-learn/assets/208217/b1bebf8c-168c-4da5-884d-8e3b8ab6c0e6)
I tried having it point to a good didactic example on the HGBT but I couldn't find one.

### Suggest a potential alternative/fix

We should create an example dedicating to showcasing the HBGT.

I think that it should start with a simple didactic example + plot (a plot a bit like the one above would be nice). It should then (in separate section) showcase a few other nice features, for instance:

- Quantile regression
- Support of missing values
- Support of categorical values
- Monotonicity constraints

There are already examples on the various features that I listed above. Ideally, I like not to add examples (we have too many). I'm not sure whether we should merge these in (not convinced, as the corresponding examples are probably too detailed) or point too them.

I would also make sure that this example is linked to from a bunch of places (for instance from the various examples on gradient boosting, just to make sure that people do find the HGBT)

Also, we should link from the HGBT docs to this example and vice-versa

- [ ] Do the example
- [ ] Replace the landing-page figure and link to example
- [ ] Cross-link in the documentation
- [ ] Cross-link in other examples

cc @ArturoAmorQ

COMMENT:
@ogrisel Do you know a public dataset for each?

Labels: Documentation
Add an example showcasing the HistGradientBoosting...

### Describe the issue linked to the documentation

The documentation is not putting forward the HGBT very much. People are not guided to it, while it is arguably the most useful model in scikit-learn

One paradigmatic problem is that on the front-page we have a nice visual on regression, but it point to an AdaBoost example :crying_cat_face: 
![image](https://github.com/scikit-learn/scikit-learn/assets/208217/b1bebf8c-168c-4da5-884d-8e3b8ab6c0e6)
I tried having it point to a good didactic example on the HGBT but I couldn't find one.

### Suggest a potential alternative/fix

We should create an example dedicating to showcasing the HBGT.

I think that it should start with a simple didactic example + plot (a plot a bit like the one above would be nice). It should then (in separate section) showcase a few other nice features, for instance:

- Quantile regression
- Support of missing values
- Support of categorical values
- Monotonicity constraints

There are already examples on the various features that I listed above. Ideally, I like not to add examples (we have too many). I'm not sure whether we should merge these in (not convinced, as the corresponding examples are probably too detailed) or point too them.

I would also make sure that this example is linked to from a bunch of places (for instance from the various examples on gradient boosting, just to make sure that people do find the HGBT)

Also, we should link from the HGBT docs to this example and vice-versa

- [ ] Do the example
- [ ] Replace the landing-page figure and link to example
- [ ] Cross-link in the documentation
- [ ] Cross-link in other examples

cc @ArturoAmorQ

COMMENT:
> Also, advertising stock price modeling with simple ML model is not helpful at managing expectations of our users. We should probably highlight more successful applications of regression. Here are candidate suggestions:
> 
> * drug response,
> * demand forecasting in retail,
> * real estate prices,
> * machine failure rates,
> * energy consumption.

@ogrisel can I start working on this?

Labels: Documentation