Talip7/scikit-learn-issues-multilabel huggingface dataset:
TimeSeriesSplit Needs a version that allows users to specify initial time

### Describe the workflow you want to enable

I would like to add a new `TimeSeriesSplit` [link](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.TimeSeriesSplit.html) called `TimeSeriesRollingOriginSplit`. In the current `TimeSeriesSplit` it is not possible for users to get an initial length.

I am happy to open a PR for this myself if it is approved.


### Describe your proposed solution


This is the proposed new `TimeSeriesRollingOriginSplit`
`class sklearn.model_selection.TimeSeriesRollingOriginSplit(initial = 5, test_size = 1, gap = 0)`

`initial`: the initial number of samples to use. it is often common to start with a larger training period.

`test_size`: Used to set the test_size

`gap`: Number of samples to exclude from the end of each train set before the test set.


### Describe alternatives you've considered, if relevant

I currently use `TimeSeriesSplit` in a loop, but exclude the initial splits until the train set is the size I want.

### Additional context

Again. I am happy to contribute this feature.

COMMENT:
Well, when I started coding I decided to change the name of the object, so the original name no longer represents what I coded. 

I am sorry. The fact that I put the wrong example in that commented out code is not helping.

I will paste a real example. 


# Example 1 - no gap

```python
X = np.arange(0,50)
```

`initial=7*3`, `test_size = 7`, `gap=0`

```sh
TRAIN: [ 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20] TEST: [21 22 23 24 25 26 27]
TRAIN: [ 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
 24 25 26 27] TEST: [28 29 30 31 32 33 34]
TRAIN: [ 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
 24 25 26 27 28 29 30 31 32 33 34] TEST: [35 36 37 38 39 40 41]
TRAIN: [ 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41] TEST: [42 43 44 45 46 47 48]
```

# Example 2 - with gap

`initial=7*3`, `test_size = 7`, `gap=2`

```
TRAIN: [ 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20] TEST: [23 24 25 26 27 28 29]
TRAIN: [ 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
 24 25 26 27] TEST: [30 31 32 33 34 35 36]
TRAIN: [ 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
 24 25 26 27 28 29 30 31 32 33 34] TEST: [37 38 39 40 41 42 43]
```

This is what I get when I run on my machine.

Labels: New Feature, module:model_selection, Needs Decision - Include Feature
TimeSeriesSplit Needs a version that allows users to specify initial time

### Describe the workflow you want to enable

I would like to add a new `TimeSeriesSplit` [link](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.TimeSeriesSplit.html) called `TimeSeriesRollingOriginSplit`. In the current `TimeSeriesSplit` it is not possible for users to get an initial length.

I am happy to open a PR for this myself if it is approved.


### Describe your proposed solution


This is the proposed new `TimeSeriesRollingOriginSplit`
`class sklearn.model_selection.TimeSeriesRollingOriginSplit(initial = 5, test_size = 1, gap = 0)`

`initial`: the initial number of samples to use. it is often common to start with a larger training period.

`test_size`: Used to set the test_size

`gap`: Number of samples to exclude from the end of each train set before the test set.


### Describe alternatives you've considered, if relevant

I currently use `TimeSeriesSplit` in a loop, but exclude the initial splits until the train set is the size I want.

### Additional context

Again. I am happy to contribute this feature.

COMMENT:
Okay I think the proposed behavior is clear. Historically, we have resisted adding another `TimeSeries*` splitter and pushed to add features to `TimeSeriesSplit` instead.

I do not have the bandwidth to push this forward. My goal here was to triage this issue into a concrete API so others can evaluate it.

Labels: New Feature, module:model_selection, Needs Decision - Include Feature
TimeSeriesSplit Needs a version that allows users to specify initial time

### Describe the workflow you want to enable

I would like to add a new `TimeSeriesSplit` [link](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.TimeSeriesSplit.html) called `TimeSeriesRollingOriginSplit`. In the current `TimeSeriesSplit` it is not possible for users to get an initial length.

I am happy to open a PR for this myself if it is approved.


### Describe your proposed solution


This is the proposed new `TimeSeriesRollingOriginSplit`
`class sklearn.model_selection.TimeSeriesRollingOriginSplit(initial = 5, test_size = 1, gap = 0)`

`initial`: the initial number of samples to use. it is often common to start with a larger training period.

`test_size`: Used to set the test_size

`gap`: Number of samples to exclude from the end of each train set before the test set.


### Describe alternatives you've considered, if relevant

I currently use `TimeSeriesSplit` in a loop, but exclude the initial splits until the train set is the size I want.

### Additional context

Again. I am happy to contribute this feature.

COMMENT:
Thanks for your patients.

Labels: New Feature, module:model_selection, Needs Decision - Include Feature
TimeSeriesSplit Needs a version that allows users to specify initial time

### Describe the workflow you want to enable

I would like to add a new `TimeSeriesSplit` [link](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.TimeSeriesSplit.html) called `TimeSeriesRollingOriginSplit`. In the current `TimeSeriesSplit` it is not possible for users to get an initial length.

I am happy to open a PR for this myself if it is approved.


### Describe your proposed solution


This is the proposed new `TimeSeriesRollingOriginSplit`
`class sklearn.model_selection.TimeSeriesRollingOriginSplit(initial = 5, test_size = 1, gap = 0)`

`initial`: the initial number of samples to use. it is often common to start with a larger training period.

`test_size`: Used to set the test_size

`gap`: Number of samples to exclude from the end of each train set before the test set.


### Describe alternatives you've considered, if relevant

I currently use `TimeSeriesSplit` in a loop, but exclude the initial splits until the train set is the size I want.

### Additional context

Again. I am happy to contribute this feature.

COMMENT:
Just wanted to state that fixed the example in the PR. Happy to comply with other changes necessary.

Labels: New Feature, module:model_selection, Needs Decision - Include Feature
TimeSeriesSplit Needs a version that allows users to specify initial time

### Describe the workflow you want to enable

I would like to add a new `TimeSeriesSplit` [link](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.TimeSeriesSplit.html) called `TimeSeriesRollingOriginSplit`. In the current `TimeSeriesSplit` it is not possible for users to get an initial length.

I am happy to open a PR for this myself if it is approved.


### Describe your proposed solution


This is the proposed new `TimeSeriesRollingOriginSplit`
`class sklearn.model_selection.TimeSeriesRollingOriginSplit(initial = 5, test_size = 1, gap = 0)`

`initial`: the initial number of samples to use. it is often common to start with a larger training period.

`test_size`: Used to set the test_size

`gap`: Number of samples to exclude from the end of each train set before the test set.


### Describe alternatives you've considered, if relevant

I currently use `TimeSeriesSplit` in a loop, but exclude the initial splits until the train set is the size I want.

### Additional context

Again. I am happy to contribute this feature.

COMMENT:
I think the `initial_value` idea could work. I am a little new to the ideology of sklearn. I have never made a contribution to this repo before. I guess this is just a tradeoff between '*do one thing and do it well*' vs *every line of code is a bug*. A new `TimeSeriesSplitInitial` adds lines of code but is focused, while simply adding the `initial_value` adds less focus to the current `TimeSeriesSplit`, but you get the benefit of having fewer lines of code. 

I don't really care either way. I am just looking for the functionality to be added in some way so I do not have to think so hard as a user when I run my forecast backtests.

Labels: New Feature, module:model_selection, Needs Decision - Include Feature
TimeSeriesSplit Needs a version that allows users to specify initial time

### Describe the workflow you want to enable

I would like to add a new `TimeSeriesSplit` [link](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.TimeSeriesSplit.html) called `TimeSeriesRollingOriginSplit`. In the current `TimeSeriesSplit` it is not possible for users to get an initial length.

I am happy to open a PR for this myself if it is approved.


### Describe your proposed solution


This is the proposed new `TimeSeriesRollingOriginSplit`
`class sklearn.model_selection.TimeSeriesRollingOriginSplit(initial = 5, test_size = 1, gap = 0)`

`initial`: the initial number of samples to use. it is often common to start with a larger training period.

`test_size`: Used to set the test_size

`gap`: Number of samples to exclude from the end of each train set before the test set.


### Describe alternatives you've considered, if relevant

I currently use `TimeSeriesSplit` in a loop, but exclude the initial splits until the train set is the size I want.

### Additional context

Again. I am happy to contribute this feature.

COMMENT:
@alexhallam Scikit-learn tends to be conservative with adding new code to minimize maintainability overhead (though I'm not a core-dev so take my words with a grain of salt). To that end, in your use case is there a reason to desire an **exact** initial size, say `initial_size == 21` or is it okay to provide a guaranteed minimum, e.g. `initial_size >= 21`? If it's the latter, I'm inclined to think there's not anything we need to do on the scikit-learn side since you can already accomplish this (as demonstrated in my code samples).

Labels: New Feature, module:model_selection, Needs Decision - Include Feature
TimeSeriesSplit Needs a version that allows users to specify initial time

### Describe the workflow you want to enable

I would like to add a new `TimeSeriesSplit` [link](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.TimeSeriesSplit.html) called `TimeSeriesRollingOriginSplit`. In the current `TimeSeriesSplit` it is not possible for users to get an initial length.

I am happy to open a PR for this myself if it is approved.


### Describe your proposed solution


This is the proposed new `TimeSeriesRollingOriginSplit`
`class sklearn.model_selection.TimeSeriesRollingOriginSplit(initial = 5, test_size = 1, gap = 0)`

`initial`: the initial number of samples to use. it is often common to start with a larger training period.

`test_size`: Used to set the test_size

`gap`: Number of samples to exclude from the end of each train set before the test set.


### Describe alternatives you've considered, if relevant

I currently use `TimeSeriesSplit` in a loop, but exclude the initial splits until the train set is the size I want.

### Additional context

Again. I am happy to contribute this feature.

COMMENT:
Well, I will argue my case. You can take it or leave it. I am sure that scikit-learn takes a lot of effort to maintain so I understand.

I am a forecaster by trade. I have been doing it for a number of years now. This kind of cross validation is pretty common in this line of work. Some examples of this functionality are [time_series_split](https://business-science.github.io/timetk/reference/time_series_split.html) and [tsCV](https://pkg.robjhyndman.com/forecast/reference/tsCV.html). It is important to be able to set the initial time period. We know exactly the size our initial training period ought to be. It is something we like to control.

Labels: New Feature, module:model_selection, Needs Decision - Include Feature
TimeSeriesSplit Needs a version that allows users to specify initial time

### Describe the workflow you want to enable

I would like to add a new `TimeSeriesSplit` [link](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.TimeSeriesSplit.html) called `TimeSeriesRollingOriginSplit`. In the current `TimeSeriesSplit` it is not possible for users to get an initial length.

I am happy to open a PR for this myself if it is approved.


### Describe your proposed solution


This is the proposed new `TimeSeriesRollingOriginSplit`
`class sklearn.model_selection.TimeSeriesRollingOriginSplit(initial = 5, test_size = 1, gap = 0)`

`initial`: the initial number of samples to use. it is often common to start with a larger training period.

`test_size`: Used to set the test_size

`gap`: Number of samples to exclude from the end of each train set before the test set.


### Describe alternatives you've considered, if relevant

I currently use `TimeSeriesSplit` in a loop, but exclude the initial splits until the train set is the size I want.

### Additional context

Again. I am happy to contribute this feature.

COMMENT:
I just added this function to my own package. If you, or other lurkers, are interested I have the documentation in the API docs. 

https://alexhallam.github.io/tablespoon/section/api/ 

Do a `Ctrl+f` for `TimeSeriesInitialSplit`

I also have a little example here 

https://github.com/alexhallam/tablespoon/blob/main/examples/time_series_cv.ipynb

I think we close this if `scikit-learn` does not want to accept this function.

Labels: New Feature, module:model_selection, Needs Decision - Include Feature
TimeSeriesSplit Needs a version that allows users to specify initial time

### Describe the workflow you want to enable

I would like to add a new `TimeSeriesSplit` [link](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.TimeSeriesSplit.html) called `TimeSeriesRollingOriginSplit`. In the current `TimeSeriesSplit` it is not possible for users to get an initial length.

I am happy to open a PR for this myself if it is approved.


### Describe your proposed solution


This is the proposed new `TimeSeriesRollingOriginSplit`
`class sklearn.model_selection.TimeSeriesRollingOriginSplit(initial = 5, test_size = 1, gap = 0)`

`initial`: the initial number of samples to use. it is often common to start with a larger training period.

`test_size`: Used to set the test_size

`gap`: Number of samples to exclude from the end of each train set before the test set.


### Describe alternatives you've considered, if relevant

I currently use `TimeSeriesSplit` in a loop, but exclude the initial splits until the train set is the size I want.

### Additional context

Again. I am happy to contribute this feature.

COMMENT:
btw [prophet](https://github.com/facebook/prophet) has a very [similar implementation](https://github.com/facebook/prophet/blob/main/python/prophet/diagnostics.py#L61) to the one described here. 

```python
def generate_cutoffs(df, horizon, initial, period):
    """Generate cutoff dates
    Parameters
    ----------
    df: pd.DataFrame with historical data.
    horizon: pd.Timedelta forecast horizon.
    initial: pd.Timedelta window of the initial forecast period.
    period: pd.Timedelta simulated forecasts are done with this period.
    Returns
    -------
    list of pd.Timestamp
    """
```
it generates a list of timestamps that represents the cutoffs of the dataframe, is not the same as the OP was pointing out, but covers the same problem, very common in time-series applications

Labels: New Feature, module:model_selection, Needs Decision - Include Feature
TimeSeriesSplit Needs a version that allows users to specify initial time

### Describe the workflow you want to enable

I would like to add a new `TimeSeriesSplit` [link](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.TimeSeriesSplit.html) called `TimeSeriesRollingOriginSplit`. In the current `TimeSeriesSplit` it is not possible for users to get an initial length.

I am happy to open a PR for this myself if it is approved.


### Describe your proposed solution


This is the proposed new `TimeSeriesRollingOriginSplit`
`class sklearn.model_selection.TimeSeriesRollingOriginSplit(initial = 5, test_size = 1, gap = 0)`

`initial`: the initial number of samples to use. it is often common to start with a larger training period.

`test_size`: Used to set the test_size

`gap`: Number of samples to exclude from the end of each train set before the test set.


### Describe alternatives you've considered, if relevant

I currently use `TimeSeriesSplit` in a loop, but exclude the initial splits until the train set is the size I want.

### Additional context

Again. I am happy to contribute this feature.

COMMENT:
Yeah, it is common enough to be a good addition to sklearn if the cost of maintenance is not to high.

Labels: New Feature, module:model_selection, Needs Decision - Include Feature
GroupKFold: "The same group will not appear in two different folds..."

https://github.com/scikit-learn/scikit-learn/blob/baf0ea25d6dd034403370fea552b21a6776bef18/sklearn/model_selection/_split.py#L456-L457

Hi, just was wondering what exactly was meant by the above lines in sklearn's docs for GroupKFold. Based on the examples on [this](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GroupKFold.html) page and the [respective explanation in the User Guide](https://scikit-learn.org/stable/modules/cross_validation.html#group-k-fold), each group is used as part of the test subset in just one fold, and for the rest of the folds it's used within the training subset, hence, each group **does** appear in different folds, but it's just never used both within the test subset and the train subset. Would appreciate it if you could clarify this, perhaps, I am missing something. Thanks in advance!

COMMENT:
I think that we mean the test fold indeed. A group will not be chosen twice to appear in the test set. The phrasing could be improved there.

Labels: Documentation, module:model_selection
GroupKFold: "The same group will not appear in two different folds..."

https://github.com/scikit-learn/scikit-learn/blob/baf0ea25d6dd034403370fea552b21a6776bef18/sklearn/model_selection/_split.py#L456-L457

Hi, just was wondering what exactly was meant by the above lines in sklearn's docs for GroupKFold. Based on the examples on [this](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GroupKFold.html) page and the [respective explanation in the User Guide](https://scikit-learn.org/stable/modules/cross_validation.html#group-k-fold), each group is used as part of the test subset in just one fold, and for the rest of the folds it's used within the training subset, hence, each group **does** appear in different folds, but it's just never used both within the test subset and the train subset. Would appreciate it if you could clarify this, perhaps, I am missing something. Thanks in advance!

COMMENT:
@glemaitre Thanks for the clarification!

Labels: Documentation, module:model_selection
GroupKFold: "The same group will not appear in two different folds..."

https://github.com/scikit-learn/scikit-learn/blob/baf0ea25d6dd034403370fea552b21a6776bef18/sklearn/model_selection/_split.py#L456-L457

Hi, just was wondering what exactly was meant by the above lines in sklearn's docs for GroupKFold. Based on the examples on [this](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GroupKFold.html) page and the [respective explanation in the User Guide](https://scikit-learn.org/stable/modules/cross_validation.html#group-k-fold), each group is used as part of the test subset in just one fold, and for the rest of the folds it's used within the training subset, hence, each group **does** appear in different folds, but it's just never used both within the test subset and the train subset. Would appreciate it if you could clarify this, perhaps, I am missing something. Thanks in advance!

COMMENT:
Sorry, closed by accident, reopening it

Labels: Documentation, module:model_selection
RFC SLEP006: allow users to enable a "strict" mode in metadata routing

ref: https://github.com/scikit-learn/scikit-learn/pull/22986/files#r862344847

Along the way we've had discussions on whether we should raise if a metadata is requested but not provided.

Mirroring the existing behavior, SLEP006 and the implementation won't raise in the following case:

``` python
pipeline = make_pipeline(LogisticRegression().set_fit_request(sample_weight=True)).fit(X, y)
```

Should we allow a strict mode, either globally or on the estimator level, which would make the above code raise?

COMMENT:
For me, I prefer the final API to raise an error. In your example, `LogisticRegression` is configured to have a `sample_weight`, but is it not provided one. If we do nothing, then the following two pipelines have the same behavior:

```python
pipeline = make_pipeline(LogisticRegression().set_fit_request(sample_weight=True)).fit(X, y)

pipeline = make_pipeline(LogisticRegression()).fit(X, y)
```

That feels weird to me.

Do you think the final API should be strict by default?

Labels: API, RFC
RFC SLEP006: allow users to enable a "strict" mode in metadata routing

ref: https://github.com/scikit-learn/scikit-learn/pull/22986/files#r862344847

Along the way we've had discussions on whether we should raise if a metadata is requested but not provided.

Mirroring the existing behavior, SLEP006 and the implementation won't raise in the following case:

``` python
pipeline = make_pipeline(LogisticRegression().set_fit_request(sample_weight=True)).fit(X, y)
```

Should we allow a strict mode, either globally or on the estimator level, which would make the above code raise?

COMMENT:
I don't mind having it strict by default, but I might be missing some edge cases. Would you have something in mind to object the default strict mode @jnothman ?

Labels: API, RFC
RFC SLEP006: allow users to enable a "strict" mode in metadata routing

ref: https://github.com/scikit-learn/scikit-learn/pull/22986/files#r862344847

Along the way we've had discussions on whether we should raise if a metadata is requested but not provided.

Mirroring the existing behavior, SLEP006 and the implementation won't raise in the following case:

``` python
pipeline = make_pipeline(LogisticRegression().set_fit_request(sample_weight=True)).fit(X, y)
```

Should we allow a strict mode, either globally or on the estimator level, which would make the above code raise?

COMMENT:
Basically by not being strict we're allowing for someone to provide us with a canned composite estimator with `sample_weight` supported but not required. If an estimator truly _requires_ `sample_weight` it will raise in any case; but as long as we're not being strict allow for default behaviours in reusable composite estimators.

By saying `set_fit_request(sample_weight=True)` I don't think the user is asking you to validate their input differently, but rather that if `sample_weight` is provided it should go to the requested place. I acknowledge that these semantics are somewhat confusing, but I'm not sure how to fix them.

Labels: API, RFC