Question: I'm looking to perform walk forward validation on my time-series data. Extensive document exists on how to perform rolling window:   or expanding window   But this validation does not correspond to what will be in my production system: I want to daily retrain a model that will make prediction 14 days in the future. So I would only add one day of data to my previous training period (where the other methods add on the following training folds an entire set of data of length test_size; 14 days in my case). Therefore, I would like to validate my model with a sliding window:   My question is that I can't come across a Python library that would do the work. TimeSeriesSplit from sklearn has no option of that kind. Basically I want to provide : test_size, n_fold, min_train_size and  if n_fold > (n_samples - min_train_size) % test_size then next training_set draw data from the previous fold test_set

Answer: Here is my solution that allows the user to specify the testing horizon and the minimum sample of data for training: from sklearn.model_selection import TimeSeriesSplit from sklearn.utils import indexable from sklearn.utils.validation import _num_samples class TimeSeriesSplitCustom(TimeSeriesSplit): def __init__(self, n_splits=5, max_train_size=None, test_size=1, min_train_size=1): super().__init__(n_splits=n_splits, max_train_size=max_train_size) self.test_size = test_size self.min_train_size = min_train_size def overlapping_split(self, X, y=None, groups=None): min_train_size = self.min_train_size test_size = self.test_size n_splits = self.n_splits n_samples = _num_samples(X) if (n_samples - min_train_size) / test_size >= n_splits: print('(n_samples - min_train_size) / test_size >= n_splits') print('default TimeSeriesSplit.split() used') yield from super().split(X) else: shift = int(np.floor( (n_samples - test_size - min_train_size) / (n_splits - 1))) start_test = n_samples - (n_splits * shift + test_size - shift) test_starts = range(start_test, n_samples - test_size + 1, shift) if start_test < min_train_size: raise ValueError( ("The start of the testing : {0} is smaller" " than the minimum training samples: {1}.").format(start_test, min_train_size)) indices = np.arange(n_samples) for test_start in test_starts: if self.max_train_size and self.max_train_size < test_start: yield (indices[test_start - self.max_train_size:test_start], indices[test_start:test_start + test_size]) else: yield (indices[:test_start], indices[test_start:test_start + test_size])  And with the visualisation: import numpy as np import matplotlib.pyplot as plt from matplotlib.patches import Patch from ModelEvaluation import TimeSeriesSplitCustom np.random.seed(1338) cmap_data = plt.cm.Paired cmap_cv = plt.cm.coolwarm n_splits = 13 # Generate the class/group data n_points = 100 X = np.random.randn(100, 10) percentiles_classes = [.1, .3, .6] y = np.hstack([[ii] * int(100 * perc) for ii, perc in enumerate(percentiles_classes)]) # Evenly spaced groups repeated once groups = np.hstack([[ii] * 10 for ii in range(10)]) fig, ax = plt.subplots() cv = TimeSeriesSplitCustom(n_splits=n_splits, test_size=20, min_train_size=12) plot_cv_indices(cv, X, y, groups, ax, n_splits) plt.show()   (To have the same result, make sure to change the for ii, (tr, tt) in enumerate(**cv.overlapping_split**(X=X, y=y, groups=group)): in the plot_cv_indices function. Cheers!