RandomForestRegressor() producing mainly constant forecast results over time-series data

### Describe the workflow you want to enable

Let's say I have [dataset](https://drive.google.com/file/d/18PGLNnOI44LVFignYriBWQFW9WBkTX5c/view?usp=share_link) contains a timestamp (non-standard timestamp column without datetime format) as a single feature and count as Label/target to predict within the following [pandas](https://stackoverflow.com/questions/tagged/pandas) dataframe format as follows:

```
   X        y
Timestamp label
+--------+-----+
|TS_24hrs|count|
+--------+-----+
|0       |157  |
|1       |334  |
|2       |176  |
|3       |86   |
|4       |89   |
 ...      ...
|270     |192  |
|271     |196  |
|270     |251  |
|273     |138  |
+--------+-----+
274 rows × 2 columns
```
I have already implemented RF regression within [sklearn](https://stackoverflow.com/questions/tagged/sklearn) pipeline() after splitting data with the following strategy for 274 records:

        split data into [training-set + validation-set] [Ref.](https://stackoverflow.com/a/59452915/10452700) e.g. The first 200 records [160 +40]
        keeping unseen [test-set] hold-on for final forecasting e.g. The last 74 records (after 200th rows\event)

```
#print(train.shape)          #(160, 2)
#print(validation.shape)     #(40, 2)
#print(test.shape)           #(74, 2)
```

I tried default pipeline as well as optimized one by tuning Hyper-parameters to get optimum results by equipping the RF pipeline with [GridSearchCV()](https://scikit-learn.org/stable/tutorial/statistical_inference/putting_together.html), however results didn't improve as follow:

```py
from sklearn.metrics import r2_score

print(f"r2 (defaults): {r2_score(test['count'], rf_pipeline2.predict(X_test))}")
print(f"r2 (opt.):     {r2_score(test['count'], rf_pipeline2o.predict(X_test))}")

#r2 (defaults): 0.025314471951056405
#r2 (opt.):     0.07593841572721849

img

Full code for reproducing the example:

# Load the time-series data as dataframe
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

df = pd.read_csv('/content/U2996_24hrs_.csv', sep=",")

# The first 200 records slice for training-set and validation-set
df200 = df[:200]          

# The rest records = 74 events (after 200th event) kept as hold-on unseen-set for forecasting
test = df[200:]   #test (keep it unseen)

# Split the data into training and testing sets
from sklearn.model_selection import train_test_split

X = df200[['TS_24hrs']]
y = df200['count']
X_train, X_val, y_train, y_val = train_test_split(X, y , test_size=0.2, shuffle=False, random_state=0)  #train + validat
X_test = test['count'].values.reshape(-1,1)


# Train and fit the RF model
from sklearn.ensemble import RandomForestRegressor
#rf_model = RandomForestRegressor(random_state=10).fit(train, train['count']) #X, y

# build an end-to-end pipeline, and supply the data into a regression model and train within pipeline. It avoids leaking the test\val-set into the train-set
from sklearn.preprocessing import MinMaxScaler
from sklearn.ensemble import RandomForestRegressor
from sklearn.pipeline import Pipeline, make_pipeline

# Pipeline (defaults) 
rf_pipeline2  = Pipeline([('scaler', MinMaxScaler()),('RF', RandomForestRegressor(random_state=10))]).fit(X_train,y_train)   #Approach 2 train-set excludes label

# Pipeline (optimum)
# Parameters of pipelines can be set using '__' separated parameter names:
from sklearn.model_selection import GridSearchCV
from sklearn.model_selection import TimeSeriesSplit

tscv = TimeSeriesSplit(n_splits = 5)
param_grid = {
    "RF__n_estimators": [10, 50, 100],
    "RF__max_depth":    [1, 5, 10, 25],
    "RF__max_features": [*np.arange(0.1, 1.1, 0.1)],}

rf_pipeline2o = Pipeline([('scaler', MinMaxScaler()),('RF', GridSearchCV(rf_pipeline2,
                                                                         param_grid=param_grid,
                                                                         n_jobs=2,
                                                                         cv=tscv,
                                                                         refit=True))]).fit(X_train,y_train) #Approach 2 train-set excludes label

# Displaying a Pipeline with a Preprocessing Step and Regression
from sklearn import set_config
set_config(display="text")

#print(rf_pipeline2) 
#print(rf_pipeline2o)

# Use the pipeline to predict over the validation-set and test-set

y_predictions_test2  = rf_pipeline2.predict(X_test)
y_predictions_test2o = rf_pipeline2o.predict(X_test)

y_predictions_val2   = rf_pipeline2.predict(X_val)
y_predictions_val2o  = rf_pipeline2o.predict(X_val)

# Convert forecast result over the test-set into dataframe for plot issue with ease
df_pre_test_rf2  = pd.DataFrame({'TS_24hrs':test['TS_24hrs'],        'count_forecast_test':y_predictions_test2})
df_pre_test_rf2o = pd.DataFrame({'TS_24hrs':test['TS_24hrs'],        'count_forecast_test':y_predictions_test2o})


# Convert predict result over the validation-set into dataframe for plot issue with ease
df_pre_val_rf2  = pd.DataFrame({'TS_24hrs':X_val['TS_24hrs'],        'count_prediction_val':y_predictions_val2})
df_pre_val_rf2o = pd.DataFrame({'TS_24hrs':X_val['TS_24hrs'],        'count_prediction_val':y_predictions_val2o})

# evaluate performance with MAE
# Evaluate performance by calculate the loss and metric over unseen test-set
from sklearn.metrics import mean_absolute_error, mean_squared_error, mean_absolute_percentage_error, explained_variance_score, r2_score 

rf_mae_test2  = mean_absolute_error(test['count'],                 df_pre_test_rf2['count_forecast_test'])
rf_mae_test2o = mean_absolute_error(test['count'],                 df_pre_test_rf2o['count_forecast_test'])

#visulize forecast or prediction of RF pipleine
import matplotlib.pyplot as plt
fig, ax = plt.subplots( figsize=(10,4))

pd.Series(y_train).plot(label='Training-set', c='b')
pd.Series(y_val).plot(label='Validation-set', linestyle=':', c='b')
test['count'].plot(label='Test-set (unseen)', c='cyan')

#predict plot over validation-set
df_pre_val_rf2['count_prediction_val'].plot(label=f'RF_predict_val (defaults)              ', linestyle='--', c='green',   marker="+")
df_pre_val_rf2o['count_prediction_val'].plot(label=f'RF_predict_val (opt.)       ', linestyle='--', c='purple',    marker="+", alpha= 0.4)

#forecast plot over test-set (unseen)
df_pre_test_rf2['count_forecast_test'].plot(label=f'RF_forecast_test (defaults)     MAE={rf_mae_test2:.2f}', linestyle='--', c='green',   marker="*")
df_pre_test_rf2o['count_forecast_test'].plot(label=f'RF_forecast_test (opt.)           MAE={rf_mae_test2o:.2f}', linestyle='--', c='purple',    marker="*", alpha= 0.4)

plt.legend()
plt.title('Plot of comparioson results of used implementation approaches trained RF pipeline ')
plt.ylabel('count', fontsize=15)
plt.xlabel('Timestamp [24hrs]', fontsize=15)
plt.show()

```


### Describe your proposed solution

I have implemented different approaches, but I have not figured out to debug the problem so far. In the past time, I had some issues with another regressor that outputs constant prediction, and I resolved them with Hyper-parameters tuning like this [post](https://stackoverflow.com/q/71285022/10452700).

Furthermore, based on this [answer](https://stats.stackexchange.com/a/235233/240550):

    "...random forest for regression / regression trees doesn't produce expected predictions for data points beyond the scope of training data range because they cannot extrapolate (well)."

Despite this could explain constant forecast concerning out-of-sample prediction over test-set (unseen), but still, I believe that even if it is the case, it should show a non-constant prediction over the validation-set in my case while it is constant.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
this is a usage question and not an issue about the library. You need to either use a more suitable model, like a model understanding periodicity e.g. a gaussian process with the right kernels, or other models. You can also consider `skforcast` or `sktime` as libraries more suitable for time series data.

Labels: New Feature, Needs Triage