Repository: ourownstory/neural_prophet

Issue:
When calling make_future_dataframe do not check for unique values of regressor
### Discussed in https://github.com/ourownstory/neural_prophet/discussions/1112

<div type='discussions-op-text'>

<sup>Originally posted by **sivikt** January 12, 2023</sup>
Hi All!

Great library - loving it so far.

I observe a strange thing. Starting from the 0.4.2 the function mode.make_future_dataframe(...) silently changes the model state: removing predefined regressors. 

Consider the code. It creates a model with 1 future regressor **r_lagged** and then trying to make a future using **make_future_dataframe** function. Result contains columns **r_lagged** but with NaNs for the last 5 past data points which is strange since the whole past is provided.

```python
## pip install neuralprophet==0.4.2

import numpy as np
import pandas as pd
from neuralprophet import NeuralProphet
from neuralprophet import set_random_seed

set_random_seed(0)

data_location = "https://raw.githubusercontent.com/ourownstory/neuralprophet-data/main/datasets/"

df = pd.read_csv(data_location + 'wp_log_peyton_manning.csv')
df['r_lagged'] = np.random.choice(5, len(df))
df.loc[df[-20:].index, 'r_lagged'] = 5

m = NeuralProphet(n_forecasts=10, n_lags=5, epochs=2)
m = m.add_future_regressor(name='r_lagged')
metrics = m.fit(df, freq='D')

last_date = pd.to_datetime(df["ds"].copy(deep=True).dropna()).sort_values().max()
future_dates = pd.date_range(start=last_date, periods=12, freq='D')
future_dates = future_dates[future_dates > last_date]
future_df = pd.DataFrame({"ds": future_dates})
future_df['r_lagged'] = np.random.choice(5, len(future_df))
future = m.make_future_dataframe(df=df, periods=10, regressors_df=future_df, n_historic_predictions=False)
print(future_df)
print(future)
```

| idx | ds | y | r_lagged |
| - | - | - | - |
| 0 | 2016-01-16  | 7.8172   |    NaN |
| 1 | 2016-01-17  |  9.2739   |    NaN |
| 2 | 2016-01-18 | 10.3338   |    NaN |
| 3 | 2016-01-19  |  9.1259    |   NaN |
| 4 | 2016-01-20 |  8.8914   |    NaN |
| 5 | 2016-01-21   |   NaN    |   0.0 |
| 6 | 2016-01-22   |   NaN    |   0.0 |
| 7 | 2016-01-23   |   NaN    |   1.0 |
| 8 | 2016-01-24   |   NaN    |   0.0 |
| 9 | 2016-01-25   |   NaN    |   4.0 |
| 10 | 2016-01-26  |    NaN   |    3.0 |
| 11 | 2016-01-27   |   NaN    |   3.0 |
| 12 | 2016-01-28  |    NaN   |    4.0 |
| 13 | 2016-01-29  |    NaN   |    2.0 |
| 14 | 2016-01-30  |    NaN   |    3.0 |

It is because of **check_single_dataframe** gets the last past historic data points where **r_lagged** has the only value 5.

```
check_single_dataframe, df_utils.py:455
check_dataframe, df_utils.py:519
_check_dataframe, forecaster.py:2155
_make_future_dataframe, forecaster.py:2765
make_future_dataframe, forecaster.py:1246
```

I do not quite understand why it so having trained model with multiple past regressor unique. Why simple utility function changes the model state, it was not the case in the previous versions. One can see that **check_single_dataframe** marks a regressor as "to be removed" if the regressor has only 1 unique value but it considers only N past points (N lags) putting aside the **future_df**.

What do you think? Should we consider opening an issue/PR to adjust the behavior or there some hidden algorithm intrinsics which require such model state manipulation?  </div>


Discussion:

Resolving patch:
diff --git a/neuralprophet/df_utils.py b/neuralprophet/df_utils.py
--- a/neuralprophet/df_utils.py
+++ b/neuralprophet/df_utils.py
@@ -208,9 +208,13 @@ def data_params_definition(
         for covar in config_lagged_regressors.keys():
             if covar not in df.columns:
                 raise ValueError(f"Lagged regressor {covar} not found in DataFrame.")
+            norm_type_lag = config_lagged_regressors[covar].normalize
+            if local_run_despite_global:
+                if len(df[covar].unique()) < 2:
+                    norm_type_lag = "soft"
             data_params[covar] = get_normalization_params(
                 array=df[covar].values,
-                norm_type=config_lagged_regressors[covar].normalize,
+                norm_type=norm_type_lag,
             )
 
     if config_regressors is not None:
@@ -457,6 +461,13 @@ def check_single_dataframe(df, check_y, covariates, regressors, events, seasonal
                     "Encountered future regressor with only unique values in training set. "
                     "Variable will be removed for global modeling if this is true for all time series."
                 )
+    if covariates is not None:
+        for covar in covariates:
+            if len(df[covar].unique()) < 2:
+                log.warning(
+                    "Encountered lagged regressor with only unique values in training set. "
+                    "Variable will be removed for global modeling if this is true for all time series."
+                )
 
     columns = []
     if check_y:
@@ -503,7 +514,13 @@ def check_single_dataframe(df, check_y, covariates, regressors, events, seasonal
 
 
 def check_dataframe(
-    df: pd.DataFrame, check_y: bool = True, covariates=None, regressors=None, events=None, seasonalities=None
+    df: pd.DataFrame,
+    check_y: bool = True,
+    covariates=None,
+    regressors=None,
+    events=None,
+    seasonalities=None,
+    future: Optional[bool] = None,
 ) -> Tuple[pd.DataFrame, List]:
     """Performs basic data sanity checks and ordering,
     as well as prepare dataframe for fitting or predicting.
@@ -523,6 +540,8 @@ def check_dataframe(
             event column names
         seasonalities : list or dict
             seasonalities column names
+        future : bool
+            if df is a future dataframe
 
     Returns
     -------
@@ -545,6 +564,16 @@ def check_dataframe(
                     "Automatically removed variable."
                 )
                 regressors_to_remove.append(reg)
+    if future:
+        return checked_df, regressors_to_remove
+    if covariates is not None:
+        for covar in covariates:
+            if len(df[covar].unique()) < 2:
+                log.warning(
+                    "Encountered lagged regressor with only unique values in training set across all IDs."
+                    "Automatically removed variable."
+                )
+                regressors_to_remove.append(covar)
     if len(regressors_to_remove) > 0:
         regressors_to_remove = list(set(regressors_to_remove))
         checked_df = checked_df.drop(*regressors_to_remove, axis=1)


Test patch:
diff --git a/tests/test_integration.py b/tests/test_integration.py
--- a/tests/test_integration.py
+++ b/tests/test_integration.py
@@ -960,27 +960,35 @@ def test_global_modeling_with_lagged_regressors():
     df2["A"] = df2["y"].rolling(10, min_periods=1).mean()
     df3["A"] = df3["y"].rolling(40, min_periods=1).mean()
     df4["A"] = df4["y"].rolling(20, min_periods=1).mean()
+    df5 = df2.copy(deep=True)
+    df5["A"] = 1
+    df6 = df4.copy(deep=True)
+    df6["A"] = 1
     df1["ID"] = "df1"
     df2["ID"] = "df2"
     df3["ID"] = "df1"
     df4["ID"] = "df2"
+    df5["ID"] = "df2"
+    df6["ID"] = "df2"
     future_regressors_df3 = pd.DataFrame(data={"A": df3["A"].iloc[:30]})
     future_regressors_df4 = pd.DataFrame(data={"A": df4["A"].iloc[:40]})
     future_regressors_df3["ID"] = "df1"
     future_regressors_df4["ID"] = "df2"
-    train_input = {0: df1, 1: pd.concat((df1, df2)), 2: pd.concat((df1, df2))}
-    test_input = {0: df3, 1: df3, 2: pd.concat((df3, df4))}
+    train_input = {0: df1, 1: pd.concat((df1, df2)), 2: pd.concat((df1, df2)), 3: pd.concat((df1, df5))}
+    test_input = {0: df3, 1: df3, 2: pd.concat((df3, df4)), 3: pd.concat((df3, df6))}
     regressors_input = {
         0: future_regressors_df3,
         1: future_regressors_df3,
         2: pd.concat((future_regressors_df3, future_regressors_df4)),
+        3: pd.concat((future_regressors_df3, future_regressors_df4)),
     }
     info_input = {
         0: "Testing single ts df train / single ts df test - single df regressors, no events",
         1: "Testing many ts df train / many ts df test - single df regressors, no events",
         2: "Testing many ts df train / many ts df test - many df regressors, no events",
+        3: "Testing lagged regressor with only unique values",
     }
-    for i in range(0, 3):
+    for i in range(0, 4):
         log.info(info_input[i])
         m = NeuralProphet(
             n_lags=5,
@@ -990,6 +998,7 @@ def test_global_modeling_with_lagged_regressors():
             learning_rate=LR,
             trend_global_local="global",
             season_global_local="global",
+            global_normalization=True if i == 3 else False,
         )
         m = m.add_lagged_regressor(names="A")
         metrics = m.fit(train_input[i], freq="D")