Repository: ourownstory/neural_prophet

Issue:
drop() got multiple values for argument 'axis'
### Discussed in https://github.com/ourownstory/neural_prophet/discussions/1220

<div type='discussions-op-text'>

<sup>Originally posted by **kzaho** March 16, 2023</sup>
F-n check_dataframe at (https://github.com/ourownstory/neural_prophet/blob/main/neuralprophet/df_utils.py) fails.
Is there an extra star symbol, which should be deleted?
```
regressors_to_remove = list(set(regressors_to_remove))
checked_df = checked_df.drop(*regressors_to_remove, axis=1)
```
[fix] Resolve "drop() got multiple values for argument 'axis'" error
#1227 


Discussion:
@kzaho thanks for reporting this. do you have a sample for us to reproduce the error?
Let's pretend that we work with such data:
`{'ds': {0: Timestamp('2022-10-16 00:00:00'),
  1: Timestamp('2022-10-17 00:00:00'),
  2: Timestamp('2022-10-18 00:00:00'),
  3: Timestamp('2022-10-19 00:00:00'),
  4: Timestamp('2022-10-20 00:00:00'),
  5: Timestamp('2022-10-21 00:00:00'),
  6: Timestamp('2022-10-22 00:00:00'),
  7: Timestamp('2022-10-23 00:00:00'),
  8: Timestamp('2022-10-24 00:00:00'),
  9: Timestamp('2022-10-25 00:00:00')},
 'y': {0: 17, 1: 18, 2: 10, 3: 11, 4: 5, 5: 12, 6: 7, 7: 20, 8: 25, 9: 10},
 'price': {0: 3.5,
  1: 3.5,
  2: 3.5,
  3: 3.5,
  4: 3.5,
  5: 3.5,
  6: 3.5,
  7: 3.5,
  8: 3.5,
  9: 3.5},
 'cost': {0: 2.5,
  1: 2.5,
  2: 2.5,
  3: 2.5,
  4: 2.5,
  5: 2.5,
  6: 2.5,
  7: 2.5,
  8: 2.5,
  9: 2.5}}`

I come up with a temporary solution, to explain how it works - we have to review a couple of things:

1st, 'neuralprophet/forecaster.py' - line 2979, "df = df[-(self.max_lags + n_historic_predictions) :]"
so, in case we have trained model with max_lags = 3 and n_historic_predictions = 0,
we will have df with 3 obs.
`{'ds': {0: Timestamp('2022-10-23 00:00:00'),
  1: Timestamp('2022-10-24 00:00:00'),
  2: Timestamp('2022-10-25 00:00:00')},
 'y': {0: 20, 1: 25, 2: 10},
 'price': {0: 3.5, 1: 3.5, 2: 3.5},
 'cost': {0: 2.5, 1: 2.5, 2: 2.5}}`

2nd, this df with 3 obs, would be analyzed and at the line 542 'neuralprophet/df_utils.py'.
In case, our columns ('price' and 'cost') would consist of 1 unique value, these columns would be
appended to 'regressors_to_remove' list. Which leads to the error.

Solution (temporary): slightly modify the last value for both columns ('price' and 'cost').
`df.iloc[-1:, 2:] = df.iloc[-1:, 2:]*1.0001`
As, we have fixed data structure with column names ['ds', 'y', 'price', 'cost'], we can put index '2:'.

@noxan are you working on this?
@leoniewgnr @kzaho I've submitted a pull request as proposed by you @kzaho, many thanks for the hint. I still receive a "list index out of range" error with the adjusted code, could you please double check?

I've also attached a jupyter notebook with my test code so we can discuss the problem with the same example code. It is all based on your example @kzaho, many thanks!

[issue-1227.ipynb.zip](https://github.com/ourownstory/neural_prophet/files/11181668/issue-1227.ipynb.zip)

@Pushkaran-P It does not make any difference for fitting the model, you're all good - only the logging message is misleading.
Hi @kzaho @noxan @leoniewgnr,
NeuralProphet generally does not allow a column that is used as a future regressor and filled with unique values (only exception: the df consists of multiple time series and one time series uses future regressors with only unique values). 
We call a column of unique values that is used as model input `Static Covariates`, which is a component we are currently adding to the framework. 

With that being said, the error message definitely is misleading. I will add a PR that removes the future regressors in a clean way, so that the code runs through without an error (and without using regressors). 
## Model Benchmark

| Benchmark     | Metric   |      main |   current | diff   |                    |
|---------------|----------|-----------|-----------|--------|--------------------|
| AirPassengers | MAE_val  |  15.2698  |  15.2698  | 0.0%   | :white_check_mark: |
| AirPassengers | RMSE_val |  19.4209  |  19.4209  | 0.0%   | :white_check_mark: |
| AirPassengers | Loss_val |   0.00195 |   0.00195 | 0.0%   | :white_check_mark: |
| AirPassengers | MAE      |   9.82902 |   9.82902 | 0.0%   | :white_check_mark: |
| AirPassengers | RMSE     |  11.7005  |  11.7005  | 0.0%   | :white_check_mark: |
| AirPassengers | Loss     |   0.00056 |   0.00056 | 0.0%   | :white_check_mark: |
| AirPassengers | time     |   4.66743 |   6.36    | 36.26% | :x:                |
| YosemiteTemps | MAE_val  |   1.71298 |   1.71298 | -0.0%  | :white_check_mark: |
| YosemiteTemps | RMSE_val |   2.2607  |   2.26069 | -0.0%  | :white_check_mark: |
| YosemiteTemps | Loss_val |   0.00095 |   0.00095 | -0.0%  | :white_check_mark: |
| YosemiteTemps | MAE      |   1.45187 |   1.45187 | 0.0%   | :white_check_mark: |
| YosemiteTemps | RMSE     |   2.16721 |   2.16721 | -0.0%  | :white_check_mark: |
| YosemiteTemps | Loss     |   0.00066 |   0.00066 | -0.0%  | :white_check_mark: |
| YosemiteTemps | time     | 115.812   | 146.81    | 26.77% | :x:                |
| PeytonManning | MAE_val  |   0.58159 |   0.58159 | 0.0%   | :white_check_mark: |
| PeytonManning | RMSE_val |   0.72216 |   0.72216 | 0.0%   | :white_check_mark: |
| PeytonManning | Loss_val |   0.01239 |   0.01239 | 0.0%   | :white_check_mark: |
| PeytonManning | MAE      |   0.41671 |   0.41671 | 0.0%   | :white_check_mark: |
| PeytonManning | RMSE     |   0.55961 |   0.55961 | -0.0%  | :white_check_mark: |
| PeytonManning | Loss     |   0.00612 |   0.00612 | -0.0%  | :white_check_mark: |
| PeytonManning | time     |  13.6127  |  17.26    | 26.79% | :x:                |

<details>
<summary>Model training plots</summary>

## Model Training

### PeytonManning

![](https://asset.cml.dev/aa0e6f931321d0a7d0dc62d2feb9fdfda54f3638?cml=svg%2Bxml&cache-bypass=d2698af3-b26e-4b91-95bf-0b7e86a84277)

### YosemiteTemps

![](https://asset.cml.dev/438a04b76bc0e38eb146a649d9cf525bc31f45a6?cml=svg%2Bxml&cache-bypass=b8c609bd-8bc3-47ff-b29a-2d0a4582c138)

### AirPassengers

![](https://asset.cml.dev/d699bb9bce78079ad9f4207d4784e90e1cc29719?cml=svg%2Bxml&cache-bypass=dcd2e267-4a2e-416f-974c-9a78bcaa3ed9)

</details>

![](https://cml.dev/watermark.png#618d908bcc054bad3543de3219b10226f849350d "CML watermark")

## [Codecov](https://codecov.io/gh/ourownstory/neural_prophet/pull/1250?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Oskar+Triebe) Report
> Merging [#1250](https://codecov.io/gh/ourownstory/neural_prophet/pull/1250?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Oskar+Triebe) (4dadeeb) into [main](https://codecov.io/gh/ourownstory/neural_prophet/commit/f61527a454fb94f672f9c89cc5f3fb419224cda9?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Oskar+Triebe) (f61527a) will **not change** coverage.
> The diff coverage is `100.00%`.

:mega: This organization is not using Codecov’s [GitHub App Integration](https://github.com/apps/codecov). We recommend you install it so Codecov can continue to function properly for your repositories. [Learn more](https://about.codecov.io/blog/codecov-is-updating-its-github-integration/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Oskar+Triebe)

```diff
@@           Coverage Diff           @@
##             main    #1250   +/-   ##
=======================================
  Coverage   89.56%   89.56%           
=======================================
  Files          35       35           
  Lines        4830     4830           
=======================================
  Hits         4326     4326           
  Misses        504      504           
```


| [Impacted Files](https://codecov.io/gh/ourownstory/neural_prophet/pull/1250?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Oskar+Triebe) | Coverage Δ | |
|---|---|---|
| [neuralprophet/df\_utils.py](https://codecov.io/gh/ourownstory/neural_prophet/pull/1250?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Oskar+Triebe#diff-bmV1cmFscHJvcGhldC9kZl91dGlscy5weQ==) | `94.80% <100.00%> (ø)` | |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Oskar+Triebe). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Oskar+Triebe)

Thank you @noxan for tackling this!

Running the notebook on branch 'issue-1227' I got the following error in your final cell
```
WARNING - (NP.forecaster.fit) - When Global modeling with local normalization, metrics are displayed in normalized scale.
WARNING - (NP.df_utils.check_single_dataframe) - Encountered future regressor with only unique values in training set. Variable will be removed for global modeling if this is true for all time series.
WARNING - (NP.df_utils.check_single_dataframe) - Encountered future regressor with only unique values in training set. Variable will be removed for global modeling if this is true for all time series.
WARNING - (NP.df_utils.check_dataframe) - Encountered future regressor with only unique values in training set across all IDs.Automatically removed variable.
WARNING - (NP.df_utils.check_dataframe) - Encountered future regressor with only unique values in training set across all IDs.Automatically removed variable.
WARNING - (NP.forecaster._check_dataframe) - Removing regressor price because it is not present in the data.
WARNING - (NP.forecaster._check_dataframe) - Removing regressor cost because it is not present in the data.
INFO - (NP.df_utils._infer_frequency) - Major frequency D corresponds to 90.0% of the data.
INFO - (NP.df_utils._infer_frequency) - Dataframe freq automatically defined as D
INFO - (NP.config.init_data_params) - Setting normalization to global as only one dataframe provided for training.
INFO - (NP.utils.set_auto_seasonalities) - Disabling yearly seasonality. Run NeuralProphet with yearly_seasonality=True to override this.
INFO - (NP.utils.set_auto_seasonalities) - Disabling weekly seasonality. Run NeuralProphet with weekly_seasonality=True to override this.
INFO - (NP.utils.set_auto_seasonalities) - Disabling daily seasonality. Run NeuralProphet with daily_seasonality=True to override this.

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
Input In [4], in <cell line: 6>()
      4 m.add_future_regressor('price')
      5 m.add_future_regressor('cost')
----> 6 m.fit(df)

File ~/github/neural_prophet/neuralprophet/forecaster.py:892, in NeuralProphet.fit(self, df, freq, validation_df, epochs, batch_size, learning_rate, early_stopping, minimal, metrics, progress, checkpointing, continue_training, num_workers)
    890 # Training
    891 if validation_df is None:
--> 892     metrics_df = self._train(
    893         df,
    894         progress_bar_enabled=bool(progress),
    895         metrics_enabled=bool(self.metrics),
    896         checkpointing_enabled=checkpointing,
    897         continue_training=continue_training,
    898         num_workers=num_workers,
    899     )
    900 else:
    901     df_val, _, _, _ = df_utils.prep_or_copy_df(validation_df)

File ~/github/neural_prophet/neuralprophet/forecaster.py:2754, in NeuralProphet._train(self, df, df_val, progress_bar_enabled, metrics_enabled, checkpointing_enabled, continue_training, num_workers)
   2752 # Set up data the training dataloader
   2753 df, _, _, _ = df_utils.prep_or_copy_df(df)
-> 2754 train_loader = self._init_train_loader(df, num_workers)
   2755 dataset_size = len(df)  # train_loader.dataset
   2757 # Internal flag to check if validation is enabled

File ~/github/neural_prophet/neuralprophet/forecaster.py:2688, in NeuralProphet._init_train_loader(self, df, num_workers)
   2685 if self.config_country_holidays is not None:
   2686     self.config_country_holidays.init_holidays(df_merged)
-> 2688 dataset = self._create_dataset(
   2689     df, predict_mode=False, prediction_frequency=self.prediction_frequency
   2690 )  # needs to be called after set_auto_seasonalities
   2692 # Determine the max_number of epochs
   2693 self.config_train.set_auto_batch_epoch(n_data=len(dataset))

File ~/github/neural_prophet/neuralprophet/forecaster.py:2350, in NeuralProphet._create_dataset(self, df, predict_mode, prediction_frequency)
   2328 """Construct dataset from dataframe.
   2329 
   2330 (Configured Hyperparameters can be overridden by explicitly supplying them.
   (...)
   2347     TimeDataset
   2348 """
   2349 df, _, _, _ = df_utils.prep_or_copy_df(df)
-> 2350 return time_dataset.GlobalTimeDataset(
   2351     df,
   2352     predict_mode=predict_mode,
   2353     n_lags=self.n_lags,
   2354     n_forecasts=self.n_forecasts,
   2355     predict_steps=self.predict_steps,
   2356     config_seasonality=self.config_seasonality,
   2357     config_events=self.config_events,
   2358     config_country_holidays=self.config_country_holidays,
   2359     config_lagged_regressors=self.config_lagged_regressors,
   2360     config_regressors=self.config_regressors,
   2361     config_missing=self.config_missing,
   2362     prediction_frequency=prediction_frequency,
   2363 )

File ~/github/neural_prophet/neuralprophet/time_dataset.py:34, in GlobalTimeDataset.__init__(self, df, **kwargs)
     32 self.length = 0
     33 for df_name, df_i in df.groupby("ID"):
---> 34     timedataset = TimeDataset(df_i, df_name, **kwargs)
     35     self.length += timedataset.length
     36     for i in range(0, len(timedataset)):

File ~/github/neural_prophet/neuralprophet/time_dataset.py:70, in TimeDataset.__init__(self, df, name, **kwargs)
     68 self.init_after_tabularized(inputs, targets)
     69 self.filter_samples_after_init(kwargs["prediction_frequency"])
---> 70 self.drop_nan_after_init(df, kwargs["predict_steps"], drop_missing)

File ~/github/neural_prophet/neuralprophet/time_dataset.py:86, in TimeDataset.drop_nan_after_init(self, df, predict_steps, drop_missing)
     83 if key in self.two_level_inputs or key == "events" or key == "regressors":
     84     # Extract tensor out of OrderedDict to see if it contains NaNs
     85     tuple_list = list(data.items())
---> 86     tensor = tuple_list[0][1]
     87     if np.isnan(np.array(tensor)).any() and (i not in nan_idx):
     88         nan_idx.append(i)

IndexError: list index out of range
```

@ourownstory yeah, thanks for confirming. that's exactly the one I wrote about in #1227 ... 🤔 

Resolving patch:
diff --git a/neuralprophet/data/process.py b/neuralprophet/data/process.py
--- a/neuralprophet/data/process.py
+++ b/neuralprophet/data/process.py
@@ -299,7 +299,7 @@ def _check_dataframe(
             checked dataframe
     """
     df, _, _, _ = df_utils.prep_or_copy_df(df)
-    df, regressors_to_remove = df_utils.check_dataframe(
+    df, regressors_to_remove, lag_regressors_to_remove = df_utils.check_dataframe(
         df=df,
         check_y=check_y,
         covariates=model.config_lagged_regressors if exogenous else None,
@@ -312,6 +312,14 @@ def _check_dataframe(
         for reg in regressors_to_remove:
             log.warning(f"Removing regressor {reg} because it is not present in the data.")
             model.config_regressors.pop(reg)
+        if len(model.config_regressors) == 0:
+            model.config_regressors = None
+    if model.config_lagged_regressors is not None:
+        for reg in lag_regressors_to_remove:
+            log.warning(f"Removing lagged regressor {reg} because it is not present in the data.")
+            model.config_lagged_regressors.pop(reg)
+        if len(model.config_lagged_regressors) == 0:
+            model.config_lagged_regressors = None
     return df
 
 
diff --git a/neuralprophet/df_utils.py b/neuralprophet/df_utils.py
--- a/neuralprophet/df_utils.py
+++ b/neuralprophet/df_utils.py
@@ -521,7 +521,7 @@ def check_dataframe(
     events=None,
     seasonalities=None,
     future: Optional[bool] = None,
-) -> Tuple[pd.DataFrame, List]:
+) -> Tuple[pd.DataFrame, List, List]:
     """Performs basic data sanity checks and ordering,
     as well as prepare dataframe for fitting or predicting.
 
@@ -556,6 +556,7 @@ def check_dataframe(
         df_aux["ID"] = df_name
         checked_df = pd.concat((checked_df, df_aux), ignore_index=True)
     regressors_to_remove = []
+    lag_regressors_to_remove = []
     if regressors is not None:
         for reg in regressors:
             if len(df[reg].unique()) < 2:
@@ -564,8 +565,6 @@ def check_dataframe(
                     "Automatically removed variable."
                 )
                 regressors_to_remove.append(reg)
-    if future:
-        return checked_df, regressors_to_remove
     if covariates is not None:
         for covar in covariates:
             if len(df[covar].unique()) < 2:
@@ -573,12 +572,18 @@ def check_dataframe(
                     "Encountered lagged regressor with only unique values in training set across all IDs."
                     "Automatically removed variable."
                 )
-                regressors_to_remove.append(covar)
+                lag_regressors_to_remove.append(covar)
+    if future:
+        return checked_df, regressors_to_remove, lag_regressors_to_remove
     if len(regressors_to_remove) > 0:
         regressors_to_remove = list(set(regressors_to_remove))
-        checked_df = checked_df.drop(*regressors_to_remove, axis=1)
+        checked_df = checked_df.drop(regressors_to_remove, axis=1)
+        assert checked_df is not None
+    if len(lag_regressors_to_remove) > 0:
+        lag_regressors_to_remove = list(set(lag_regressors_to_remove))
+        checked_df = checked_df.drop(lag_regressors_to_remove, axis=1)
         assert checked_df is not None
-    return checked_df, regressors_to_remove
+    return checked_df, regressors_to_remove, lag_regressors_to_remove
 
 
 def _crossvalidation_split_df(df, n_lags, n_forecasts, k, fold_pct, fold_overlap_pct=0.0):


Test patch:
diff --git a/tests/test_integration.py b/tests/test_integration.py
--- a/tests/test_integration.py
+++ b/tests/test_integration.py
@@ -49,7 +49,7 @@ def test_train_eval_test():
         learning_rate=LR,
     )
     df = pd.read_csv(PEYTON_FILE, nrows=95)
-    df, _ = df_utils.check_dataframe(df, check_y=False)
+    df, _, _ = df_utils.check_dataframe(df, check_y=False)
     df = _handle_missing_data(m, df, freq="D", predicting=False)
     df_train, df_test = m.split_df(df, freq="D", valid_p=0.1)
     metrics = m.fit(df_train, freq="D", validation_df=df_test)
@@ -61,7 +61,7 @@ def test_train_eval_test():
 def test_df_utils_func():
     log.info("testing: df_utils Test")
     df = pd.read_csv(PEYTON_FILE, nrows=95)
-    df, _ = df_utils.check_dataframe(df, check_y=False)
+    df, _, _ = df_utils.check_dataframe(df, check_y=False)
 
     # test find_time_threshold
     df, _, _, _ = df_utils.prep_or_copy_df(df)
@@ -1631,3 +1631,18 @@ def test_selective_forecasting():
     )
     metrics_df = m.fit(df, freq="H")
     forecast = m.predict(df)
+
+
+def test_unused_future_regressors():
+    df = pd.DataFrame(
+        {
+            "ds": {0: "2022-10-16 00:00:00", 1: "2022-10-17 00:00:00", 2: "2022-10-18 00:00:00"},
+            "y": {0: 17, 1: 18, 2: 10},
+            "price": {0: 3.5, 1: 3.5, 2: 3.5},
+            "cost": {0: 2.5, 1: 2.5, 2: 2.5},
+        }
+    )
+    m = NeuralProphet(epochs=1, learning_rate=0.01)
+    m.add_future_regressor("price")
+    m.add_lagged_regressor("cost")
+    m.fit(df, freq="D")
diff --git a/tests/test_regularization.py b/tests/test_regularization.py
--- a/tests/test_regularization.py
+++ b/tests/test_regularization.py
@@ -57,7 +57,7 @@ def test_reg_func_abs():
 def test_regularization_holidays():
     log.info("testing: regularization of holidays")
     df = generate_holiday_dataset(y_holidays_override=Y_HOLIDAYS_OVERRIDE)
-    df, _ = df_utils.check_dataframe(df, check_y=False)
+    df, _, _ = df_utils.check_dataframe(df, check_y=False)
 
     m = NeuralProphet(
         epochs=20,
@@ -89,7 +89,7 @@ def test_regularization_holidays():
 def test_regularization_events():
     log.info("testing: regularization of events")
     df, events = generate_event_dataset(y_events_override=Y_EVENTS_OVERRIDE)
-    df, _ = df_utils.check_dataframe(df, check_y=False)
+    df, _, _ = df_utils.check_dataframe(df, check_y=False)
 
     m = NeuralProphet(
         epochs=50,
@@ -141,7 +141,7 @@ def test_regularization_lagged_regressor():
     """
     log.info("testing: regularization lagged regressors")
     df, lagged_regressors = generate_lagged_regressor_dataset(periods=100)
-    df, _ = df_utils.check_dataframe(df, check_y=False)
+    df, _, _ = df_utils.check_dataframe(df, check_y=False)
 
     m = NeuralProphet(
         epochs=30,
diff --git a/tests/test_unit.py b/tests/test_unit.py
--- a/tests/test_unit.py
+++ b/tests/test_unit.py
@@ -78,7 +78,7 @@ def test_time_dataset():
     config_missing = configure.MissingDataHandling()
     df_train, df_val = df_utils.split_df(df_in, n_lags, n_forecasts, valid_p)
     # create a tabularized dataset from time series
-    df, _ = df_utils.check_dataframe(df_train)
+    df, _, _ = df_utils.check_dataframe(df_train)
     local_data_params, global_data_params = df_utils.init_data_params(df=df, normalize="minmax")
     df = df.drop("ID", axis=1)
     df = df_utils.normalize(df, global_data_params)
@@ -218,7 +218,7 @@ def check_split(df_in, df_len_expected, n_lags, n_forecasts, freq, p=0.1):
             n_lags=n_lags,
             n_forecasts=n_forecasts,
         )
-        df_in, _ = df_utils.check_dataframe(df_in, check_y=False)
+        df_in, _, _ = df_utils.check_dataframe(df_in, check_y=False)
         df_in = _handle_missing_data(m, df_in, freq=freq, predicting=False)
         assert df_len_expected == len(df_in)
         total_samples = len(df_in) - n_lags - 2 * n_forecasts + 2
@@ -816,7 +816,7 @@ def test_too_many_NaN():
         limit_linear=config_missing.impute_linear,
         rolling=config_missing.impute_rolling,
     )
-    df, _ = df_utils.check_dataframe(df)
+    df, _, _ = df_utils.check_dataframe(df)
     local_data_params, global_data_params = df_utils.init_data_params(df=df, normalize="minmax")
     df = df.drop("ID", axis=1)
     df = df_utils.normalize(df, global_data_params)