`past_time_features` attribute for TimeSeriesTransformer is not optional
Author: simonMoisselin
Pull request: no

### System Info


Hello, 
I am trying to use `TimeSeriesTransformer` with `past_time_features=None` but I don't see anything in the code taking into account when this parameter is not defined, for example inside the method `create_network_inputs`:

https://github.com/huggingface/transformers/blob/v4.25.1/src/transformers/models/time_series_transformer/modeling_time_series_transformer.py#L1566

```
# time feature
      time_feat = (
          torch.cat(
              (
                  past_time_features[:, self._past_length - self.config.context_length :, ...],
                  future_time_features,
              ),
              dim=1,
          )
          if future_values is not None
          else past_time_features[:, self._past_length - self.config.context_length :, ...]
        )
```

We should either update the documentation to make it mandatory, or update `create_network_inputs` and construct the inputs differently if `past_Time_features` is not present?


### Who can help?

@LysandreJik  

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```python
number_qty = 3
number_features = 100
prediction_length = 10
context_length = 10

nrows = 15
static_real_features = np.zeros((nrows, 100))
future_values = ...
past_values = ...
configuration = TimeSeriesTransformerConfig(input_size=number_qty, 
                                            num_static_real_features=number_features,
                                            prediction_length=prediction_length,
                                            context_length=context_length,
                                            # past_time_features
                                           )

model = TimeSeriesTransformerModel(configuration)
# model
model.forward(past_values, 
              static_real_features=static_real_features, 
              future_values=future_values, 
              past_time_features=None,
              past_observed_mask=None,
              static_categorical_features=None
             )
```


### Expected behavior

We expect the `forward` method to works without past_time_features defined

Comments:
cc @kashif and @NielsRogge 

Thank you for the issue! My feeling was that, since the transformer is a permutation equivariant layer, time features should be mandatory. For the case when you do not have date times, you can add positional encoding of a size of your choosing.

What are your thoughts about this @simonMoisselin ?



This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Is it possible to reopen this.  

I'm personally in favor of having the interface support both datasets that have {past,future}_time_features and those that do not contain them.

However, it's not call.  But would it be possible to update the documentation, if it's not changed? In its current state it is misleading. https://huggingface.co/docs/transformers/model_doc/time_series_transformer#transformers.TimeSeriesTransformerModel.forward.past_observed_mask


thanks, @nathanhack just to confirm, the discussion above was about the `past_time_features`  being required and the like you have is for the observation mask... can you kindly clarify?

Correct. The link I gave was a mistake. past_observation_mask is right below past_time_feature. When I copied the link my page was correctly displaying past_time_features. Which is clearly wrong. Thank you for catching it and I'm sorry it was confusing. The correct link should have been:

https://huggingface.co/docs/transformers/model_doc/time_series_transformer#transformers.TimeSeriesTransformerModel.forward.past_time_features

Not that this is the right place but, but lags_sequence also says it's optional but it can't be None and can't be an empty list as it will cause an exception on the following line: https://github.com/huggingface/transformers/blob/75a208ef66c0176fc12a4c98922728ced5befbf9/src/transformers/models/time_series_transformer/modeling_time_series_transformer.py#L1445
self.config.context_length + max(self.config.lags_sequence)

@kashif we should probably make lags sequence optional as these are just additional features

so `lags_sequence` is set to optional since if you do not specify it, it will default to a pre-specified array, namely [1, 2, 3, 4, 5, 6, 7]. I believe you can just ignore this option, and everything should work... It serves to offset the input so that we train to predict the next time step, as well as the "output dim size" of a "token embedding" so that the input vectors have some dimension to them (especially in the univariate setting) and also allows us to trade-off sequence length with feature size. If you do not want lags you can set the `lags_sequence=[1]` for example.

> Correct. The link I gave was a mistake. past_observation_mask is right below past_time_feature. When I copied the link my page was correctly displaying past_time_features. Which is clearly wrong. Thank you for catching it and I'm sorry it was confusing. The correct link should have been:
> 
> https://huggingface.co/docs/transformers/model_doc/time_series_transformer#transformers.TimeSeriesTransformerModel.forward.past_time_features

Fixed in PR #21020