IndexError: index out of range in self
Author: Bateoriginal
Pull request: no

My batch shape looked like below: 

- past_values: torch.Size([17, 35])
- past_time_features: torch.Size([17, 35, 9])
- past_observed_mask: torch.Size([17, 35])
- static_categorical_features: torch.Size([17, 4])
- static_real_features: torch.Size([17, 2])
- future_values: torch.Size([17, 7])
- future_time_features: torch.Size([17, 7, 9])

I run 

```
model = TimeSeriesTransformerModel.from_pretrained("huggingface/time-series-transformer-tourism-monthly")

# during training, one provides both past and future values
# as well as possible additional features

outputs = model(
    past_values=batchTrain["past_values"],
    past_time_features=batchTrain["past_time_features"],
    past_observed_mask=batchTrain["past_observed_mask"],
    static_categorical_features=batchTrain["static_categorical_features"],
    static_real_features=batchTrain["static_real_features"],
    future_values=batchTrain["future_values"],
    future_time_features=batchTrain["future_time_features"],
)

last_hidden_state = outputs.last_hidden_state
```
```

Below is the error message. 

Some weights of the model checkpoint at huggingface/time-series-transformer-tourism-monthly were not used when initializing TimeSeriesTransformerModel: ['parameter_projection.proj.2.weight', 'parameter_projection.proj.2.bias', 'parameter_projection.proj.1.weight', 'parameter_projection.proj.0.bias', 'parameter_projection.proj.0.weight', 'parameter_projection.proj.1.bias']
- This IS expected if you are initializing TimeSeriesTransformerModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing TimeSeriesTransformerModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Output exceeds the [size limit](command:workbench.action.openSettings?[). Open the full output data [in a text editor](command:workbench.action.openLargeOutput?fd064990-38d7-46ff-ab2b-610b0cd790a4)
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
Cell In[34], line 53
     48 model = TimeSeriesTransformerModel.from_pretrained("huggingface/time-series-transformer-tourism-monthly")
     50 # during training, one provides both past and future values
     51 # as well as possible additional features
---> 53 outputs = model(
     54     past_values=batchTrain["past_values"],
     55     past_time_features=batchTrain["past_time_features"],
     56     past_observed_mask=batchTrain["past_observed_mask"],
     57     static_categorical_features=batchTrain["static_categorical_features"],
     58     static_real_features=batchTrain["static_real_features"],
     59     future_values=batchTrain["future_values"],
     60     future_time_features=batchTrain["future_time_features"],
     61 )
     63 last_hidden_state = outputs.last_hidden_state

File ~/opt/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py:1501, in Module._call_impl(self, *args, **kwargs)
   1496 # If we don't have any hooks, we want to skip the rest of the logic in
   1497 # this function, and just call forward.
   1498 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   1499         or _global_backward_pre_hooks or _global_backward_hooks
   1500         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1501     return forward_call(*args, **kwargs)
   1502 # Do not call functions when jit is used
   1503 full_backward_hooks, non_full_backward_hooks = [], []

File ~/opt/anaconda3/lib/python3.8/site-packages/transformers/models/time_series_transformer/modeling_time_series_transformer.py:1417, in TimeSeriesTransformerModel.forward(self, past_values, past_time_features, past_observed_mask, static_categorical_features, static_real_features, future_values, future_time_features, decoder_attention_mask, head_mask, decoder_head_mask, cross_attn_head_mask, encoder_outputs, past_key_values, output_hidden_states, output_attentions, use_cache, return_dict)
   1414 use_cache = use_cache if use_cache is not None else self.config.use_cache
   1415 return_dict = return_dict if return_dict is not None else self.config.use_return_dict
-> 1417 transformer_inputs, loc, scale, static_feat = self.create_network_inputs(
   1418     past_values=past_values,
   1419     past_time_features=past_time_features,
   1420     past_observed_mask=past_observed_mask,
   1421     static_categorical_features=static_categorical_features,
   1422     static_real_features=static_real_features,
   1423     future_values=future_values,
   1424     future_time_features=future_time_features,
   1425 )
   1427 if encoder_outputs is None:
   1428     enc_input = transformer_inputs[:, : self.config.context_length, ...]

File ~/opt/anaconda3/lib/python3.8/site-packages/transformers/models/time_series_transformer/modeling_time_series_transformer.py:1324, in TimeSeriesTransformerModel.create_network_inputs(self, past_values, past_time_features, static_categorical_features, static_real_features, past_observed_mask, future_values, future_time_features)
   1322     static_feat = torch.cat((static_real_features, static_feat), dim=1)
   1323 if static_categorical_features is not None:
-> 1324     embedded_cat = self.embedder(static_categorical_features)
   1325     static_feat = torch.cat((embedded_cat, static_feat), dim=1)
   1326 expanded_static_feat = static_feat.unsqueeze(1).expand(-1, time_feat.shape[1], -1)

File ~/opt/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py:1501, in Module._call_impl(self, *args, **kwargs)
   1496 # If we don't have any hooks, we want to skip the rest of the logic in
   1497 # this function, and just call forward.
   1498 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   1499         or _global_backward_pre_hooks or _global_backward_hooks
   1500         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1501     return forward_call(*args, **kwargs)
   1502 # Do not call functions when jit is used
   1503 full_backward_hooks, non_full_backward_hooks = [], []

File ~/opt/anaconda3/lib/python3.8/site-packages/transformers/models/time_series_transformer/modeling_time_series_transformer.py:76, in TimeSeriesFeatureEmbedder.forward(self, features)
     72 else:
     73     cat_feature_slices = [features]
     75 return torch.cat(
---> 76     [
     77         embed(cat_feature_slice.squeeze(-1))
     78         for embed, cat_feature_slice in zip(self.embedders, cat_feature_slices)
     79     ],
     80     dim=-1,
     81 )

File ~/opt/anaconda3/lib/python3.8/site-packages/transformers/models/time_series_transformer/modeling_time_series_transformer.py:77, in <listcomp>(.0)
     72 else:
     73     cat_feature_slices = [features]
     75 return torch.cat(
     76     [
---> 77         embed(cat_feature_slice.squeeze(-1))
     78         for embed, cat_feature_slice in zip(self.embedders, cat_feature_slices)
     79     ],
     80     dim=-1,
     81 )

File ~/opt/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py:1501, in Module._call_impl(self, *args, **kwargs)
   1496 # If we don't have any hooks, we want to skip the rest of the logic in
   1497 # this function, and just call forward.
   1498 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   1499         or _global_backward_pre_hooks or _global_backward_hooks
   1500         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1501     return forward_call(*args, **kwargs)
   1502 # Do not call functions when jit is used
   1503 full_backward_hooks, non_full_backward_hooks = [], []

File ~/opt/anaconda3/lib/python3.8/site-packages/torch/nn/modules/sparse.py:162, in Embedding.forward(self, input)
    161 def forward(self, input: Tensor) -> Tensor:
--> 162     return F.embedding(
    163         input, self.weight, self.padding_idx, self.max_norm,
...
   2208     # remove once script supports set_grad_enabled
   2209     _no_grad_embedding_renorm_(weight, input, max_norm, norm_type)
-> 2210 return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)

IndexError: index out of range in self
```

Comments:
cc @kashif 

@Bateoriginal thank you for the report. Since you are using the model weights for the tourism dataset the cardinality of the embedding layers is fixed to that from this dataset. Thus it seems you are passing it some integer id which is too large with respect to what the model expects. 

May I ask if you are training with the tourism dataset?


Thank you for your response.

I'm currently working with retail transaction data.

It's reassuring to understand that the number of elements (cardinality) in the embedding layers remains constant.

Could you elaborate on how the cardinality of these embedding layers is determined based on the original data used for training? Is it possible to predict the cardinality just by examining the dimensions of the batch?

When you refer to an 'integer ID', does this pertain to static categorical/real features or something else?


@Bateoriginal yes the embedding layer will output a vector for a specific number of ids typically from 0, ... cardinality-1 and if given an id outside this range it will error out as it is internally a mapping from these ids to a vector.

this cardinality as mentioned is set for your specific problem and dataset and corresponds to static covariates, and thus the cardinality is not something that is predictable from the batch as a batch is some random collection of some time series within your batch and also you need to specify it when initializing your model.

So the cardinality is chosen at the start and has to remain fixed for the duration of the model's life cycle. This is both good and bad... it's good because for example this way the model can be given information about say the id of each time series in a dataset but it is bad as it constrains your model from only being able to do predictions on time series with a known id...

In any case, i encourage you to initialize a model with the configurations of your dataset rather than loading a model trained on the tourism dataset. If you can have a look at the blog post: https://huggingface.co/blog/time-series-transformers and try to replicate it for your dataset

Hopefully, that helps!

Thank you for your time! 

so what i meant to say is that you can chose to train your model without using the categorical covariates and sometimes such a model performs (paradoxically) better than with categorical covariates.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.