Bug: InformerModel, decoder_input torch.cat size of tensor mismatch error otherwise
Author: jhzsquared
Pull request: no

Possible solutions: Should shift=0,  at L2020? 
Referencing: https://github.com/huggingface/transformers/blame/4fdf58afb72b0754da30037fc800b6044e7d9c99/src/transformers/models/informer/modeling_informer.py#L2020

I've trained/tested an Informer model but when generating the prediction, run into a "RuntimeError: Sizes of tensors must match except in Dimension 2..." when running line 2029 in `modeling_informer.py`.

I broke it apart a bit, and after playing around, it looks like shift=1 in Line 2020 may have been mistakenly hardcoded? Otherwise, the tensor shape for `reshaped_lagged_sequence` at dimension 1 will always be one less than `repeated_features`. 
Alternatively of course, `repeated_features` could not use k+1 in L2026.  I'm not clear on the author's intuition behind the shift vs not. 


Comments:
It seems like you are encountering a runtime error in the Informer model while generating predictions, specifically at line 2029 in `modeling_informer.py`. You have identified that the issue might be related to the `shift=1` parameter at line 2020, which could be causing the tensor shapes mismatch.

To address this issue, you can try the following possible solutions:

1. **Change `shift=1` at Line 2020 to `shift=0`:**
   - Update the `shift` parameter to `0` at line 2020 in `modeling_informer.py`. This change may help align tensor shapes correctly during prediction generation.

2. **Adjust `repeated_features` at Line 2026:**
   - Instead of using `k+1` in line 2026 for `repeated_features`, you can consider using `k` to ensure that the tensor shapes are consistent with `reshaped_lagged_sequence`.

3. **Seek clarification on the author's intent:**
   - If you are unsure about the author's reasoning behind the `shift` parameter or the use of `k+1`, you may want to refer to the documentation or reach out to the author or community for clarification on the intended design and behavior.

By implementing one of these solutions or a combination of them, you may be able to resolve the tensor shape mismatch issue and avoid the `RuntimeError` during prediction generation in the Informer model. Testing the changes and observing the model's behavior after the adjustments will help confirm if the issue has been effectively addressed.

In the context of the error you encountered while generating predictions with an Informer model, it seems that the issue may be related to the manipulation of tensor shapes and the specific handling of the "shift" parameter in the code snippet provided.

After reviewing the relevant portion of the code in the Transformers library's `modeling_informer.py` file and considering the observed error message, it appears that the discrepancy in tensor sizes leading to the RuntimeError could be attributed to the way the "shift" parameter is utilized within the code.

Here are some possible solutions and considerations for addressing this issue:

1. **Investigating the Shift Parameter (shift=1):**
   - The "shift" parameter in line 2020 of the code snippet you referenced may indeed impact the tensor shapes, potentially causing mismatches in dimension sizes during prediction generation.
   - By setting "shift=0" or adjusting the logic related to how the "shift" parameter is utilized, you may align the tensor sizes properly and eliminate the dimension mismatch issue.

2. **Reviewing Tensor Reshaping Logic:**
   - The reshaping of tensors, specifically concerning the `reshaped_lagged_sequence` and `repeated_features`, plays a critical role in maintaining consistent tensor sizes for subsequent operations.
   - Ensure that the reshaping logic in lines 2020 and 2026 appropriately handles the dimensions of the tensors involved to prevent size discrepancies.

3. **Author's Intuition on Shift Parameter:**
   - Understanding the author's intent behind incorporating the "shift" parameter in the code can provide insights into the rationale for using a specific value (such as "shift=1") and its impact on tensor operations.
   - Consider revisiting the documentation or comments related to the "shift" parameter within the code to clarify its intended purpose and how it influences tensor manipulations.

4. **Testing and Validation:**
   - Experiment with different values for the "shift" parameter (e.g., 0, 1) in line 2020 to observe how they affect tensor shapes and the overall prediction generation process.
   - Validate the model's behavior and predictions after making adjustments to the "shift" parameter to confirm that the dimension mismatch issue has been resolved.

In summary, by investigating the logic surrounding the "shift" parameter, reviewing the tensor reshaping procedures, and considering the author's reasoning behind its usage, you can work towards addressing the dimension mismatch error encountered during prediction generation with the Informer model. Experimenting with different values for "shift" and ensuring consistent tensor shapes can help resolve the RuntimeError and optimize the model's performance in generating accurate predictions.

cc @kashif 

@jhzsquared  so the intention was that the model is learning the next step's distribution given the past as well as the covariates up till the time step at which one is forecasting... 

can you paste in your `lag_seq` vectors that you are using?

I'm not using any lag right now, so have an initial model input `lags_sequence = [0]` .

And thanks! Conceptually that makes sense... functionally though, when `k=0`, the `get_lagged_subsequences` function with shift=1 is manifesting as a `context_length` size tensor at dimension 1, while the `repeated_features[:,k+1]` is always size `k+1` at dimension 1 of course. When `k>0`, then the `lagged_sequence` shape at dimension 1 is always 1 less (`-1`) the size of the corresponding subset of `repeated_features` it is supposed to be combined with at Line 2026.   

right so if you dont want lags set that array to [1], and increase you context lengh by 1 more time step...can you check if that works?

Ohh okay did not realize that should have been `[1]`. That fixed it! Thank you so much!

Hello @kashif , I have the similar issues but cannot solved by setting it to [1]. My task is to let model to only use the 10 history length to predict 1 future value.
```
prediction = model.generate(x_batch, past_time_features=time_batch, 
[/usr/local/lib/python3.10/dist-packages/transformers/models/informer/modeling_informer.py]in generate(self, past_values, past_time_features, future_time_features, past_observed_mask, static_categorical_features, static_real_features, output_attentions, output_hidden_states)
   2021             reshaped_lagged_sequence = lagged_sequence.reshape(lags_shape[0], lags_shape[1], -1)
   2022 
-> 2023             decoder_input = torch.cat((reshaped_lagged_sequence, repeated_features[:, : k + 1]), dim=-1)
   2024 
   2025             dec_output = decoder(inputs_embeds=decoder_input, encoder_hidden_states=repeated_enc_last_hidden)

RuntimeError: Sizes of tensors must match except in dimension 2. Expected size 10 but got size 1 for tensor number 1 in the list.
```

These are my configs.
```
InformerConfig {
  "activation_dropout": 0.1,
  "activation_function": "gelu",
  "attention_dropout": 0.1,
  "attention_type": "prob",
  "cardinality": [
    0
  ],
  "context_length": 10,
  "d_model": 64,
  "decoder_attention_heads": 2,
  "decoder_ffn_dim": 32,
  "decoder_layerdrop": 0.1,
  "decoder_layers": 3,
  "distil": true,
  "distribution_output": "student_t",
  "dropout": 0.1,
  "embedding_dimension": [
    0
  ],
  "encoder_attention_heads": 2,
  "encoder_ffn_dim": 32,
  "encoder_layerdrop": 0.1,
  "encoder_layers": 3,
  "feature_size": 5,
  "init_std": 0.02,
  "input_size": 1,
  "is_encoder_decoder": true,
  "lags_sequence": [
    0
  ],
  "loss": "nll",
  "model_type": "informer",
  "num_dynamic_real_features": 0,
  "num_parallel_samples": 100,
  "num_static_categorical_features": 0,
  "num_static_real_features": 0,
  "num_time_features": 5,
  "output_attention": false,
  "output_hidden_states": true,
  "prediction_length": 1,
  "sampling_factor": 5,
  "scaling": "mean",
  "transformers_version": "4.44.2",
  "use_cache": true
}
```

If I set lags_sequence to [1], I will get the following error.
```
forecast = model(x_batch, past_time_features = time_batch, past_observed_mask=mask_batch, future_values=y_batch, future_time_features=y_time_batch)

[/usr/local/lib/python3.10/dist-packages/transformers/models/informer/modeling_informer.py](https://localhost:8080/#) in get_lagged_subsequences(self, sequence, subsequences_length, shift)
   1487 
   1488         if max(indices) + subsequences_length > sequence_length:
-> 1489             raise ValueError(
   1490                 f"lags cannot go further than history length, found lag {max(indices)} "
   1491                 f"while history length is only {sequence_length}"

ValueError: lags cannot go further than history length, found lag 1 while history length is only 11
```
Could you kindly share how could I solve these issues? Should I set context_length to 9 and lags_seq to [1] and input 10 sequence data? I'm confused about why we need lags_seq and context_length separately.

Hello @kashif @amyeroberts , could you help with this issue? How the model learn the past 10 time sequence without lags_sequence

so you can set the lag_seq to `[1]` for example and then in the input context length add an extra time step at the start (e.g. if your context length is 10, then use a vector of 11 time steps, or context_length + max(lag_seq)) hope that helps!

Hi @kashif , thanks for your help, but I am comparing with another model using 10 past time sequence, isn't using a lag_seq [1] actually letting model using 11 past time sequence? I am afraid that this is not align with other benchmark's input. 

yes you can then make your context_length be 9 with lag_seq=[1] and then it should be equivalent

Thank you @kashif , but won't making the 10th sequence the extra feature (due to lag_seq) affect the learning of the model compared to letting all 10 time sequence in context length? I am wondering the difference of impact to model between lag_seq and context.

so in both cases one would need to go from a single number per time step to a vector representation of the inputs, so by having context length of 9 with lag index [1] means only information from a context window of size 10 is used to learn the input representation, which would be the same if another "tokenization" method is used for a context window of 10. So i don't think there will be any impact...