#27237
Author: oublalkhalid
Pull request: yes

#27237  Solved ✅

I suggest a resolution for situations where ``sequence_lags=[0]``. In such instances, the length of the context past sequence aligns with the overall sequence length. Let me provide some clarification on this matter

The issue stems from line `1230` in `modeling_time_series_transformer.py.` In cases where the lag parameter is set to 0, the index is assigned a value of -1. This results in only one data point being lagged, creating a discrepancy when using model.generate(). For example, if the size is 48, selecting a lag of [0] produces 49, rendering it unsuitable for model.generate().

```
sequence_length = sequence.shape[1]
indices = [lag - shift for lag in self.config.lags_sequence]
if max(indices) + subsequences_length > sequence_length:
    raise ValueError(
        f"lags cannot go further than history length, found lag {max(indices)} "
        f"while history length is only {sequence_length}"
    )
```
We can modify the code as shown below to rectify index 0 when negative values are encountered ✅:
```
sequence_length = sequence.shape[1]
# (Khalid Oublal) -> addressed the issue regarding the scenario where lag equals 0.
# The previous implementation was: indices = [lag - shift for lag in self.config.lags_sequence]
indices = [lag - shift if lag > 0 else 0 for lag in self.config.lags_sequence]
if max(indices) + subsequences_length > sequence_length:
    raise ValueError(
        f"lags cannot go further than history length, found lag {max(indices)} "
        f"while history length is only {sequence_length}"
    )
```
### Check DataLoader

In the analysis below, it's evident that there are no lags indicated by `sequence_lags=[0]`. The length of the context in this batch matches the provided context length.

![Screenshot 2024-01-29 at 21 58 42](https://github.com/huggingface/transformers/assets/76509145/0079c3a8-eed6-4d82-bb41-fc6dcaf04a5a)

### Confirming Training Status

Below, it's apparent that the training is progressing smoothly. Some additional print statements were added to verify that the lags are 0, implying the indices should be `[0]`.

![Screenshot 2024-01-29 at 21 54 03](https://github.com/huggingface/transformers/assets/76509145/f159c725-ff81-4a11-89e0-5ef59ac0763e)

### Generating with `model.generate()`

Now, with `sequence_lags=[0]`, we observe that predictions can be made without any issues.

![Screenshot 2024-01-29 at 21 55 31](https://github.com/huggingface/transformers/assets/76509145/dfe7915b-8859-41ad-9664-f72f6c4754c7)

Best,
khalid oublal

Comments:
Hey! do you need help adding tests here? 🤗 

Hi Arthur,

Yeah, is there any documentation available on how to add tests? Thanks in
advance.

Koublal,

Le ven. 23 févr. 2024 à 10:24, Arthur ***@***.***> a écrit :

> Hey! do you need help adding tests here? 🤗
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/pull/28766#issuecomment-1960984416>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ASHW7WIHS4SQK3OPKZC37DTYVBN6LAVCNFSM6AAAAABCQFBGTCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRQHE4DINBRGY>
> .
> You are receiving this because you authored the thread.Message ID:
> ***@***.***>
>


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.