Repository: huggingface/transformers
Pull request: Fix condition for emitting warning when generation exceeds max model length

## Fix condition for emitting warning when generation exceeds max model length

### Purpose
The current transformers version prints inconsistent warnings when exceeding the max model length L.

For prefills of a prompt of max model length L (max_position_embeddings) no warning is printed. This is expected, as the positional embedding of the (L + 1)-th token, which exceeds the limit, would only be needed when doing another model forward pass to generate the (L + 2)-th token.

However, when doing decodes with a context length of max model length L (max_position_embeddings) the warning comes one token too early. 
Example: doing a prefill of (L - 1) tokens and requesting 2 tokens will generate the L-th token (on context length L-1) and the  (L + 1)-th token (on context length L), which is perfectly legitimate without exceeding the limit max_position_embeddings. Nevertheless, the current transformer version does emit the warning:
```
This is a friendly reminder - the current text generation call will exceed the model's predefined maximum length (L). Depending on the model, you may observe exceptions, performance degradation, or nothing at all.
```

This PR adjusts the limit for which this warning is logged and makes it consistently correct across prefills and decodes.
Specifically, the PR targets the general case where the prompt is of length (L - K) and the number of new tokens are (K + 1).


### Testing

The behavior can be tested with the following script.

Note the different input_output (prompt length, max new tokens) combinations:
The effect of the PR can be observed for for L = 2048 and K = 1 -> input_output = (L - K, K + 1) = (2047, 2). Without the change in this PR the warning is logged (wrong), but with the change is not emitted (correct). 
The rest of the the test cases below are untouched by the change. 

```
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "JackFram/llama-160m"

input_output = (2048, 1) # warning before PR NO / after PR NO
# input_output = (2048, 2) # warning before PR YES / after PR YES
# input_output = (2047, 1) # warning before PR NO / after PR NO
# input_output = (2047, 2) # warning before PR YES / after PR NO <--- EFFECT OF THIS PR
# input_output = (2047, 3) # warning before PR YES / after PR YES

prompt_len, max_new_tokens = input_output

model = AutoModelForCausalLM.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)
prompt = '1 0 '* (2048//4)

tokenized_prompt = tokenizer([prompt])["input_ids"][0][:prompt_len]

assert len(tokenized_prompt)==prompt_len

hf_input_tokens = torch.tensor(tokenized_prompt).unsqueeze(0)
print('input shape:', hf_input_tokens.shape)
hf_output = model.generate(hf_input_tokens,
                           do_sample=False,
                           min_new_tokens=max_new_tokens,
                           max_new_tokens=max_new_tokens,
                           return_dict_in_generate=True,
                           output_scores=True)
print('ouput shape ', hf_output.sequences.shape)
# decode output tokens after first removing input tokens (prompt)
hf_generated_tokens = hf_output.sequences[:, len(hf_input_tokens[0]):]
print(hf_generated_tokens.shape)
hf_generated_text = tokenizer.batch_decode(hf_generated_tokens)[0]
print('generated', len(hf_generated_tokens[0]), 'HF tokens')
print(f"\ngenerated tokens:\n {hf_generated_tokens!r}\n")
print(f"\ngenerated text:\n {hf_generated_text!r}\n")
print("-----------------------------------")
```

Changed files:
filename: src/transformers/generation/stopping_criteria.py
patch: @@ -76,9 +76,9 @@ def __init__(self, max_length: int, max_position_embeddings: Optional[int] = Non
     def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor, **kwargs) -> torch.BoolTensor:
         cur_len = input_ids.shape[1]
         is_done = cur_len >= self.max_length
-        if self.max_position_embeddings is not None and not is_done and cur_len >= self.max_position_embeddings:
+        if self.max_position_embeddings is not None and not is_done and cur_len > self.max_position_embeddings:
             logger.warning_once(
-                "This is a friendly reminder - the current text generation call will exceed the model's predefined "
+                "This is a friendly reminder - the current text generation call has exceeded the model's predefined "
                 f"maximum length ({self.max_position_embeddings}). Depending on the model, you may observe "
                 "exceptions, performance degradation, or nothing at all."
             )