`max_length` and `stopping_criteria` in generate()
Author: nitaytech
Pull request: no
Labels: name: bug

### System Info

```shell
- `transformers` version: 4.19.2
- Platform: Linux-5.13.0-44-generic-x86_64-with-glibc2.31
- Python version: 3.10.4
- Huggingface_hub version: 0.7.0
- PyTorch version (GPU?): 1.11.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
```


### Who can help?

@patrickvonplaten

Hi,
I don't get the logic behind `max_length` and `stopping_criteria` for `generate(self,...)` function for encoder-decoder models. 
Accordingly, when you pass `max_length` you get the deprecated warning, which is ok - however, it recommends using the `StoppingCriteriaList` object with the `MaxLengthCriteria`. 

Now the real problem happens:
The `generate()` function uses the following code:
*******************************
      # 5. Prepare `max_length` depending on other stopping criteria
      # if `max_new_tokens` is passed, but not `max_length` -> set `max_length = max_new_tokens`
      if max_length is None and max_new_tokens is not None:
          max_length = max_new_tokens + input_ids_seq_length
      elif max_length is not None and max_new_tokens is not None:
          # Both are set, this is odd, raise a warning
          warnings.warn(
              "Both `max_length` and `max_new_tokens` have been set "
              f"but they serve the same purpose. `max_length` {max_length} "
              f"will take priority over `max_new_tokens` {max_new_tokens}.",
              UserWarning,
          )
      # default to config if still None
      max_length = max_length if max_length is not None else self.config.max_length
*******************************
As you can see, `max_length ` is going to have a value no matter what (even if you pass `max_length=None` the value is set to be `self.config.max_length` which is equal to 20 for T5, and this extremely bad for users who are not aware to it... in older versions this wasn't the behavior of the `generate()` function.) 

Now, if you pass `stopping_criteria=StoppingCriteriaList(MaxLengthCriteria(100))` you get an Exception (because max_length is not None and you init the default StoppingCriteriaList with MaxLengthCriteria(max_length==self.config.max_length)).

If you pass `max_length=100` you get the warning.
If you don't pass `max_length ` you still get the warning (because `max_length = max_length if max_length is not None else self.config.max_length`)

So how exactly someone is supposed to set the max length? 
Should I change self.config.max_length? This is not a good practice...
Of course, I can pass `max_length` for `generate()`, however, the warning says not to do that...


### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

call model.generate(max_length=...) and model.generate(stopping_criteria=StoppingCriteriaList([MaxLengthCriteria=100)]),...)

### Expected behavior

```shell
change the warning,
change the logic behind default StoppingCriteriaList
change how you infer max_length
make sure users are aware of max_length=model.configs.max_length when max_length is None
```


Comments:
Hey @nitaytech 👋 The behavior surrounding `max_length` can't be changed, as it would not be backwards compatible. But we also don't like its behavior for the reasons you described, hence the plans to deprecate (and the warning).

As per [this recent discussion](https://github.com/huggingface/transformers/issues/17414#issuecomment-1148836312), we have decided to give preference to the `max_new_tokens` argument, as it is clearer for all types of models. We are working on updating warnings and documentations to make it clear it is the correct way to control the maximum length of the generated text :)

Meanwhile, can you confirm that `max_new_tokens` works properly in your case? 

@nitaytech [this comment](https://github.com/huggingface/transformers/pull/17196#issuecomment-1159147143) might also be relevant to your pain points.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

related: #18018

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

from transformers import Seq2SeqTrainingArguments

training_args = Seq2SeqTrainingArguments(
    output_dir="./kirah/fcv_s2t",  # change to a repo name of your choice
    per_device_train_batch_size=16,
    gradient_accumulation_steps=1,  # increase by 2x for every 2x decrease in batch size
    learning_rate=1e-5,
    warmup_steps=500,
    max_steps=7,
    gradient_checkpointing=True,
    fp16=False,
    evaluation_strategy="steps",
    per_device_eval_batch_size=8,
    predict_with_generate=True,
    max_new_tokens=1000,  # <======== Before max_length=1000,
    save_steps=1000,
    eval_steps=1000,
    logging_steps=25,
    report_to=["tensorboard"],
    load_best_model_at_end=True,
    metric_for_best_model="wer",
    greater_is_better=False,
    push_to_hub=True,
)
__init__() got an unexpected keyword argument 'max_new_tokens'


Hey @jhoanmartinez -- Seq2SeqTrainingArguments doesn't have `max_new_tokens` yet