Generate: use `GenerationConfig` as the basis for `.generate()` parametrization
Author: gante
Pull request: yes

# What does this PR do?

This PR introduces `generation_config` as the main controller of `.generate()` calls.

In particular:
1. It adds a `from_model_config` class method to `GenerateConfig`, to load a generation config from a (legacy) model config;
2. Adds a `generation_config` argument to `.generate()`. If it is not passed, it will be loaded from a pre-determined sequence (check for `generation_config.json` -> if it fails, load from the model config);
3. Because we always have a `generation_config` in `.generate()`, which holds all parametrization, gets rid of all local variables;
4. ⚠️ Changes the arguments to `generate()` (and corresponding docstring) so as to exclude `generate_config` parameters (i.e. they were moved to `**kwargs`). This is mostly to avoid a massive docstring and list of arguments that make `.generate()` very messy at the moment -- `GenerationConfig`'s docstring explains all the ways `.generate()` can be controlled, organized by type of manipulation, while `.generate()`'s docstring focuses on the API.

Notes: I've successfully run SLOW tests of GPT2 (which has a `generate_config.json`) and BART (which does not) against this PR.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Fully agree with @sgugger here. 

Totally ok to just link to the `GenerateConfig` doc page -> think this make the docs online also cleaner actually. 
Also I'd maybe rename `generate_config` to just `config` in generate or do you think this will cause confusion with the model's config?

Overall, this is a great improvement !

@sgugger @patrickvonplaten It is ready for review.

Major changes since the last review request:
1. `ModelClass.from_pretrained()` pre-loads a `generation_config` attribute to the model if a `generation_config.json` exists, as suggested above
2. Handle the case where the model config has nested dictionaries (e.g. a `decoder` component)
3. Keep full retrocompatibility, including ad hoc `model.config` changes before calling `GenerationMixin` functions (that's why you'll see `GenerationConfig.from_model_config` in so many places, all those functions may be called independently 😓 )
4. Add documentation and enhance examples

Also FYI, I'm off until the 8th of Dec 🌴 

Agreed with you @patrickvonplaten , that's a very good idea!

@sgugger @patrickvonplaten 

Here is a summary of the key changes since your last review:
- (thanks for the suggestion!) In `model.from_pretrained`, `model.generation_config` is set from the model config if the generation config doesn’t exist, effectively making all future generation-capable models hold a default generation config parameter. NOTE: This required minor legacy handling logic, for the case where the user makes ad hoc model config changes to control generation (which the previous solution intentionally accounted for)
- added a default `prepare_inputs_for_generation`, which raises `NotImplementedError`, and updated the new `can_generate` check accordingly. Contrarily to @patrickvonplaten's suggestion, I've kept the  `_validate_model()` check -- it returns an informative exception to the user if they try to generate with an incorrect class of a model with generation capabilities, like `AutoModel.from_pretrained(“gpt2”)`. Not using the right class was a common source of issues in the past.
- Improved the example to use named generation config files with an actual T5 example. I think two named generation configs would make the example too long 🤔 (cc @patrickvonplaten) 

I was thinking of doing the following in a follow-up PR (to avoid adding more features to this already long PR that is blocking Arthur on Whisper work):
- Add the needed modifications such that `model.save_pretrained` can push to the hub a default generation config if the file doesn’t yet exist, from the `model.generation_config` parameter (as @sgugger suggested)

@patrickvonplaten -- I'm merging to unblock @ArthurZucker's work on Whisper. 

Comments to the points above are still helpful, and I can include them in a subsequent PR! :D 

The addition of `can_generate()` is breaking in Optimum, where we use `generate()` on models which do not inherit from `PreTrainedModel`. Why isn't `can_generate()` in `GenerationMixin`? Can a model inherit from `GenerationMixin` but not use `generate()`? cc @gante 

@fxmarty `can_generate()` is called in `PreTrainedModel` at initialization time, to initialize the (new) generation config if it's a generation-compatible model. All models in `transformers` inherit `GenerationMixin`, regardless of whether they can generate, but in fact `can_generate()` is tangling the two classes at the moment, which is undesirable.

I may be able to rework this part, but I need to know -- what breaks on your end exactly?

> All models in transformers inherit GenerationMixin

Yes thanks, I forgot this part!

The PR I linked fix the issue on our end. I think what is breaking is that `generate()` is no more usable on models that are not inheriting from `PreTrainedModel` or that don't redefine `can_generate()`, because of https://github.com/huggingface/transformers/blob/8637316e5e94ba0a2493e5df7846f2f23f46eaef/src/transformers/generation/utils.py#L934

But it's a very minor issue, and the fix is easy, so it's probably not too important.

@fxmarty 👍 

In the long run, I'd like to see if it's possible to separate the two (`PreTrainedModel` and `GenerationMixin`, where a model only inherits `GenerationMixin` if it can generate). It should help libraries downstream like `optimum`!

Let me know if I can be of further assistance.