Make gradient_checkpointing a training argument
Author: sgugger
Pull request: yes

# What does this PR do?

This PR reworks the logic behind gradient accumulation. It is currently set as a configuration argument which is annoying because:
- it's not easily discoverable
- when someone pushes a model trained with gradient checkpointing activated to the Hub, that models keeps this gradient checkpointing even if new users don't want to use it.

That's why this PR depractes the `gradient_checkpointing` argument in any config and adds:
- a method `gradient_checkpointing_enable` to `PreTrainedModel` to activate gradient checkpointing
- a training argument for the users using the `Trainer` API that will call that `gradient_checkpointing` method.

Internally, the implementation still relies on the config as it's the easiest place to set something that needs to pass several layers of a model (if we have a `BertForMaskedLM` for instance, the actual gradient checkpointing only applies to the `BertEncoder` inside the `BertModel` inside that `BertForMaskedLM`) but that argument is made private and not saved to the model Hub.

Comments:
I took the liberty to also document this feature in https://huggingface.co/transformers/performance.html and pushed it here, so if you rename the method please adjust the doc as well. Thank you!

I'm not very happy about keeping `gradient_checkpointing` in the config internally as it adds IMO significantly more complexity to what a user has to know now about model configurations. Before this PR, every configuration parameter that one sees in `configuration_utils.py` is stored when saving the configuration file. If we introduce now private configuration parameters that are not saved when the model is saved, it forces users to learn/understand a new exception and makes the code harder to understand/read.

I'm very much in favor of removing `gradient_checkpointing` from the config, but the better option IMO is not to go over the config anymore at all but to provide `_disable_gradient_checkpointing`, `_enable_gradient_checkpointing` functions to all sub-modules. It's much more work, but IMO there are also much more upsides to having this approach. 

> I'm not very happy about keeping gradient_checkpointing in the config internally as it adds IMO significantly more complexity to what a user has to know now about model configurations. Before this PR, every configuration parameter that one sees in configuration_utils.py is stored when saving the configuration file. If we introduce now private configuration parameters that are not saved when the model is saved, it forces users to learn/understand a new exception and makes the code harder to understand/read.

I am not following since this is all private. The user does not have to know anything about model configurations for this option. I'm also not sure which new exceptions you are mentioning?

> I'm very much in favor of removing `gradient_checkpointing` from the config, but the better option IMO is not to go over the config anymore at all but to provide `_disable_gradient_checkpointing`, `_enable_gradient_checkpointing` functions to all sub-modules. It's much more work, but IMO there are also much more upsides to having this approach.

Note that those submodules are often not even `PreTrainedModel`, so we will have to add those functions manually to a tons of `nn.Module`. For backward compatibility, we will also need to still have something stored in the config, since the config can't call the method `gradient_checkpointing_enable` on the model, so this effort is a bit pointless before v5 in the sense that there will be private parameters not saved anyway.

In any case, if this second approach is selected, I would still urge to merge this PR as soon as possible to avoid any merge conflict or many user diverging from the templates. We can then change the internal implementation on the models added more progressively.

I'm just a bit worried that we'll start using the "private" configuration parameters of `PreTrainedConfig` just as a way to easily pass flags to all the `nn,Modules` even though those parameters shouldn't be in the config at all. For me the configuration should really just be static configuration and not serve any other purpose than defining the model architecture. 

For a user that just looks at the configuration on the hub this PR is great, but for users that actually looks into the code, adding a `NO_SAVE_CONFIG_KEYS` option to `PreTrainedConfig` adds a new layer of complexity for the reader to understand. This could be avoided IMO. 

Think we should be able to add a single method to the `BertPreTrainedModel` like this:

```python
def _enable_gradient_checkpointing(self):
    model = self
    if hasattr(model, self.base_model_prefix):
        model = getattr(model, self.base_model_prefix)
    
    # set gradient checkpointing to True in the encoder
    model.encoder.gradient_checkpointing = True
```

=> this should work just fine no? 

Given that we will have to leave it in the config anyways until v5, I'm fine with leveraging the config I guess - I just don't think it's good practice to introduce "special" configuration parameters with `NO_SAVE_CONFIG_KEYS`

If we leave the config as is, as proposed by Patrick, should we perhaps discuss the ability for the user to choose what goes into the published model's config? We are sort of trying to do DWIM (do what I mean) and magically have the published model have all the right settings.

So adding to the model saving interface our default filters which for example will automatically disable `gradient_checkpointing` and then allowing users to override those if they need to? So we have the ease of use of having sensible defaults and then allow users to override any of the defaults? 

In the current PR the user has no control over `NO_SAVE_CONFIG_KEYS`

And we won't need to wait till v5 to do so.

@stas00 This is out of scope of this PR (which does not contain the `NO_SAVE_CONFIG_KEYS` anymore btw, to address Patrick's comments), so maybe the discussion should be moved elsewhere? 

I was just following up to Patrick's comment. I have no problem with not discussing it here.