T5 Gradient Checkpointing
Author: ceshine
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Partially fixes #6564. This is inspired by @xFinal's workaround. However, instead of modifying PyTorch's implementation of gradient checkpointing, I only modify `T5Block` here and replace the None value with a dummy tensor that requires a gradient.

~Gradient checkpointing is only enabled for the encoder. Some of the outputs of the decoder don't require gradients and will cause problems. PyTorch 1.8.0 has fixed this (see note 1 below).~

Additional notes:

1. `require_grad = True` for the dummy tensor is no longer required since PyTorch 1.8.0 ([From this PR](https://github.com/pytorch/pytorch/pull/45934)).
2. None as a return value is allowed since [this PyTorch PR](https://github.com/pytorch/pytorch/pull/52422). It has not been released yet (the latest release is 1.8.1 at the time of writing). We won't even need the dummy tensor after that.
3. I tested this code locally with PyTorch 1.7.1.
4. I did not write any additional test because it seems that [test_training_gradient_checkpointing](https://github.com/huggingface/transformers/blob/81009b7a5c5cb183a9275c15bf347bdc988b02c4/tests/test_modeling_common.py#L242) in `ModelTesterMixin` already covers it.

~EDIT: I forgot this PR only covers the encoder, not the decoder. The description has been updated to reflect this fact.~
EDIT: This PR covers the decoder now.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation). (Updated docstring for T5Config.)
- [ ] Did you write any new necessary tests? (Already covered by existing tests.)


## Who can review?

@patrickvonplaten (T5)
@sgugger (Documentation)

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments: