Correct AutoConfig call docstrings

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

The `AutoConfig` class has no method named `from_json_file`, so the [examples in the documentation](https://huggingface.co/transformers/model_doc/auto.html#transformers.TFAutoModelForSequenceClassification.from_pretrained) are incorrect. Most likely the intention is to call `from_pretrained`.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


checkpoint breaks with deepspeed

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.3.3
- Platform: linux
- Python version: 3.7
- PyTorch version (GPU?): 1.8
- Tensorflow version (GPU?): -
- Using GPU in script?: -
- Using distributed or parallel set-up in script?: -

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

Model hub:

- for issues with a model report at https://discuss.huggingface.co/ and tag the model's creator.

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->
deepspeed: @stas00

## Information
Dear @stas00
Having your permission, I opened up this bug, you are really my only hope with this issue and I truly appreciate your help. Thank you very much.

I am using mt5 model, I modified it with adding adapters layers. The problem arises when:
* loading checkpoints from the model trained with deepspeed

The tasks I am working on is:
* paraphrase detection using paws-x dataset on mt5 model

## To reproduce

Steps to reproduce the behavior:

```
git clone git@github.com:dorost1234/codes.git
conda create --name deepspeed python=3.7
conda activate deepspeed
conda install pytorch torchvision torchaudio cudatoolkit=11.1 -c pytorch -c conda-forge
python setup.py develop
pip install deepspeed
```

running the codes:
```
deepspeed run_seq2seq.py  configs/test.json
```

I save a checkpoint every 10 steps, the output would look like the below:

```
After first checkpoint I kill the codes, here is the output:
onfiguration saved in outputs/checkpoint-10/config.json
Model weights saved in outputs/checkpoint-10/pytorch_model.bin
[2021-03-20 15:18:45,897] [INFO] [logging.py:60:log_dist] [Rank 0] Saving model checkpoint: outputs/checkpoint-10/global_step10/mp_rank_00_model_states.pt
[2021-03-20 15:18:51,783] [INFO] [engine.py:1680:_save_zero_checkpoint] zero checkpoint saved outputs/checkpoint-10/global_step10/zero_pp_rank_0_mp_rank_00optim_states.pt
Configuration saved in outputs/config.json
Model weights saved in outputs/pytorch_model.bin
```

Then, I contunue training with running the command again:
```
deepspeed run_seq2seq.py  configs/test.json
```

once loading the checkpoint, it cannot load it with deepspeed:
```
successfully loaded 1 ZeRO state_dicts for rank 0
Traceback (most recent call last):
  File "run_seq2seq.py", line 512, in <module>
    main()
  File "run_seq2seq.py", line 476, in main
    train_result = trainer.train(resume_from_checkpoint=checkpoint)
  File "/users/dara/dev/debug_codes/seq2seq/third_party/trainers/trainer.py", line 780, in train
    self._load_optimizer_and_scheduler(resume_from_checkpoint)
  File "/users/dara/dev/debug_codes/seq2seq/third_party/trainers/trainer.py", line 1169, in _load_optimizer_and_scheduler
    self.deepspeed.load_checkpoint(checkpoint, load_optimizer_states=True, load_lr_scheduler_states=True)
  File "/users/dara/libs/anaconda3/envs/deepspeed/lib/python3.7/site-packages/deepspeed/runtime/engine.py", line 1416, in load_checkpoint
    load_optimizer_states=load_optimizer_states)
  File "/users/dara/libs/anaconda3/envs/deepspeed/lib/python3.7/site-packages/deepspeed/runtime/engine.py", line 1488, in _load_zero_checkpoint
    load_from_fp32_weights=self.zero_load_from_fp32_weights())
  File "/users/dara/libs/anaconda3/envs/deepspeed/lib/python3.7/site-packages/deepspeed/runtime/zero/stage2.py", line 1844, in load_state_dict
    self._restore_base_optimizer_state(state_dict_list)
  File "/users/dara/libs/anaconda3/envs/deepspeed/lib/python3.7/site-packages/deepspeed/runtime/zero/stage2.py", line 1805, in _restore_base_optimizer_state
    self.optimizer.state[p][key].data.copy_(saved.data)
RuntimeError: The size of tensor a (302612288) must match the size of tensor b (129296512) at non-singleton dimension 0
Killing subprocess 23829
Traceback (most recent call last):
  File "/users/dara/libs/anaconda3/envs/deepspeed/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/users/dara/libs/anaconda3/envs/deepspeed/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/users/dara/libs/anaconda3/envs/deepspeed/lib/python3.7/site-packages/deepspeed/launcher/launch.py", line 171, in <module>
    main()
  File "/users/dara/libs/anaconda3/envs/deepspeed/lib/python3.7/site-packages/deepspeed/launcher/launch.py", line 161, in main
    sigkill_handler(signal.SIGTERM, None)  # not coming back
  File "/users/dara/libs/anaconda3/envs/deepspeed/lib/python3.7/site-packages/deepspeed/launcher/launch.py", line 139, in sigkill_handler
    raise subprocess.CalledProcessError(returncode=last_return_code, cmd=cmd)
subprocess.CalledProcessError: Command '['/users/dara/anaconda3/envs/deepspeed/bin/python', '-u', 'run_seq2seq.py', '--local_rank=0', 'configs/test.json']' returned non-zero exit status 1.
```

## Expected behavior

being able to continue training from the saved checkpoints


JSONLINES support on examples/seq2seq/run_translation.py

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- Platform: windows 10
- Python version: 3.8
- PyTorch version (GPU?): 1.8.0
- Using GPU in script?: yes, tesla v100
- Using distributed or parallel set-up in script?: no

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

Model hub:

- for issues with a model report at https://discuss.huggingface.co/ and tag the model's creator.

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

@sgugger @stas00 @LysandreJik
## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

It is said in the seq2seq READ.me that
> The task of translation supports only custom JSONLINES files

However, in the line 202, the extension of file should be `.json`
```py
if self.train_file is not None:
    extension = self.train_file.split(".")[-1]
    assert extension == "json", "`train_file` should be a json file."
```

Even if I changed it to
```py
assert extension in ("json", "jsonl")
```

it throws another error, which says that there's no jsonl process file in library `datasets`

```py
Traceback (most recent call last):
  File "/home/zchelllo/anaconda3/envs/ex/lib/python3.8/site-packages/datasets/load.py", line 323, in prepare_module
    local_path = cached_path(file_path, download_config=download_config)
  File "/home/zchelllo/anaconda3/envs/ex/lib/python3.8/site-packages/datasets/utils/file_utils.py", line 274, in cached_path
    output_path = get_from_cache(
  File "/home/zchelllo/anaconda3/envs/ex/lib/python3.8/site-packages/datasets/utils/file_utils.py", line 614, in get_from_cache
    raise FileNotFoundError("Couldn't find file at {}".format(url))
FileNotFoundError: Couldn't find file at https://raw.githubusercontent.com/huggingface/datasets/master/datasets/jsonl/jsonl.py

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/zchelllo/anaconda3/envs/ex/lib/python3.8/site-packages/datasets/load.py", line 335, in prepare_module
    local_path = cached_path(file_path, download_config=download_config)
  File "/home/zchelllo/anaconda3/envs/ex/lib/python3.8/site-packages/datasets/utils/file_utils.py", line 274, in cached_path
    output_path = get_from_cache(
  File "/home/zchelllo/anaconda3/envs/ex/lib/python3.8/site-packages/datasets/utils/file_utils.py", line 614, in get_from_cache
    raise FileNotFoundError("Couldn't find file at {}".format(url))
FileNotFoundError: Couldn't find file at https://raw.githubusercontent.com/huggingface/datasets/master/datasets/jsonl/jsonl.py

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "examples/seq2seq/run_translation.py", line 562, in <module>
    main()
  File "examples/seq2seq/run_translation.py", line 295, in main
    datasets = load_dataset(extension, data_files=data_files)
  File "/home/zchelllo/anaconda3/envs/ex/lib/python3.8/site-packages/datasets/load.py", line 707, in load_dataset
    module_path, hash, resolved_file_path = prepare_module(
  File "/home/zchelllo/anaconda3/envs/ex/lib/python3.8/site-packages/datasets/load.py", line 343, in prepare_module
    raise FileNotFoundError(
FileNotFoundError: Couldn't find file locally at jsonl/jsonl.py, or remotely at https://raw.githubusercontent.com/huggingface/datasets/master/datasets/jsonl/jsonl.py.
The file is also not present on the master branch on github.
```

Is this part in the stage of dev now? I think it is related to [#1943](https://github.com/huggingface/datasets/pull/1943)
Could you add the original csv support before this implementation is over?
Thanks in advance!


mt5 getting nans with fp16

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.4.2
- Platform: linux
- Python version: 3.7
- PyTorch version (GPU?): 1.8
- Tensorflow version (GPU?): -
- Using GPU in script?: -
- Using distributed or parallel set-up in script?: -

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

Model hub:

- for issues with a model report at https://discuss.huggingface.co/ and tag the model's creator.

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

 t5: @patrickvonplaten, @patil-suraj

## Information

I am using mt5-small model:
* the problem arises when using fp16 with mt5

The tasks I am working on is:
* translation

## To reproduce

Steps to reproduce the behavior:

`python run_translation.py     --model_name_or_path google/mt5-small     --do_train     --do_eval     --source_lang en     --target_lang ro     --dataset_name wmt16     --dataset_config_name ro-en     --output_dir  test/tst-translation     --per_device_train_batch_size=4     --per_device_eval_batch_size=4     --overwrite_output_dir     --predict_with_generate --max_train_samples 100 --fp16`

outputs:
```
***** eval metrics *****
  epoch                     =     3.0
  eval_bleu                 =  0.0039
  eval_gen_len              =    2.95
  eval_loss                 =     nan
  eval_mem_cpu_alloc_delta  =     4MB
  eval_mem_cpu_peaked_delta =     5MB
  eval_mem_gpu_alloc_delta  =     0MB
  eval_mem_gpu_peaked_delta =  1080MB
  eval_runtime              = 72.1865
  eval_samples              =    1999
  eval_samples_per_second   =  27.692
```

## Expected behavior

being able to use fp16 with mt5 models. Thank you very much for your help, this is really crucial for me to be able to run these models with fp16 to be able to fit more data into old GPUs I have access to and I appreciate a lot your help.


Bump jinja2 from 2.11.2 to 2.11.3 in /examples/research_projects/lxmert

dependencies

Pull requests that update a dependency file

Bumps [jinja2](https://github.com/pallets/jinja) from 2.11.2 to 2.11.3.

Release notes

Sourced from jinja2's releases.

2.11.3

This contains a fix for a speed issue with the urlize filter. urlize is likely to be called on untrusted user input. For certain inputs some of the regular expressions used to parse the text could take a very long time due to backtracking. As part of the fix, the email matching became slightly stricter. The various speedups apply to urlize in general, not just the specific input cases.

PyPI: https://pypi.org/project/Jinja2/2.11.3/

Changes: https://jinja.palletsprojects.com/en/2.11.x/changelog/#version-2-11-3

Changelog

Sourced from jinja2's changelog.

Version 2.11.3

Released 2021-01-31

Improve the speed of the urlize filter by reducing regex
backtracking. Email matching requires a word character at the start
of the domain part, and only word characters in the TLD. :pr:1343

Commits

cf21539 release version 2.11.3

15ef8f0 Merge pull request #1343 from pallets/urlize-speedup

ef658dc speed up urlize matching

eeca0fe Merge pull request #1207 from mhansen/patch-1

2dd7691 Merge pull request #1209 from mhansen/patch-3

4892940 do_dictsort: update example ready to copy/paste

7db7d33 api.rst: bugfix in docs, import PackageLoader

9ec465b fix changelog header

See full diff in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=jinja2&package-manager=pip&previous-version=2.11.2&new-version=2.11.3)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/huggingface/transformers/network/alerts).


[vulnerability] in example deps fix

Takes care of:
https://github.com/huggingface/transformers/security/dependabot/examples/research_projects/lxmert/requirements.txt/jinja2/open

@LysandreJik


[trainer] figuring out why eval with `--fp16_full_eval` is 25% slower

Good Second Issue

Issues that are more difficult to do than "Good First" issues - give it a try if you want!

Recently HF trainer was extended to support full fp16 eval via `--fp16_full_eval`. I'd have expected it to be either equal or faster than eval with fp32 model, but surprisingly I have noticed a 25% slowdown when using it.

This may or may not impact deepspeed as well, which also runs eval in fp16, but we can't compare it to a baseline, since it only runs fp16.

I wonder if someone would like to research where the slowdown comes from.

I'd probably isolate the `model.half()` call which should be a constant and focus on the rest of the eval. I'm thinking that some component doesn't take well to fp16 variables. e.g.  label smoothing was problematic and now should be fixed in https://github.com/huggingface/transformers/pull/10815, but I tested w/ and w/o label smoothing and it's not adding to the slowdown.

Here are the script and the corresponding metrics.

First w/o  `--fp16_full_eval`,

```
export BS=16; rm -r output_dir; PYTHONPATH=src USE_TF=0 CUDA_VISIBLE_DEVICES=0 \
./examples/seq2seq/run_translation.py --model_name_or_path t5-small --output_dir /tmp/zero3 \
--overwrite_output_dir --max_train_samples 10 --max_val_samples 100 --max_source_length 12 \
--max_target_length 128 --val_max_target_length 128 --do_train --num_train_epochs 1 \
--per_device_train_batch_size 2 --learning_rate 3e-3 --warmup_steps 8 --predict_with_generate \
--logging_steps 0 --save_steps 2 --eval_steps 1 --group_by_length --adafactor --dataset_name wmt16 \
--dataset_config ro-en --source_lang en --target_lang ro \
--source_prefix "translate English to Romanian: " --do_eval

***** train metrics *****
  epoch                      =    1.0
  init_mem_cpu_alloc_delta   =    2MB
  init_mem_cpu_peaked_delta  =    0MB
  init_mem_gpu_alloc_delta   =  230MB
  init_mem_gpu_peaked_delta  =    0MB
  train_mem_cpu_alloc_delta  =   60MB
  train_mem_cpu_peaked_delta =   63MB
  train_mem_gpu_alloc_delta  =  231MB
  train_mem_gpu_peaked_delta =  194MB
  train_runtime              = 7.7162
  train_samples              =     10
  train_samples_per_second   =  0.648

***** eval metrics *****
  epoch                     =    1.0
  eval_bleu                 = 2.4612
  eval_gen_len              =  18.53
  eval_loss                 =  5.017
  eval_mem_cpu_alloc_delta  =    0MB
  eval_mem_cpu_peaked_delta =    0MB
  eval_mem_gpu_alloc_delta  =    0MB
  eval_mem_gpu_peaked_delta =  244MB
  eval_runtime              = 4.6481
  eval_samples              =    100
  eval_samples_per_second   = 21.514
```

now let's add `--fp16_full_eval`:

```
export BS=16; rm -r output_dir; PYTHONPATH=src USE_TF=0 CUDA_VISIBLE_DEVICES=0 \
./examples/seq2seq/run_translation.py --model_name_or_path t5-small --output_dir /tmp/zero3 \
--overwrite_output_dir --max_train_samples 10 --max_val_samples 100 --max_source_length 12 \
--max_target_length 128 --val_max_target_length 128 --do_train --num_train_epochs 1 \
--per_device_train_batch_size 2 --learning_rate 3e-3 --warmup_steps 8 --predict_with_generate \
--logging_steps 0 --save_steps 2 --eval_steps 1 --group_by_length --adafactor --dataset_name wmt16 \
--dataset_config ro-en --source_lang en --target_lang ro \
--source_prefix "translate English to Romanian: " --do_eval  \
--fp16_full_eval

***** train metrics *****
  epoch                      =    1.0
  init_mem_cpu_alloc_delta   =    2MB
  init_mem_cpu_peaked_delta  =    0MB
  init_mem_gpu_alloc_delta   =  230MB
  init_mem_gpu_peaked_delta  =    0MB
  train_mem_cpu_alloc_delta  =   60MB
  train_mem_cpu_peaked_delta =   63MB
  train_mem_gpu_alloc_delta  =  231MB
  train_mem_gpu_peaked_delta =  194MB
  train_runtime              = 7.1477
  train_samples              =     10
  train_samples_per_second   =    0.7

***** eval metrics *****
  epoch                     =    1.0
  eval_bleu                 = 2.4612
  eval_gen_len              =  18.53
  eval_loss                 = 5.0168
  eval_mem_cpu_alloc_delta  =    0MB
  eval_mem_cpu_peaked_delta =    0MB
  eval_mem_gpu_alloc_delta  = -231MB
  eval_mem_gpu_peaked_delta =  262MB
  eval_runtime              = 6.0125
  eval_samples              =    100
  eval_samples_per_second   = 16.632
```

As you can see w/o `--fp16_full_eval`: we get ~22 samples per sec and w/ it only ~17/ - that's a huge difference.

I also tested with a larger sample and the gap remains constant.

The halving happens here:

https://github.com/huggingface/transformers/blob/21e86f99e6b91af2e4df3790ba6c781e85fa0eb5/src/transformers/trainer.py#L1800

Thank you!


[trainer] fix nan in full-fp16 label_smoothing eval

This PR fixes the issue of getting NaN eval loss with any inference that uses full fp16 model - which is the case with deepspeed or when `--fp16_full_eval` is passed.

The problem is that `log_probs.sum` runs over 30-50K of numbers overflows easily in fp16, so this PR switches it to fp32 internally. Which surprisingly requires almost no extra memory. As the conversion happens on the hardware level and we only need an extra `2 bytes * batch_size` of additional memory.

Here is some data showing the the metrics remain the same after this fix:

```
export BS=16; rm -r output_dir; PYTHONPATH=src USE_TF=0 CUDA_VISIBLE_DEVICES=0 \
./examples/seq2seq/run_translation.py --model_name_or_path t5-small --output_dir /tmp/zero3 \
--overwrite_output_dir --max_train_samples 10 --max_val_samples 100 --max_source_length 12 \
--max_target_length 128 --val_max_target_length 128 --do_train --num_train_epochs 1 \
--per_device_train_batch_size 2 --learning_rate 3e-3 --warmup_steps 8 --predict_with_generate \
--logging_steps 0 --save_steps 2 --eval_steps 1 --group_by_length --adafactor --dataset_name wmt16 \
--dataset_config ro-en --source_lang en --target_lang ro \
--source_prefix "translate English to Romanian: " --do_eval --label_smoothing 0.1

***** train metrics *****
  epoch                      =    1.0
  init_mem_cpu_alloc_delta   =    2MB
  init_mem_cpu_peaked_delta  =    0MB
  init_mem_gpu_alloc_delta   =  230MB
  init_mem_gpu_peaked_delta  =    0MB
  train_mem_cpu_alloc_delta  =   60MB
  train_mem_cpu_peaked_delta =   63MB
  train_mem_gpu_alloc_delta  =  231MB
  train_mem_gpu_peaked_delta =  194MB
  train_runtime              = 7.7162
  train_samples              =     10
  train_samples_per_second   =  0.648

***** eval metrics *****
  epoch                     =    1.0
  eval_bleu                 = 2.4612
  eval_gen_len              =  18.53
  eval_loss                 =  5.017
  eval_mem_cpu_alloc_delta  =    0MB
  eval_mem_cpu_peaked_delta =    0MB
  eval_mem_gpu_alloc_delta  =    0MB
  eval_mem_gpu_peaked_delta =  244MB
  eval_runtime              = 4.6481
  eval_samples              =    100
  eval_samples_per_second   = 21.514
```

now let's add `--fp16_full_eval`, which before this PR leads to ` eval_loss = nan`
```
export BS=16; rm -r output_dir; PYTHONPATH=src USE_TF=0 CUDA_VISIBLE_DEVICES=0 \
./examples/seq2seq/run_translation.py --model_name_or_path t5-small --output_dir /tmp/zero3 \
--overwrite_output_dir --max_train_samples 10 --max_val_samples 100 --max_source_length 12 \
--max_target_length 128 --val_max_target_length 128 --do_train --num_train_epochs 1 \
--per_device_train_batch_size 2 --learning_rate 3e-3 --warmup_steps 8 --predict_with_generate \
--logging_steps 0 --save_steps 2 --eval_steps 1 --group_by_length --adafactor --dataset_name wmt16 \
--dataset_config ro-en --source_lang en --target_lang ro \
--source_prefix "translate English to Romanian: " --do_eval --label_smoothing 0.1 \
--fp16_full_eval

***** train metrics *****
  epoch                      =    1.0
  init_mem_cpu_alloc_delta   =    2MB
  init_mem_cpu_peaked_delta  =    0MB
  init_mem_gpu_alloc_delta   =  230MB
  init_mem_gpu_peaked_delta  =    0MB
  train_mem_cpu_alloc_delta  =   60MB
  train_mem_cpu_peaked_delta =   63MB
  train_mem_gpu_alloc_delta  =  231MB
  train_mem_gpu_peaked_delta =  194MB
  train_runtime              = 7.1477
  train_samples              =     10
  train_samples_per_second   =    0.7

***** eval metrics *****
  epoch                     =    1.0
  eval_bleu                 = 2.4612
  eval_gen_len              =  18.53
  eval_loss                 = 5.0168
  eval_mem_cpu_alloc_delta  =    0MB
  eval_mem_cpu_peaked_delta =    0MB
  eval_mem_gpu_alloc_delta  = -231MB
  eval_mem_gpu_peaked_delta =  262MB
  eval_runtime              = 6.0125
  eval_samples              =    100
  eval_samples_per_second   = 16.632
```
`eval_loss` is off by 0.0002.

I spent quite some time trying to find where to add a test, but it's a tricky situation where the input has to be pretty huge. I remember seeing it in some deepspeed tests, but I can't find at the moment, currently all tests return a normal number.

One interesting things I noticed is that `--fp16_full_eval` makes eval slower by 20-25% which is strange, but I tested this PR has no impact on the speed. I file a separate issue about it https://github.com/huggingface/transformers/issues/10816

Fixes: https://github.com/huggingface/transformers/issues/10674

@sgugger, @LysandreJik


[makefile] autogenerate target

As a follow up to https://github.com/huggingface/transformers/pull/10801 this PR proposes to group autogeneration code in a separate target. I think as the number of little things the makefile does this helps with clarity.

There is no functional change.

@sgugger, @LysandreJik


Example code for ReformerForMaskedLM

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.4.2
- Platform: Linux
- Python version: 3.8
- PyTorch version (GPU?): 1.7.1[11.0]
- Tensorflow version (GPU?):
- Using GPU in script?:yes
- Using distributed or parallel set-up in script?:no

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.
@patrickvonplaten
Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

Model hub:

- for issues with a model report at https://discuss.huggingface.co/ and tag the model's creator.

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

## Information

Model I am using (Bert, XLNet ...): ReformerForMaskedLM

The problem arises when using:
* [x ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ x] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:
Running the example code for ReformerForMaskedLM:
```
from transformers import ReformerTokenizer, ReformerForMaskedLM
import torch

tokenizer = ReformerTokenizer.from_pretrained('google/reformer-crime-and-punishment')
model = ReformerForMaskedLM.from_pretrained('google/reformer-crime-and-punishment')

inputs = tokenizer("The capital of France is [MASK].", return_tensors="pt")
labels = tokenizer("The capital of France is Paris.", return_tensors="pt")["input_ids"]

outputs = model(**inputs, labels=labels)
loss = outputs.loss
logits = outputs.logits
```
causes:
```AssertionError: If you want to use `ReformerForMaskedLM` make sure `config.is_decoder=False` for bi-directional self-attention.```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->


Domain adaptation

Hi all,

I'm just wondering how to do model adaptation of pre-trained Camembert model on my custom dataset ?

I haven't found any information in Transformers documentation.

Best regards,

Lematmat
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:4.4.0
- Platform:Jupyter Notebook
- Python version:3.7
- PyTorch version (GPU?):1.8, no GPU
- Tensorflow version (GPU?):
- Using GPU in script?:no
- Using distributed or parallel set-up in script?:no

 -->

## Information

Model I am using (Bert, XLNet ...): Camembert


Add transformers id to hub requests

# What does this PR do?

This PR adds a `TRANSFORMERS_ID` const, which helps us to group the several files request against the hub.


handle_impossible_answer not working in the question answering pipeline for ROBERTa model

### Environment info
- Platform: Linux 20.04
- Python version 3.8.5
- `transformers` version `3.5.0` and `4.3.2`

### The issue
I'm using the `pipeline("question-answering")` with QA Models downloaded [from community](https://huggingface.co/models?pipeline_tag=question-answering). I'm evaluating models on the SQUAD 2.0 dataset which doesn't always have an answer to the given question - that's what the `handle_impossible_answer` flag in the pipeline is for.

I noticed that ROBERTa model (any ROBERTa, not just a specific model) in version 4 of `transformers` always produces an answer despite the `handle_impossible_answer` flag - even if the same model for the same example didn't produce an answer (returned "" as an answer) while using version 3 of the library.

```python
bert_model_name = 'deepset/bert-base-cased-squad2'
roberta_model_name = 'deepset/roberta-base-squad2'

bert_tokenizer = AutoTokenizer.from_pretrained(bert_model_name)
bert_model = AutoModelForQuestionAnswering.from_pretrained(bert_model_name, return_dict=True)
roberta_tokenizer = AutoTokenizer.from_pretrained(roberta_model_name)
roberta_model = AutoModelForQuestionAnswering.from_pretrained(roberta_model_name, return_dict=True)

bert_qa = pipeline('question-answering', tokenizer=bert_tokenizer, model=bert_model)
roberta_qa = pipeline('question-answering', tokenizer=roberta_tokenizer, model=roberta_model)

# Random SQUAD 2.0 example which doesn't have an answer to the question
question = 'What was the name of the only ship operating in the Indian Ocean?'
context = 'In September 1695, Captain Henry Every, an English pirate on board the Fancy, reached the Straits of Bab-el-Mandeb, where he teamed up with five other pirate captains to make an attack on the Indian fleet making the annual voyage to Mocha. The Mughal convoy included the treasure-laden Ganj-i-Sawai, reported to be the greatest in the Mughal fleet and the largest ship operational in the Indian Ocean, and its escort, the Fateh Muhammed. They were spotted passing the straits en route to Surat. The pirates gave chase and caught up with Fateh Muhammed some days later, and meeting little resistance, took some £50,000 to £60,000 worth of treasure.'

print(bert_qa(question=question, context=context, handle_impossible_answer=True))
# transformers 3.5.0: {'score': 0.999398410320282, 'start': 0, 'end': 0, 'answer': ''}
# transformers 4.3.2: {'score': 0.999398410320282, 'start': 0, 'end': 0, 'answer': ''}

print(roberta_qa(question=question, context=context, handle_impossible_answer=True))
# transformers 3.5.0: {'score': 0.979897797107697, 'start': 0, 'end': 0, 'answer': ''}
# transformers 4.3.2: {'score': 0.222181886434555, 'start': 422, 'end': 436, 'answer': 'Fateh Muhammed'}
```

### Probable issue reason
I've found out that in the `question_answering.py` file in the `pipeline` directory in version 4 of `transformers` there is a condition that provides ROBERTa models from adjusting the `p_mask` for this task. It looks simply like this: `if self.tokenizer.cls_token_id`. And since ROBERTa's `cls_token_id = 0` the condition isn't met and the `p_mask` isn't changed for the `cls_token`. This results in omitting the token while answering the question (it behaves like e.g the token was a part of a question). For example BERT's `cls_token_id = 101` so the condition is met.

### Plausible solution
Possibly the easy solution is to expand the condition to `if self.tokenizer.cls_token_id is not None`. However, there wasn't such a condition in version 3 at all so maybe it performs some crucial function in its current form that I'm not aware of...

```python
# originally the condition here was more general and looked like this
# if self.tokenizer.cls_token_id:

if self.tokenizer.cls_token_id is not None:
    cls_index = np.nonzero(encoded_inputs["input_ids"] == self.tokenizer.cls_token_id)
    p_mask[cls_index] = 0
```


[Flax] Add general conversion script

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

This PR changes the weight architecture of `FlaxBertModel` so that it corresponds 1-to-1 to PyTorch's version of `BertModel`. This means that some weights had to be renamed (*e.g.* "layer_norm" -> "LayerNorm" since PyTorch uses "LayerNorm") and also some new `flax.linen.Modules`, such as `FlaxBertSelfOutput` had to be created.

As can be seen, the PT=>Flax conversion function is now kept very general and can be applied to all models so that we can fully delete any model-specific conversion logic.

The PR has one drawback however:

Flax official [SelfAttention Module](https://flax.readthedocs.io/en/latest/_autosummary/flax.linen.SelfAttention.html#flax-linen-selfattention) cannot be used anymore since it doesn't give us enough flexibility to convert PyTorch weights to flax weights without having a model-specific conversion function. FlaxBERT's new attention modules fully correspond to PyTorchBERT's attention modules and are IMO still kept quite short by relying on Flax's [`dot_product_attention` function](https://flax.readthedocs.io/en/latest/_autosummary/flax.linen.dot_product_attention.html). Another drawback is that for auto-regressive Transformers models we will have to manually add all the code corresponding to cached / auto-regressive attention to the attention module (which we do for PyTorch anyways) instead of being able to use already existing code of `nn.linen.SelfAttention` -> see here: https://github.com/google/flax/blob/e31063da71bd7a4df137b000df6a48b0cea35a2b/flax/linen/attention.py#L202.

All in all, rewriting parts of `flax.linen.SelfAttention` is the right choice here though because it allows us to have a much cleaner conversion function with very little downside IMO (slightly higher maintenance because we need to copy-paste a bit more code).

@LysandreJik @sgugger - could you check if you agree more or less with my solution here (below I left some comments to showcase the trade-offs a bit better). I'll clean the code & upload the new weight structure then :-)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


wav2vec doc tweaks

tiny tweaks


I am finetuning mBART for summarization using finetune_trainer.py on custom dataset, but I keep getting this error.

This is the traceback:

`thread '<unnamed>' panicked at 'index out of bounds: the len is 453 but the index is 453', /__w/tokenizers/tokenizers/tokenizers/src/tokenizer/normalizer.rs:382:21
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread '<unnamed>' panicked at 'range end index 140732665363856 out of range for slice of length 0', /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/alloc/src/vec.rs:1317:42
stack backtrace:
   0:     0x7f7340048b40 - std::backtrace_rs::backtrace::libunwind::trace::h04d12fdcddff82aa
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/../../backtrace/src/backtrace/libunwind.rs:100:5
   1:     0x7f7340048b40 - std::backtrace_rs::backtrace::trace_unsynchronized::h1459b974b6fbe5e1
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:     0x7f7340048b40 - std::sys_common::backtrace::_print_fmt::h9b8396a669123d95
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/sys_common/backtrace.rs:67:5
   3:     0x7f7340048b40 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::he009dcaaa75eed60
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/sys_common/backtrace.rs:46:22
   4:     0x7f734006806c - core::fmt::write::h77b4746b0dea1dd3
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/core/src/fmt/mod.rs:1078:17
   5:     0x7f7340046362 - std::io::Write::write_fmt::heb7e50902e98831c
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/io/mod.rs:1518:15
   6:     0x7f734004afb5 - std::sys_common::backtrace::_print::h2d880c9e69a21be9
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/sys_common/backtrace.rs:49:5
   7:     0x7f734004afb5 - std::sys_common::backtrace::print::h5f02b1bb49f36879
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/sys_common/backtrace.rs:36:9
   8:     0x7f734004afb5 - std::panicking::default_hook::{{closure}}::h658e288a7a809b29
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/panicking.rs:208:50
   9:     0x7f734004ac58 - std::panicking::default_hook::hb52d73f0da9a4bb8
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/panicking.rs:227:9
  10:     0x7f734004b751 - std::panicking::rust_panic_with_hook::hfe7e1c684e3e6462
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/panicking.rs:593:17
  11:     0x7f734004b297 - std::panicking::begin_panic_handler::{{closure}}::h42939e004b32765c
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/panicking.rs:499:13
  12:     0x7f7340048ffc - std::sys_common::backtrace::__rust_end_short_backtrace::h9d2070f7bf9fd56c
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/sys_common/backtrace.rs:141:18
  13:     0x7f734004b1f9 - rust_begin_unwind
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/panicking.rs:495:5
  14:     0x7f7340065fd1 - core::panicking::panic_fmt::ha0bb065d9a260792
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/core/src/panicking.rs:92:14
  15:     0x7f7340069d32 - core::slice::index::slice_end_index_len_fail::hcd7c711938bf4c03
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/core/src/slice/index.rs:41:5
  16:     0x7f733fd95d63 - core::ptr::drop_in_place::h2923a820a2e4a8d4
  17:     0x7f733fd9b01c - <rayon::vec::IntoIter<T> as rayon::iter::IndexedParallelIterator>::with_producer::hd6f8d390195a749b
  18:     0x7ffee086ff90 - <unknown>
thread panicked while panicking. aborting.`

I am using Colab for finetuning mBART. Any help will be appreciated. Thank you:)


[XLSR-Wav2Vec2 Info doc] Add a couple of lines

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


ONNX export outputs many warnings

I was testing ONNX export via your **04-onnx-export.ipynb** notebook and when calling `!python -m transformers.convert_graph_to_onnx --framework pt --model bert-base-cased --opset 11 --quantize onnx/bert-base-cased2.onnx` I get many Warnings like:
```
Warning: Unsupported operator LayerNormalization. No schema registered for this operator.
Warning: Unsupported operator Attention. No schema registered for this operator.
Warning: Unsupported operator LayerNormalization. No schema registered for this operator.
Warning: Unsupported operator Gelu. No schema registered for this operator.
...
```

They only appear when using `--quantize` flag.

I know it is just a warning ... but still ... does it affect the exporting process in any way?


Initializing ddp is extremely slow when finetuning RAG

Hi, when I am finetuning the RAG model, it seems that the DDP process is extremely slow. I waited 1 day but still did not see the training process.

loading file None
loading file None
loading file /nfs/users/s_xiangru/model/rag/rag/rag-sequence-nq/generator_tokenizer/special_tokens_map.json
loading file /nfs/users/s_xiangru/model/rag/rag/rag-sequence-nq/generator_tokenizer/tokenizer_config.json
Global seed set to 42
Global seed set to 42
LOCAL_RANK: 2 - CUDA_VISIBLE_DEVICES: [0,1,2,3]
Using native 16bit precision.
Global seed set to 42
LOCAL_RANK: 1 - CUDA_VISIBLE_DEVICES: [0,1,2,3]
Using native 16bit precision.
Global seed set to 42
INFO:__main__:Custom init_ddp_connection.
initializing ddp: GLOBAL_RANK: 2, MEMBER: 3/4
INFO:__main__:Custom init_ddp_connection.
initializing ddp: GLOBAL_RANK: 1, MEMBER: 2/4


How much vRAM should I have for fine tuning DeBERTa v2 xxlarge?

I'm fine tuning DeBERTa v2 xxlarge with 1.5B parameters on Nvidia Tesla T4 (16GB vRAM) and it returns "CUDA out of memory".
How much vRAM is enough?
@LysandreJik


addressing vulnerability report in research project deps

This PR addresses this security alert:
https://github.com/huggingface/transformers/security/dependabot/examples/research_projects/lxmert/requirements.txt/Pillow/open

@LysandreJik


Sort init import

# What does this PR do?

Not a high-priority item but I get bored at nights and I like writing those kinds of scripts 😅

So this PR adds a script to properly sort the import inside `_import_structure` because people have been absolutely ruthless and putting their objects in any kind of random order. That's not very feng-shui so I'm bringing back harmony by having the same sort as isort applied to all `__init__` that contain an `_import_structure`.


How to get a probability for the result of t5_tokenizer.decode(output,...)?

Hello,

I am using `t5-base` to map phrases into categories, for example: "I want to eat" -> "hunger".
Is there any way to get the probability for `result` values?

For example, if the input is "He is hungry", the model returns 5 labels. These results seem to be ordered by some relevance rank, so that the most relevant label is always first in `outputs`. So, my question is how can I retrieve these probabilities?

My final goal is to set a threshold on the probability, so that `outputs` would only include results that pass this threshold, or it can be empty if nothing relevant found.

```
t5_tokenizer = T5Tokenizer.from_pretrained('t5-base')
t5_model = T5ForConditionalGeneration.from_pretrained('t5-base')
...
model.model.eval()
outputs = model.model.generate(
    input_ids=test_input_ids,attention_mask=test_attention_mask,
    max_length=64,
    early_stopping=True,
    num_beams=10,
    num_return_sequences=5,
    no_repeat_ngram_size=2
)

for output in outputs:
    result = t5_tokenizer.decode(output, skip_special_tokens=True, clean_up_tokenization_spaces=True)
    print(result)
```

Thanks.


Expand a bit the presentation of examples

# What does this PR do?

This PR adds a bit more information to the examples README (main and specific per example), copying some information from the main philosophy and expanding a bit, to make sure all users know what we want for the examples.


Truncated words on GPT-2 output

Hi! I use the GPT-2 model for the seq2seq task, but unfortunately, at the output of the model, words are cut off and sentences are not added, how can I make the model end sentences at the output and not cut off words? (increasing the maximum length does not correct the situation)
P.S.
I'm sorry, this question is probably very stupid, but I just can't figure it out.


Pretrained XLNetTokenizer not returning tokenizer

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.4.1
- Platform: Linux-4.19.112+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.10
- PyTorch version (GPU?): 1.8.0+cu101 (False)
- Tensorflow version (GPU?): 2.4.1 (False)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.-->

@patrickvonplaten
@LysandreJik

## Information

I am using XLNet Tokenizer. When trying to use `XLNetTokenizer.from_pretrained()`, `None` object is returned.
I last worked with it in december and it was working fine till then.

## To reproduce

Steps to reproduce the behavior:

```
from transformers import XLNetTokenizer

tokenizer = XLNetTokenizer.from_pretrained("xlnet-base-cased")

print(tokenizer)
```

 Output is `None`

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

A tokenizer should be returned instead of `None`.
<!-- A clear and concise description of what you would expect to happen. -->


[Example] Fix a NaN bug in the flax mlm example

## What does this PR do?

Fix a NaN bug in the flax masked language model example. This is a bug introduced in #9133
The min should be max. Otherwise, we will get a NaN.

## Who can review?
@TevenLeScao  @mfuntowicz


Fix distributed evaluation

# What does this PR do?

#10778 introduced a bug in the distributed evaluation, this PR fixes it.
cc @philschmid


Add new community notebook - wav2vec2 with GPT

* Update:community.md, new nb add
* feat: notebook of wav2vec xlsr ctc decoding with gpt logit adjustment
* Update: Wav2vec2 CTC decoding with gpt2 adjustment


[doc] no more bucket


[Example] Updating Question Answering examples for Predict Stage

# What does this PR do?

Fixes #10482
1. It fixes the error that comes while using SQAUD_V2 on question-answering task using `max_val_sample_***`
2. Adds predict method for question-answering examples

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?
@stas00 @sgugger


run_summarization script breaks with label_smoothing_factor and pad_to_max_length true

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: '4.5.0.dev0' (from source)
- Platform: Linux
- Python version: 3.6.9
- PyTorch version (GPU?): '1.8.0' (yes)

## Information

I am running the `examples/seq2seq/run_summarization.py` script with BartForConditionalGeneration.

The script breaks whenever these two parameters are passed together:
- label_smoothing_factor
- pad_to_max_length

It seems that the source of this behaviour is setting collator to `default_data_collator` if `pad_to_max_length` is defined:
https://github.com/huggingface/transformers/blob/5f19c07a704eca4db376b56f950b729dcaa73039/examples/seq2seq/run_summarization.py#L469-L477

while `prepare_decoder_input_ids_from_labels` is only handled by DataCollatorForSeq2Seq:
https://github.com/huggingface/transformers/blob/5f19c07a704eca4db376b56f950b729dcaa73039/src/transformers/data/data_collator.py#L292-L294

It seems to be related with: [10452](https://github.com/huggingface/transformers/issues/10452), where passing a model argument to DataCollatorForSeq2Seq solves the problem
`data_collator = DataCollatorForSeq2Seq(tokenizer, model=model)`

This is more of a question than an issue as it is work in progress. A more general one would be:
Is the `default_data_collator` intended for use with seq2seq models (e.g: Bart), with special cases (like label smoothing) to be handled by `DataCollatorForSeq2Seq`?
Or should `DataCollatorForSeq2Seq` always be used with Seq2SeqTrainer in the future?

The problem arises when using:
* [x ] the official example scripts: (give details below)
examples/seq2seq/run_summarization.py
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x ] an official GLUE/SQUaD task: (give the name) (xsum)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

```
python examples/seq2seq/run_summarization.py \
    --model_name_or_path sshleifer/distilbart-xsum-12-3  \
    --do_train \
    --do_eval \
    --dataset_name xsum \
    --output_dir /tmp/output_dir \
    --per_device_train_batch_size=4 \
    --per_device_eval_batch_size=4 \
    --overwrite_output_dir \
    --predict_with_generate \
    --max_train_samples 500 \
    --max_val_samples 500 \
    --max_source_length 128 \
    --max_target_length 64 \
    --label_smoothing_factor 0.1 \
    --pad_to_max_length true
```

Output:

```
Traceback (most recent call last):
  File "examples/seq2seq/run_summarization.py", line 595, in <module>
    main()
  File "examples/seq2seq/run_summarization.py", line 533, in main
    train_result = trainer.train(resume_from_checkpoint=checkpoint)
  File "/opt/anaconda3/envs/tensorflow2/lib/python3.6/site-packages/transformers/trainer.py", line 1082, in train
    tr_loss += self.training_step(model, inputs)
  File "/opt/anaconda3/envs/tensorflow2/lib/python3.6/site-packages/transformers/trainer.py", line 1472, in training_step
    loss = self.compute_loss(model, inputs)
  File "/opt/anaconda3/envs/tensorflow2/lib/python3.6/site-packages/transformers/trainer.py", line 1511, in compute_loss
    loss = self.label_smoother(outputs, labels)
  File "/opt/anaconda3/envs/tensorflow2/lib/python3.6/site-packages/transformers/trainer_pt_utils.py", line 439, in __call__
    smoothed_loss.masked_fill_(padding_mask, 0.0)
RuntimeError: The expanded size of the tensor (128) must match the existing size (64) at non-singleton dimension 1.  Target sizes: [4, 128, 1].  Tensor sizes: [4, 64, 1]
  0%|
```
<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior
Script works for a parameter set including:
- label_smoothing_factor
- pad_to_max_length

Or info which collator class should be used in the future
<!-- A clear and concise description of what you would expect to happen. -->


HerbertTokenizer doesn't work on version 3.5.1

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.5.1
- Platform: MacOS X, Linux
- Python version: 3.7
- PyTorch version (GPU?): 1.6.0

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

Model hub:

- for issues with a model report at https://discuss.huggingface.co/ and tag the model's creator.

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

## Information

Model I am using (Bert, XLNet ...): allegro/herbert-base-cased

I tried to use official script on model hub page with transformers version 3.5.1. Week ago it worked just fine, but now I am getting error listed below.
@rmroczkowski maybe you have some information on this topic, I saw some new commits on model hub, but they shouldn't change anything

For latest version it works fine with AutoTokenizers (EDIT: only version 4.4 works, I tasted version 3.5.1, 4.0.0, 4.3 and got same error)

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)
I tried importing AutoTokenizers and HerbertTokenizer, but got the same error
`OSError: Can't load tokenizer for 'allegro/herbert-base-cased'. Make sure that:

- 'allegro/herbert-base-cased' is a correct model identifier listed on 'https://huggingface.co/models'

- or 'allegro/herbert-base-cased' is the correct path to a directory containing relevant tokenizer files`

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. install transformers 3.5.1
2. try to use official script from https://huggingface.co/allegro/herbert-base-case

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

tokenizer loads and works


[Deepspeed ZeRO-3] Broken model save on fresh Transformers branch

I have my own model, which utilize two T5 encoders, and I train it via DeepSpeed. It has it's own save_pretrained() and from_pretrained() methods, which makes a custom load/save logic:
https://github.com/exelents/try_t5_siamese/blob/4140194978ac113c45e7370f40b3d9b932d0b35b/siamese_model.py#L80

When I run training and trainer starts to save checkpoint, there are going something strange: weights file for every saved encoder is going to be e few kilobytes - weights are not going to be saved.
On the start of training trainer tries to load checkpoint using model.load_checkpoint(), but it seems this function has it's own loading logic, because it cannot exec my load model logic and throws an error:
`ValueError: [deepspeed] failed to resume from checkpoint ./templates/siamese-t5-small-v1_1-template`
I can comment this code, which loads checkpoint, but then I got described before problem with saving checkpoint...

What should I do to make save my own custom model properly? It worked a month ago, but today I refreshed my Transformers repo and everything has broken.


TypeError: __init__() got an unexpected keyword argument 'filepath' when using RAG model

I was finetuning RAG model with cmd:

python finetune_rag.py \
    --data_dir ../../../../data/ms-marco/ \
    --output_dir ../../../../data/ms-marco/ \
    --model_name_or_path ~/model/rag/rag/rag-sequence-nq \
    --model_type rag_sequence \
    --fp16 \
    --gpus 8 \
    --do_train --do_predict

where ~/model/rag/rag/rag-sequence-nq was completely download from https://huggingface.co/facebook/rag-sequence-nq.

Here is the log:

Model name '/nfs/users/s_xiangru/model/rag/rag/rag-sequence-nq' not found in model shortcut name list (facebook/dpr-question_encoder-single-nq-base, facebook/dpr-question_encoder-multiset-base). Assuming '/nfs/users/s_xiangru/model/rag/rag/rag-sequence-nq' is a path, a model identifier, or url to a directory containing tokenizer files.
Didn't find file /nfs/users/s_xiangru/model/rag/rag/rag-sequence-nq/question_encoder_tokenizer/tokenizer.json. We won't load it.
Didn't find file /nfs/users/s_xiangru/model/rag/rag/rag-sequence-nq/question_encoder_tokenizer/added_tokens.json. We won't load it.
loading file /nfs/users/s_xiangru/model/rag/rag/rag-sequence-nq/question_encoder_tokenizer/vocab.txt
loading file None
loading file None
loading file /nfs/users/s_xiangru/model/rag/rag/rag-sequence-nq/question_encoder_tokenizer/special_tokens_map.json
loading file /nfs/users/s_xiangru/model/rag/rag/rag-sequence-nq/question_encoder_tokenizer/tokenizer_config.json
Model name '/nfs/users/s_xiangru/model/rag/rag/rag-sequence-nq' not found in model shortcut name list (facebook/bart-base, facebook/bart-large, facebook/bart-large-mnli, facebook/bart-large-cnn, facebook/bart-large-xsum, yjernite/bart_eli5). Assuming '/nfs/users/s_xiangru/model/rag/rag/rag-sequence-nq' is a path, a model identifier, or url to a directory containing tokenizer files.
Didn't find file /nfs/users/s_xiangru/model/rag/rag/rag-sequence-nq/generator_tokenizer/tokenizer.json. We won't load it.
Didn't find file /nfs/users/s_xiangru/model/rag/rag/rag-sequence-nq/generator_tokenizer/added_tokens.json. We won't load it.
loading file /nfs/users/s_xiangru/model/rag/rag/rag-sequence-nq/generator_tokenizer/vocab.json
loading file /nfs/users/s_xiangru/model/rag/rag/rag-sequence-nq/generator_tokenizer/merges.txt
loading file None
loading file None
loading file /nfs/users/s_xiangru/model/rag/rag/rag-sequence-nq/generator_tokenizer/special_tokens_map.json
loading file /nfs/users/s_xiangru/model/rag/rag/rag-sequence-nq/generator_tokenizer/tokenizer_config.json
Traceback (most recent call last):
  File "finetune_rag.py", line 629, in <module>
    main(args)
  File "finetune_rag.py", line 597, in main
    checkpoint_callback=get_checkpoint_callback(args.output_dir, model.val_metric),
  File "/nfs/users/s_xiangru/transformers/examples/research_projects/rag/callbacks_rag.py", line 41, in get_checkpoint_callback
    period=1,  # maybe save a checkpoint every time val is run, not just end of epoch.
TypeError: __init__() got an unexpected keyword argument 'filepath'


Can DeepSpeed ZeRO-3 be applied for training?

# 🌟 New model addition

We have applied DeepSpeed v0.3.10(ZeRO-2) on T5 training.
I heard DeepSpeed ZeRO-3 library has been released 10 days ago(8/MAR).
I'd like to adopt ZeRO-3 for our training.
Can this library be applied, especially for T5 training?
Do you have any experience applying this library?
If any, could share your experience?


Add XLSR-Wav2Vec2 Fine-Tuning README.md

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Typo in M2M100 model page

Seems like there's a typo in the [m2m 100 page](https://huggingface.co/facebook/m2m100_418M):
```python
from transformers import M2M100ForConditionalGeneration, M2M100Tokenizer

hi_text = "जीवन एक चॉकलेट बॉक्स की तरह है।"
chinese_text = "生活就像一盒巧克力。"

model = M2M100ForConditionalGeneration.from_pretrained("faGreekook/m2m100_418M")
tokenizer = M2M100Tokenizer.from_pretrained("facebook/m2m100_418M")
```

Pretty sure it should be "facebook" instead of "faGreekook"


How to interpret fine-tuned model results and use model

Hello,

Apologies if this is the wrong forum to ask these kinds of questions, but I was unable to find this in the documentation.

I fine-tuned a seq2seq model on my custom dataset using the tutorial found here: https://github.com/huggingface/transformers/tree/master/examples/seq2seq

I am trying to find out the F1 and EM accuracy for the fine-tuned model, but am not sure how to interpret the output. I've attached a link to the training's output below:
https://github.com/zakerytclarke/transformers/tree/master/modelResults
```
{
    "epoch": 3.0,
    "eval_gen_len": 55.7429,
    "eval_loss": 2.063843250274658,
    "eval_mem_cpu_alloc_delta": 1998448,
    "eval_mem_cpu_peaked_delta": 638828,
    "eval_rouge1": 33.8505,
    "eval_rouge2": 13.1365,
    "eval_rougeL": 27.8332,
    "eval_rougeLsum": 31.5921,
    "eval_runtime": 119.8097,
    "eval_samples": 35,
    "eval_samples_per_second": 0.292
}
```
Can you point me to documentation about how to interpret these results and how I can load my fine-tuned model in order to evaluate it on a new piece of text?

Thanks for your help,
--Zak


Fix bug in input check for LengthGroupSampler

# What does this PR do?

This commit fixes a bug in the LengthGroupSampler where if
model_input_name is not set, the default value is None instead of
"input_ids"

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

I did not write a test for this, but if necessary I can. Neither this sampler nor the distributed version currently have test coverage for the ValueError this bug raises, but it might not be bad to have.

## Who can review?

@patrickvonplaten


add dockerfile for zero optimzier

This PR adds a dockerfile for zero optimzier


Add support for detecting intel-tensorflow version

`intel-tensorflow` pypi package is not currently detected by transformers.

This PR adds support for detecting Intel TF version.


Improve the speed of adding tokens from added_tokens.json

# What does this PR do?

This PR significantly improves the speed of adding tokens from `added_tokens.json`, when it contains a large number of tokens (e.g., 20,000+). When adding one token at a time, it uses `bisect` to insert the token into `PreTrainedTokenizer.unique_no_split_tokens`. Please see a detailed description and motivation in this issue: https://github.com/huggingface/transformers/issues/10676

This change relies on the requirement that `unique_no_split_tokens` is sorted. (I'm not sure if this is a fair assumption, otherwise I can check if it's already sorted before the insertion.)

Fixes #10676

## Benchmark
MacOS Mojave (CPU: 2.6 GHz Intel Core i7)
python==3.7.9
torch==1.7.1+cpu
transformers==3.5.1 or transformers==4.4.2 (similar results)

```python
import json
from timeit import default_timer as timer
from transformers import DistilBertTokenizer

model_dir = '/home/username/saved_model'

# Load a pretrained model's tokenizer, save it to {model_dir}
tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-uncased')
tokenizer.save_pretrained(model_dir)
print('len(tokenizer) of a pretrained model distilbert-base-uncased:', len(tokenizer))

# Generate n values as token suffix, to randomize the insertion position of added tokens
low = 30522
high = 82181   # exclusive
random_suffix = list(range(low, high))
random.shuffle(random_suffix)

# Save the n new tokens with correct indices to added_tokens.json
added_tokens = {f'addedtoken{val}': low + idx for idx, val in enumerate(random_suffix)}
with open(f'{model_dir}/added_tokens.json', 'w') as f:
    json.dump(added_tokens, f)
print(f'saved {len(added_tokens)} tokens to added_tokens.json')

# Load the tokenizer from {model_dir}, and print the elapsed time
start = timer()
tokenizer = DistilBertTokenizer.from_pretrained(model_dir)
print('len(tokenizer) after loading from saved model:', len(tokenizer))
end = timer()
print('Elapsed (seconds):', round(end - start, 3))

# Make sure tokenizer.unique_no_split_tokens remains sorted
all_values = tokenizer.unique_no_split_tokens
assert all(all_values[i+1] > all_values[i] for i in range(len(all_values) - 1))
```

**If we save 21659 tokens in added_tokens.json**, output before the change:
```bash
len(tokenizer) of a pretrained model distilbert-base-uncased: 30522
saved 21659 tokens to added_tokens.json
len(tokenizer) after loading from saved model: 52181
*** Elapsed (seconds): 76.95
```

output after the change:
```bash
len(tokenizer) of a pretrained model distilbert-base-uncased: 30522
saved 21659 tokens to added_tokens.json
len(tokenizer) after loading from saved model: 52181
*** Elapsed (seconds): 0.308
```

**If we save 51659 tokens in added_tokens.json**, output before the change:
```bash
len(tokenizer) of a pretrained model distilbert-base-uncased: 30522
saved 51659 tokens to added_tokens.json
len(tokenizer) after loading from saved model: 82181
*** Elapsed (seconds): 527.795
```

output after the change:
```bash
len(tokenizer) of a pretrained model distilbert-base-uncased: 30522
saved 51659 tokens to added_tokens.json
len(tokenizer) after loading from saved model: 82181
*** Elapsed (seconds): 0.83
```

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

## Who can review?

@lhoestq @LysandreJik

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


EncoderDecoderModel with different model dimensions

## Who can help
@patrickvonplaten, @patil-suraj

## Information

When instantiating an `EncoderDecoderModel` from two pretrained models whose model dimensions are different, a `RunTimeError` occurs at the `CrossAttention` calculation step.

The reason is, that regardless of a potentially different encoder model dimension, the projection layers for key and value are initialized with the decoder model dimension.

This leads to a dimensionality mismatch when performing the matrix multiplication of encoder outputs (encoder model dimension) in the key and value projection layers (decoder model dimension).

Looking a little bit deeper in the API I would suspect it should be easy to provide the correct encoder model dimension to the `Attention` module in most Model implementations and their key/value projection layers, if the `add_cross_attention=True` argument is set. Also, I think the encoder model dimension should be easily accessible via `self.encoder.config.d_model` or something along these lines.

Generally, I think there is no reason against using `EncoderDecoderModel` with `encoder='bert-large-cased'` (`d_model=1024`) and `decoder='gpt2'` (`d_model=768`), but currently this setup doesnt't work.

Thanks a lot for looking into it :)

Best regards
Lars


Smmp batch not divisible by microbatches fix

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes:
Batch size not divisible by microbatches issue in sagemaker model parallel. Following is summary of changes:
1. Updated SequentialDistributedSampler to generate samples of multiples of batchsize.
2. Updated preds_gatherer and labels_gatherer calls to be multiple of batch size.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

@sgugger @philschmid @anirudh2290

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @jplu

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


[trainer] make failure to find a resume checkpoint fatal + tests

As a follow up to https://github.com/huggingface/transformers/pull/10760 this PR:
- makes a failure to find a valid checkpoint to resume from fatal - when an explicit `resume_from_checkpoint` was passed
- extends `test_can_resume_training` to validate this change and also the boolean `resume_from_checkpoint` case.
- adds a small `test_can_resume_training` refactoring - so it's easy to see they are the same args on each invocation.

@sgugger


[examples] document resuming

This PR documents how one can resume training in examples. Thanks to @sgugger for the notes.

@sgugger


Check copies blackify

# What does this PR do?

This PR update the check_copies utils to apply black when checking if a copy has diverged from the original when replacement happen. An example of the problem is given with the diff in `modeling_mobilebert.py` here, where the check copies could not be applied to whole class because of styling divergences.

It also fixes a bug where the check was not applied on functions after the end of the definition (it wasn't checking the function but was stopping at the first unindent when the closing parenthesis was). As a consequence, three files are changed because they diverged from the original function:
- modeling_m2m_100.py
- modeling_roberta.py
- modeling_speech_to_text.py

I'm not sure if the check should be removed on those or not (cc @patil-suraj )