Addition of MMI-antiLM decoding

# 🚀 Feature request

Hugging Face does a great job of including popular decoding strategies such as nucleus sampling, top-k, and temperature. There are also other really interesting decoding strategies for chatbots to fix the response "blandness" or "I don't know" problem, such as using the **Maximum Mutual Information anti-Language Model objective (MMI anti-LM)**. The algorithm is defined in [A Diversity-Promoting Objective Function for Neural Conversation Models](https://www.aclweb.org/anthology/N16-1014.pdf).

## Motivation

I'm requesting this as a feature because I used this in a narrative generation paper. From this point forward I will be using "we" to reference my co-authors (@XiangLi1999 also contributed to the code) and I. The results of our work show that antiLM decoding does in fact help make the generated output more interesting without hurting fluency. Our work is [Decoding Methods for Neural Narrative Generation](https://arxiv.org/abs/2010.07375) and the rest of our code is in our [paper repo](https://github.com/AADeLucia/gpt2-narrative-decoding).

We think others would also be interested in using this decoding method for their work.

## Your contribution

We have a working implementation in a hacked version of the `generation_utils.py` file. It's not pretty (sorry) but maybe a good starting point? The code is in PR #7931.

Also the author's implementation (not huggingface-based) is in [Jiwei Li's repo](https://github.com/jiweil/Neural-Dialogue-Generation).


MMI-antiLM decoding

# What does this PR do?

Implements **Maximum Mutual Information anti-Language Model objective (MMI anti-LM)** decoding from [A Diversity-Promoting Objective Function for Neural Conversation Models](https://www.aclweb.org/anthology/N16-1014.pdf).


update model cards of Illuin models

model card

Related to pretrained model cards

# What does this PR do?

Updates models cards of Illuin uploaded models to provide various informations.

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


Reformer model does not work with padded sequences

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.4.0
- Platform: Linux
- Python version: 3.8.5
- PyTorch version (GPU?): 1.6.0 (No)
- Tensorflow version (GPU?): 2.3.0
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 blenderbot: @mariamabarham
 Bart: @sshleifer
 Marian: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->
@patrickvonplaten
## Information

Model I am using (Bert, XLNet ...): Reformer

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: (give the name) CommonGen
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:
```python
from transformers import ReformerTokenizer, ReformerModel

tokenizer = ReformerTokenizer.from_pretrained('google/reformer-crime-and-punishment')
seq = tokenizer(['Hello this is a test.', 'This is a test as well'], padding=True, return_tensors='pt')
reformer = ReformerModel.from_pretrained('google/reformer-crime-and-punishment')
out = reformer(**seq)
```
```python
Traceback (most recent call last):
  File "reformerbug.py", line 20, in <module>
    out = reformer(**seq)
  File "/home/fabian/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/fabian/.local/lib/python3.8/site-packages/transformers/modeling_reformer.py", line 2096, in forward
    embedding_output = self.embeddings(
  File "/home/fabian/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/fabian/.local/lib/python3.8/site-packages/transformers/modeling_reformer.py", line 252, in forward
    inputs_embeds = self.word_embeddings(input_ids)
  File "/home/fabian/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/fabian/.local/lib/python3.8/site-packages/torch/nn/modules/sparse.py", line 124, in forward
    return F.embedding(
  File "/home/fabian/.local/lib/python3.8/site-packages/torch/nn/functional.py", line 1814, in embedding
    return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
IndexError: index out of range in self
```
<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior
The model should properly calculate the forward pass given the encoded sequence.
<!-- A clear and concise description of what you would expect to happen. -->


Respect the 119 line chars

Respect the 119 lien chars limit in the model summary


[ProphetNet] add model summary

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


Validation loop gives OOM when finetuning T5

While finetuning T5-base on summarization task, using `--sortish_sampler` it gives an OOM error starting from a particular index during the validation loop. After removing those indices and training again, I still get the OOM error, but in the second validation loop, whereas the validation loop worked well the first time in the 0th epoch. I have finetuned t5-base previously as well on the same dataset and the same environment, and it never gave this error.
I am using a batch size of 1 during training and evaluation both.

I am using Colab to finetune the model on a single GPU.
GPU specs:
GPU: Tesla K80
RAM: 12GB

This is the traceback:

Epoch 0:  91% 5644/6231 [25:04<02:36,  3.75it/s, loss=2.089, v_num=16]
Epoch 0:  91% 5645/6231 [25:06<02:36,  3.75it/s, loss=2.089, v_num=16]
Epoch 0:  91% 5646/6231 [25:07<02:36,  3.75it/s, loss=2.089, v_num=16]
Epoch 0:  91% 5647/6231 [25:08<02:35,  3.74it/s, loss=2.089, v_num=16]
Epoch 1:  84% 5247/6231 [18:08<03:24,  4.82it/s, loss=1.859, v_num=16]
Validating: 0it [00:00, ?it/s]
Epoch 1:  84% 5248/6231 [18:10<03:24,  4.81it/s, loss=1.859, v_num=16]
Epoch 1:  84% 5249/6231 [18:11<03:24,  4.81it/s, loss=1.859, v_num=16]Traceback (most recent call last):
  File "finetune.py", line 441, in <module>
    main(args)
  File "finetune.py", line 416, in main
    logger=logger,
  File "/content/drive/My Drive/Colab Notebooks/transformers_20_Oct_summarization/transformers/examples/lightning_base.py", line 386, in generic_train
    trainer.fit(model)
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/trainer.py", line 440, in fit
    results = self.accelerator_backend.train()
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/accelerators/gpu_accelerator.py", line 54, in train
    results = self.train_or_test()
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/accelerators/accelerator.py", line 66, in train_or_test
    results = self.trainer.train()
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/trainer.py", line 483, in train
    self.train_loop.run_training_epoch()
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/training_loop.py", line 569, in run_training_epoch
    self.trainer.run_evaluation(test_mode=False)
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/trainer.py", line 568, in run_evaluation
    output = self.evaluation_loop.evaluation_step(test_mode, batch, batch_idx, dataloader_idx)
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/evaluation_loop.py", line 171, in evaluation_step
    output = self.trainer.accelerator_backend.validation_step(args)
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/accelerators/gpu_accelerator.py", line 76, in validation_step
    output = self.__validation_step(args)
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/accelerators/gpu_accelerator.py", line 86, in __validation_step
    output = self.trainer.model.validation_step(*args)
  File "finetune.py", line 181, in validation_step
    return self._generative_step(batch)
  File "finetune.py", line 221, in _generative_step
    max_length=self.eval_max_length,
  File "/usr/local/lib/python3.6/dist-packages/torch/autograd/grad_mode.py", line 15, in decorate_context
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/transformers/generation_utils.py", line 489, in generate
    model_kwargs=model_kwargs,
  File "/usr/local/lib/python3.6/dist-packages/transformers/generation_utils.py", line 665, in _generate_beam_search
    outputs = self(**model_inputs, return_dict=True)  # (batch_size * num_beams, cur_len, vocab_size)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_t5.py", line 1212, in forward
    return_dict=return_dict,
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_t5.py", line 767, in forward
    output_attentions=output_attentions,
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_t5.py", line 556, in forward
    output_attentions=output_attentions,
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_t5.py", line 478, in forward
    output_attentions=output_attentions,
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_t5.py", line 374, in forward
    q, k.transpose(3, 2)
RuntimeError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 14.73 GiB total capacity; 13.70 GiB already allocated; 13.88 MiB free; 13.72 GiB reserved in total by PyTorch)
Epoch 1:  84%|████████▍ | 5249/6231 [18:13<03:24,  4.80it/s, loss=1.859, v_num=16]

Any help is appreciated. :)


# Add whole word mask support for lm fine-tune

This PR add support for **wwm** (whole word mask) proxy when fine-tune BERT like model.
And it can be divided into two part : English Model Support and Chinese Model Support

For English, it's simple. The original tokenizer res contains symbols like '##ing'.
I just use the same mask proxy in [data_collator.py](https://github.com/wlhgtc/transformers/blob/master/src/transformers/data/data_collator.py#L168) by [Google.](https://github.com/google-research/bert/blob/master/create_pretraining_data.py#L342)

For Chinese, it's hard. We need to rely on (word level) tokenizer,  cause BERT is char level in Chinese.
So I do things as follow to get word level tokens:
1.  add get info code in [chinese_ref.py](https://github.com/wlhgtc/transformers/blob/master/examples/language-modeling/chinese_ref.py#L79)
2. create a new dataset to keep ref info [language_model.py](https://github.com/wlhgtc/transformers/blob/master/src/transformers/data/datasets/language_modeling.py#L117)
3. create word level ref according to ref [data_collator.py](https://github.com/wlhgtc/transformers/blob/master/src/transformers/data/data_collator.py#L150)

Then, it's all same to English.

And I add two parameters (`wwm` and `chinese_ref_path` ) to run lm.


EncoderDecoderModel not working with DDP

## Environment info
- `transformers` version: 3.3.1
- Platform: Linux-3.10.0-1127.19.1.el7.x86_64-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.8
- PyTorch version (GPU?): 1.6.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Using Distributed

### Who can help
 Trainer: @sgugger
 EncoderDecoderModel: @patrickvonplaten

## Information
I am using a combination of `distilroberta-base` as encoder and `distilgpt2` as a decoder. Used `EncoderDecoderModel` for model initialisation and `Trainer` class for fine-tuning model. For launching distributed processes, used `torch.distributed.launch`.

Below is the model config.
```
{
    "add_cross_attention": false,
    "architectures": null,
    "bad_words_ids": null,
    "bos_token_id": null,
    "chunk_size_feed_forward": 0,
    "decoder": {
        "_num_labels": 1,
        "activation_function": "gelu_new",
        "add_cross_attention": true,
        "architectures": [
            "GPT2LMHeadModel"
        ],
        "attn_pdrop": 0.1,
        "bad_words_ids": null,
        "bos_token_id": 50256,
        "chunk_size_feed_forward": 0,
        "decoder_start_token_id": null,
        "do_sample": false,
        "early_stopping": false,
        "embd_pdrop": 0.1,
        "eos_token_id": 50256,
        "finetuning_task": null,
        "gradient_checkpointing": false,
        "id2label": {
            "0": "LABEL_0"
        },
        "initializer_range": 0.02,
        "is_decoder": true,
        "is_encoder_decoder": false,
        "label2id": {
            "LABEL_0": 0
        },
        "layer_norm_epsilon": 1e-05,
        "length_penalty": 1.0,
        "max_length": 20,
        "min_length": 0,
        "model_type": "gpt2",
        "n_ctx": 1024,
        "n_embd": 768,
        "n_head": 12,
        "n_inner": null,
        "n_layer": 6,
        "n_positions": 1024,
        "no_repeat_ngram_size": 0,
        "num_beams": 1,
        "num_return_sequences": 1,
        "output_attentions": false,
        "output_hidden_states": false,
        "pad_token_id": null,
        "prefix": null,
        "pruned_heads": {},
        "repetition_penalty": 1.0,
        "resid_pdrop": 0.1,
        "return_dict": false,
        "sep_token_id": null,
        "summary_activation": null,
        "summary_first_dropout": 0.1,
        "summary_proj_to_labels": true,
        "summary_type": "cls_index",
        "summary_use_proj": true,
        "task_specific_params": {
            "text-generation": {
                "do_sample": true,
                "max_length": 50
            }
        },
        "temperature": 1.0,
        "tie_encoder_decoder": false,
        "tie_word_embeddings": true,
        "tokenizer_class": null,
        "top_k": 50,
        "top_p": 1.0,
        "torchscript": false,
        "use_bfloat16": false,
        "use_cache": true,
        "vocab_size": 50257,
        "xla_device": null
    },
    "decoder_start_token_id": null,
    "do_sample": false,
    "early_stopping": false,
    "encoder": {
        "add_cross_attention": false,
        "architectures": [
            "RobertaForMaskedLM"
        ],
        "attention_probs_dropout_prob": 0.1,
        "bad_words_ids": null,
        "bos_token_id": 0,
        "chunk_size_feed_forward": 0,
        "decoder_start_token_id": null,
        "do_sample": false,
        "early_stopping": false,
        "eos_token_id": 2,
        "finetuning_task": null,
        "gradient_checkpointing": false,
        "hidden_act": "gelu",
        "hidden_dropout_prob": 0.1,
        "hidden_size": 768,
        "id2label": {
            "0": "LABEL_0",
            "1": "LABEL_1"
        },
        "initializer_range": 0.02,
        "intermediate_size": 3072,
        "is_decoder": false,
        "is_encoder_decoder": false,
        "label2id": {
            "LABEL_0": 0,
            "LABEL_1": 1
        },
        "layer_norm_eps": 1e-05,
        "length_penalty": 1.0,
        "max_length": 20,
        "max_position_embeddings": 514,
        "min_length": 0,
        "model_type": "roberta",
        "no_repeat_ngram_size": 0,
        "num_attention_heads": 12,
        "num_beams": 1,
        "num_hidden_layers": 1,
        "num_return_sequences": 1,
        "output_attentions": false,
        "output_hidden_states": false,
        "pad_token_id": 1,
        "prefix": null,
        "pruned_heads": {},
        "repetition_penalty": 1.0,
        "return_dict": false,
        "sep_token_id": null,
        "task_specific_params": null,
        "temperature": 1.0,
        "tie_encoder_decoder": false,
        "tie_word_embeddings": true,
        "tokenizer_class": null,
        "top_k": 50,
        "top_p": 1.0,
        "torchscript": false,
        "type_vocab_size": 1,
        "use_bfloat16": false,
        "use_cache": true,
        "vocab_size": 50265,
        "xla_device": null
    },
    "eos_token_id": null,
    "finetuning_task": null,
    "id2label": {
        "0": "LABEL_0",
        "1": "LABEL_1"
    },
    "is_decoder": false,
    "is_encoder_decoder": true,
    "label2id": {
        "LABEL_0": 0,
        "LABEL_1": 1
    },
    "length_penalty": 1.0,
    "max_length": 20,
    "min_length": 0,
    "model_type": "encoder_decoder",
    "no_repeat_ngram_size": 0,
    "num_beams": 1,
    "num_return_sequences": 1,
    "output_attentions": false,
    "output_hidden_states": false,
    "pad_token_id": null,
    "prefix": null,
    "pruned_heads": {},
    "repetition_penalty": 1.0,
    "return_dict": false,
    "sep_token_id": null,
    "task_specific_params": null,
    "temperature": 1.0,
    "tie_encoder_decoder": false,
    "tie_word_embeddings": true,
    "tokenizer_class": null,
    "top_k": 50,
    "top_p": 1.0,
    "torchscript": false,
    "use_bfloat16": false,
    "use_cache": true,
    "xla_device": null
}
```

Got this error when using `Trainer` to fine-tune this model.
```
Traceback (most recent call last):
  File "training/run_training.py", line 200, in <module>
    raise e
  File "training/run_training.py", line 197, in <module>
    main()
  File "training/run_training.py", line 163, in main
    trainer.train(model_path=model_path)
  File "/usr/local/lib/python3.6/dist-packages/transformers/trainer.py", line 768, in train
    tr_loss += self.training_step(model, inputs)
  File "/usr/local/lib/python3.6/dist-packages/transformers/trainer.py", line 1116, in training_step
    loss = self.compute_loss(model, inputs)
  File "/usr/local/lib/python3.6/dist-packages/transformers/trainer.py", line 1142, in compute_loss
    outputs = model(**inputs)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/parallel/distributed.py", line 526, in forward
    self.reducer.prepare_for_backward(list(_find_tensors(output)))
RuntimeError: Expected to have finished reduction in the prior iteration before starting a new one. This error indicates that your module has parameters that were not used in producing loss. You can enable unused parameter detection by (1) passing the keyword argument `find_unused_parameters=True` to `torch.nn.parallel.DistributedDataParallel`; (2) making sure all `forward` function outputs participate in calculating loss. If you already have done the above two steps, then the distributed data parallel module wasn't able to locate the output tensors in the return value of your module's `forward` function. Please include the loss function and the structure of the return value of `forward` of your module when reporting this issue (e.g. list, dict, iterable).
```

I checked this variable is `find_unused_parameters` set as true. Also above training works well in a single GPU setting.


Loading a pytorch quantized model

Hi, I quantized a pre-trained model via **torch.quantization.quantize_dynamic** and saved it using **save_pretrained**.
based on https://colab.research.google.com/github/pytorch/tutorials/blob/gh-pages/_downloads/dynamic_quantization_bert_tutorial.ipynb#scrollTo=foe-dVxHIgOC .
On reloading it using **from_pretrained** again the model blows up to its original size and the resultant predictions are garbage as well.
Is there a way to properly save the quantized weights and reload them?
Opening an issue because I couldn't find a resolution for the same.
@mfuntowicz @VictorSanh @patrickvonplaten


Is it possible to recommend the deployment method for implementing trained mode

# ❓ Questions & Help

I trained a model the Class Trainer. Could you recommend some high performance deployment proposal ？


[testing] experiment with a different way of skipping torch-only test modules

This is an experiment.

I was getting irked by `require_pytorch` followed by yet another `if is_torch_available()` code in many test modules, specifically this part:

```
@require_torch
class BARTModelTest(ModelTesterMixin, unittest.TestCase):
    all_model_classes = (
        (BartModel, BartForConditionalGeneration, BartForSequenceClassification, BartForQuestionAnswering)
        if is_torch_available()
        else ()
    )
    all_generative_model_classes = (BartForConditionalGeneration,) if is_torch_available() else ()
```
`require_torch` doesn't stop the parser from compiling the rest of the code, so the ugly workaround is used.

I tried to find a better solution, to tell the parser to ignore the whole class, since we did tell it to skip it - but didn't succeed.

But then I noticed that all of the module classes/tests requires pytorch, so I thought why not skip the whole module and not need to repeatedly ask if pytorch is available. That is:

```
if not is_torch_available():
    raise unittest.SkipTest("Skip the whole module as it requires pytorch")
```
and then we can code worry-free, removing any torch checks. We can then alias this whole thing in `testing_utils.py` and call as something like:

```
from testing_utils import skip_module_require_torch
skip_module_require_torch()
```

This PR is such possible solution applied to just one pure pytorch test module. To see it in action, run:

```
 USE_TF=1 pytest tests/test_modeling_bart.py
```

The only drawback is that it doesn't count/report any of the skipped tests, so we get just:

```
collected 0 items / 1 skipped
```
from pytest. But this will only happen on _tf CI job, so it doesn't matter anyway.

We can do exactly the same for tf-only tests, with its own `skip_module_require_tf`.

As a bonus the test suite will run marginally faster for those pt/tf-only jobs, as it won't need to load/parse any modules - should be a very negligible improvement.

The current way is just fine. But I thought I'd share my experiment in case perhaps it'd lead to a more readable code.

Thank you for reading.

@LysandreJik, @sgugger, @sshleifer, @patrickvonplaten


what's the values of start_positon and end_position while the answer is impossible in run_squad.py

# ❓ Questions & Help
### Details
 I want to know what's the values of `start_positon` and `end_position` while the answer is **impossible** in `run_squad.py`.
`start_positions = end_postions = -1` OR `start_positions = end_postions = 0`


Expose the Flax code quality problems

# What does this PR do?

This PR is just there to expose the problems related to the code quality with objects introduced in the flax PR. #7914 contains a fix to a few of them, maybe all.


Add Flax dummy objects

# What does this PR do?

Following the first JAX models, this PR adds the dummy objects to make sure the library always has the same objects available.
cc @mfuntowicz for information


New run glue script

# What does this PR do?

This PR cleans up the `run_glue.py` script to use the Datasets library. Along the way it adds a few fixes in Trainer. The script supports all glue tasks as well as custom user tasks (passed along with a training and validation file in csv or json format). It has been tested on the following setups:
- single GPU
- multi-GPU with DataParallel
- multi-GPU with DistributedDataParallel
- TPU

The README has been updated to reflect the changes, there is just one breaking change from before which is that `data_dir` is not an accepted argument anymore (since Datasets will take care of downloading the data files).


TypeError: __init__() got an unexpected keyword argument 'vocab_file' in transformers/tokenization_gpt2.py", line 380

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.3.1
- `tokenizers` version: 0.9.2
- Platform: Linux-3.10.0-1062.4.1.el7.x86_64-x86_64-with-redhat-7.7-Maipo
- Python version: 3.7.6
- PyTorch version (GPU?): 1.6.0 (False)
- Tensorflow version (GPU?): 2.3.1 (False)
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help @mfuntowicz
## Information

Model I am using (Bert, XLNet ...): RoBERTa-base

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)
fairseq

## To reproduce

I use the **RobertaTokenizerFast** and it seems an arg name mismatch.
Steps to reproduce the behavior:

1. self.tokenizer = RobertaTokenizerFast.from_pretrained('roberta-base', cache_dir=args.cache_dir)

In transformers.tokenization_gpt2.py L376 it is:
`ByteLevelBPETokenizer(
                vocab_file=vocab_file,
                merges_file=merges_file,
                add_prefix_space=add_prefix_space,
                trim_offsets=trim_offsets,
            )`

But in tokenizers.implementations.ByteLevelBPETokenizer it is expected to be `vocab`.

## Expected behavior
`  File "/zfs1/hdaqing/rum20/kp/fairseq-kpg/fairseq/data/encoders/hf_bpe.py", line 31, in __init__
    self.tokenizer = RobertaTokenizerFast.from_pretrained(args.pretrained_model, cache_dir=args.cache_dir)
  File "/ihome/hdaqing/rum20/anaconda3/envs/kp/lib/python3.7/site-packages/transformers/tokenization_utils_base.py", line 1428, in from_pretrained
    return cls._from_pretrained(*inputs, **kwargs)
  File "/ihome/hdaqing/rum20/anaconda3/envs/kp/lib/python3.7/site-packages/transformers/tokenization_utils_base.py", line 1575, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
  File "/ihome/hdaqing/rum20/anaconda3/envs/kp/lib/python3.7/site-packages/transformers/tokenization_roberta.py", line 380, in __init__
    **kwargs,
  File "/ihome/hdaqing/rum20/anaconda3/envs/kp/lib/python3.7/site-packages/transformers/tokenization_gpt2.py", line 380, in __init__
    trim_offsets=trim_offsets,
TypeError: __init__() got an unexpected keyword argument 'vocab_file'`


[EncoderDecoder] Fix Typo

# What does this PR do?

Remove dead code

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


[flax] fix repo_check

Unless, this is actually a problem, this PR adds `modeling_flax_utils` to ignore list. otherwise currently it expects to have  `tests/test_modeling_flax_utils.py` for this module.

It also adds the 2 new tests that don't run common tests to `TEST_FILES_WITH_NO_COMMON_TESTS`

For context please see: https://github.com/huggingface/transformers/pull/3722#issuecomment-712360415

now check_repo is happy.

@sgugger


Hello,
I understand that when I add the `add_prefix_space=True` option in the BPE tokenizer statement, the tokenizer will add a space in the beginning of every sequence.

Is there some specific advantages of using the `add_prefix_space=True` option for BPE tokenizer (compared to when I don't use the option), given that all my sequences start without a space in the beginning.?

Thanks,


run_tf_text_classification.py giving "ValueError: too many values to unpack"

I am trying to run this script for token classification

https://github.com/huggingface/transformers/blob/master/examples/text-classification/run_tf_text_classification.py

Accoring to the instructions here

https://github.com/huggingface/transformers/tree/master/examples/text-classification

I formatted the data according> to the instructions

>the CSV files must have a header corresponding to the column names and not more than three columns: one column for the id, one column for the text and another column for a second piece of text in case of an entailment classification for example.

However, I am getting this error

> Dataset csv downloaded and prepared to /root/.cache/huggingface/datasets/csv/default-57112360018dd326/0.0.0/49187751790fa4d820300fd4d0707896e5b941f1a9c644652645b866716a4ac4. Subsequent calls will reuse this data.
> 10/19/2020 18:03:50 - INFO - filelock -   Lock 140266200732560 released on /root/.cache/huggingface/datasets/_root_.cache_huggingface_datasets_csv_default-57112360018dd326_0.0.0_49187751790fa4d820300fd4d0707896e5b941f1a9c644652645b866716a4ac4.lock
>   0% 0/5 [00:00<?, ?ba/s]Traceback (most recent call last):
>   File "/content/transformers/examples/text-classification/run_tf_text_classification.py", line 292, in <module>
>     main()
>   File "/content/transformers/examples/text-classification/run_tf_text_classification.py", line 231, in main
>     max_seq_length=data_args.max_seq_length,
>   File "/content/transformers/examples/text-classification/run_tf_text_classification.py", line 68, in get_tfds
>     batched=True,
>   File "/usr/local/lib/python3.6/dist-packages/datasets/arrow_dataset.py", line 1256, in map
>     update_data=update_data,
>   File "/usr/local/lib/python3.6/dist-packages/datasets/arrow_dataset.py", line 156, in wrapper
>     out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
>   File "/usr/local/lib/python3.6/dist-packages/datasets/fingerprint.py", line 163, in wrapper
>     out = func(self, *args, **kwargs)
>   File "/usr/local/lib/python3.6/dist-packages/datasets/arrow_dataset.py", line 1517, in _map_single
>     batch, indices, check_same_num_examples=len(self.list_indexes()) > 0, offset=offset
>   File "/usr/local/lib/python3.6/dist-packages/datasets/arrow_dataset.py", line 1435, in apply_function_on_filtered_inputs
>     function(*fn_args, effective_indices, **fn_kwargs) if with_indices else function(*fn_args, **fn_kwargs)
>   File "/content/transformers/examples/text-classification/run_tf_text_classification.py", line 66, in <lambda>
>     padding="max_length",
>   File "/usr/local/lib/python3.6/dist-packages/transformers/tokenization_utils_base.py", line 2323, in batch_encode_plus
>     **kwargs,
>   File "/usr/local/lib/python3.6/dist-packages/transformers/tokenization_utils.py", line 560, in _batch_encode_plus
>     ids, pair_ids = ids_or_pair_ids
> ValueError: too many values to unpack (expected 2)
>   0% 0/5 [00:00<?, ?ba/s]

It looks like the issue may be the script itself. I was having a previous issue running the script, and it looks like it was due to the datasets library

https://github.com/huggingface/datasets/issues/705#event-3839135529

It looks like the error is now with the script, or possibly the tokenizer. It sort of looks like the training wants only two types inputs, but is being passed all of the inputs from `batch_encode_plus`, which may be more than two (token id type, attention id type, segment id type, etc)

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- Platform: colab
- Python version: version 3, colab default
- PyTorch version (GPU?): colab default
- Tensorflow version (GPU?): colab default
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

### Who can help

I'm not sure, most likely the bug seems to be due to the example script itself, but could be the dataset, or tokenizer.

## Information

Model I am using: Bert, specifically scibert

The problem arises when using:
* [x ] the official example scripts: (give details below)

The tasks I am working on is:
* [x ] my own task or dataset: I am working with the chemprot dataset, for token classification. I following the instructions to have the data in a csv file, with two columns (one for label, another for text), and headers.

## To reproduce

Here is a colab notebook of the issue.

https://colab.research.google.com/drive/1r3XCKYA8RBtfYmU2jqHVJT-uTt1ii04S?usp=sharing

## Expected behavior

Should train without error.


[Docstring] fix t5 training docstring

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes T5 docstring according to recent tokenizer changes.

Fixes #7904

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


[T5] Ignore sentinel indices for unsupervised denoising / masking objective?

The [docs](https://huggingface.co/transformers/model_doc/t5.html#training) state that the masked language modeling objective is simply

```
input_ids = tokenizer.encode('The <extra_id_0> walks in <extra_id_1> park', return_tensors='pt')
labels = tokenizer.encode('<extra_id_0> cute dog <extra_id_1> the <extra_id_2> </s>', return_tensors='pt')
model(input_ids=input_ids, labels=labels)
```

I was wondering if I need to manually set the `additional_special_tokens_ids` (corresponding to the `<extra_id_#>` sentinels) in the `labels` to `-100` during training so that they are ignored by the loss, as I believe would be the case for the `[MASK]` tokens in BERT? It seems that at least the `pad_token_id` is ignored in [`examples/seq2seq`](https://github.com/huggingface/transformers/blob/a09fe140c1c059baf05c4f97e5b4e83c719608db/examples/seq2seq/finetune.py#L153), but it's not clear if this ought to be true for the sentinels as well. My suspicion is _no_, but since there's no canonical MLM code for T5, I figured it was worth checking.

(I asked this in the forums and in a somewhat related issue, but was recommended to post here & tag @patrickvonplaten / @thomwolf)


pegasus/cnn_dm  12-2 distillation performing poorly

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.1.0
- Platform: Linux-5.4.0-1028-aws-x86_64-with-debian-buster-sid
- Python version: 3.7.6
- PyTorch version (GPU?): 1.6.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

### Who can help

@sshleifer
## Information

I am trying to distil the pegasus model to reduce the runtime and memory requirements. I am following **No Teacher Distillation** approach. However, the model generates bad quality text.

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: (give the name): CNN
* [ ] my own task or dataset: (give details below)

## To reproduce
I have trained the model using below command:

**Download data:**
wget https://cdn-datasets.huggingface.co/summarization/cnn_dm_v2.tgz
tar -xzvf cnn_dm_v2.tgz  # empty lines removed
mv cnn_cln cnn_dm

**Command to train:**
python finetune.py --learning_rate=3e-5 --do_train --do_predict --val_check_interval 0.25 --n_val 500 --num_train_epochs 6 --freeze_encoder --freeze_embeds --data_dir ./cnn_dm/ --max_target_length 142 --val_max_target_length=142 --train_batch_size=1 --eval_batch_size=1 --gradient_accumulation_steps=256 --model_name_or_path sshleifer/student_pegasus_cnn_12_2 --tokenizer_name google/pegasus-cnn_dailymail --warmup_steps 500 --output_dir distilpegasus-cnn-12-2 --gpus 1 --adafactor --num_workers=0 --fp16_opt_level=O1 --fp16

**Inference code:**

```
from transformers import PegasusForConditionalGeneration, PegasusTokenizer, PegasusConfig
import torch

PEGASUS_MODEL = '/home/ubuntu/finetune/transformers/examples/seq2seq/distilpegasus-cnn-12-2/best_tfmr'
PEGASUS_TOKENIZER = 'google/pegasus-cnn_dailymail'

class PegasusSummarizer:
    def __init__(self):
        self.torch_device = 'cpu'
        self.tokenizer = PegasusTokenizer.from_pretrained(PEGASUS_TOKENIZER)
        self.model = PegasusForConditionalGeneration.from_pretrained(PEGASUS_MODEL).to(self.torch_device)

    def summarize(self, text):
        src_text = text
        batch = self.tokenizer.prepare_seq2seq_batch([src_text],truncation=True,padding='longest').to(self.torch_device)
        translated = self.model.generate(**batch)
        tgt_text = self.tokenizer.batch_decode(translated, skip_special_tokens=True)
        return tgt_text

summarizer = PegasusSummarizer()
print(summarizer.summarize('''(CNN)For the first time in eight years, a TV legend returned to doing what he does best. Contestants told to "come on down!" on the April 1 edition of "The Price Is Right" encountered not host Drew Carey but another familiar face in charge of the proceedings. Instead, there was Bob Barker, who hosted the TV game show for 35 years before stepping down in 2007. Looking spry at 91, Barker handled the first price-guessing game of the show, the classic "Lucky Seven," before turning hosting duties over to Carey, who finished up. Despite being away from the show for most of the past eight years, Barker didn't seem to miss a beat.'''))

```
**Output:** ['"It\'s time for the first time in a five-year anniversary of the show.']
**Output of google/pegasus-cnn_dailymail model**:['Barker hosted "The Price Is Right" for 35 years.<n>He stepped down in 2007.']

test_results.txt output:
src_pad_frac = tensor(0., device='cuda:0')
src_pad_tok = tensor(0, device='cuda:0')
step_count = 26
test_avg_gen_len = 48.63716275021758
test_avg_gen_time = 1.3503953615824382
test_avg_loss = 3.6937525272369385
test_avg_rouge1 = 19.983542428198433
test_avg_rouge2 = 4.130034786771105
test_avg_rougeL = 14.352700217580503
test_avg_rougeLsum = 18.460456248912102
test_loss = tensor(3.6938, device='cuda:0')
test_rouge2 = tensor(4.1300, device='cuda:0')
tpb = tensor(511, device='cuda:0')
val_avg_gen_len = 50.144
val_avg_gen_time = 1.513235685825348
val_avg_loss = 3.77506422996521
val_avg_rouge1 = 16.9548154
val_avg_rouge2 = 3.1666046
val_avg_rougeL = 12.980990400000001
val_avg_rougeLsum = 15.404284
val_loss = tensor(3.7751, device='cuda:0')
val_rouge2 = tensor(3.1666, device='cuda:0')

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior
I expect the output to be much cleaner and higher Rouge score. Any help in this regard would be of great help.

I am trying to retrain the model by removing  **--freeze_encoder**.
<!-- A clear and concise description of what you would expect to happen. -->


[Model] M2M-100 Multilingual machine translation

# 🌟 New model addition

## Model description
Facebook AI is introducing,
M2M-100
 the first multilingual machine translation (MMT) model that translates between any pair of 100 languages without relying on English data.
<!-- Important information -->

## Open source status

* [x] the model implementation is available: (give details)  https://github.com/pytorch/fairseq/tree/master/examples/m2m_100?fbclid=IwAR2Oqew-PAwZpTmHMrq_yiXN2dwdzzbTMZ-4HfbNKfdoZ_M5TpQiPY3dYFo
* [x] the model weights are available: (give details) https://dl.fbaipublicfiles.com/m2m_100/12b_last_checkpoint.pt
* [ ] who are the authors: (mention them, if possible by @gh-username)


Reproducing Bart Xsum from Bart Large

 Summarization: @sshleifer
 Bart: @sshleifer

## Information

I'm trying to finetune bart large for Xsum and unable to reproduce the results from the paper.

When I try eval with facebook/bart-large-xsum, I get R1=45.3595, RLSum=37.1717 so I assume my eval script is working ok. For finetuning bart large, I use the same config as bart-large-xsum with vocab size=50265 to enable starting from bart-large. However, I am unable to reach the same scores. The best I have is R1=45.4188, RLSum=36.6986 with LR=1.2e-4, gbs=128 and --max_target_length=60 --max_source_length=1024 --val_check_interval 0.1 --val_max_target_length=60 --warmup_steps 50 --max_steps 5000.

How can I reproduce the results?


labels and decoder_input_ids to Glossary

Completes the glossary with entries for `labels` and `decoder_input_ids`.

Closes https://github.com/huggingface/transformers/issues/7865

Pinging @sshleifer and @patrickvonplaten for advice regarding the `decoder_input_ids`, @sgugger for docs.


[RAG] How to extract generated strings from `RetrievAugLMMarginOutput`

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers
     -->
How to extract generated strings from `RetrievAugLMMarginOutput`?

## Details
<!-- Description of your issue -->

When using `RagSequenceForGeneration` and `retriever` separately we can't use `model.generate` (refer #7829). And calling `model.__call__` directly return `RetrievAugLMMarginOutput`. I not able to to find way to extract `generated_ids` from it.

<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:


T5 Docs training example has shifted labels

https://github.com/huggingface/transformers/blob/master/docs/source/model_doc/t5.rst#L42

Here is that link quoted:

#### Unsupervised denoising training
In teacher-forcing style, the target sequence is then appended by the EOS token and corresponds to the `labels`.

In this setup spans of the input sequence are masked by so-called sentinel tokens (*a.k.a* unique mask tokens)  and the output sequence is formed as a concatenation of the same sentinel tokens and the *real* masked tokens.

Each sentinel token represents a unique mask token for this sentence and should start with `<extra_id_0>`, `<extra_id_1>`, ... up to `<extra_id_99>`. As a default, 100 sentinel tokens are available in `transformers.T5Tokenizer`.

For instance, the sentence "The cute dog walks in the park" with the masks put on "cute dog" and "the" should be processed as follows:

```python
  input_ids = tokenizer.encode('The <extra_id_0> walks in <extra_id_1> park', return_tensors='pt')
  labels = tokenizer.encode('<extra_id_0> cute dog <extra_id_1> the <extra_id_2> </s>', return_tensors='pt')
  # the forward function automatically creates the correct decoder_input_ids
  model(input_ids=input_ids, labels=labels)

```

1) Shouldn't the labels be unshifted, given that `decoder_input_ids = shift_right(labels)` @patrickvonplaten @patil-suraj ?

2) @craffel does this look correct to you?


Modelling Encoder-Decoder | Error :- `decoder_config` used before intialisation

Getting error when sending `decoder_config` as a parameter while initializing an encoder-decoder model from pretrained.

# What does this PR do?
fixes "UnboundLocalError: local variable 'decoder_config' referenced before assignment"

## Who can review?
@patrickvonplaten @sgugger


change TokenClassificationTask class methods to static methods

Since we do not require self in the class methods of TokenClassificationTask we should probably switch to static methods. Also, since the class TokenClassificationTask does not contain a constructor it is currently unusable as is. By switching to static methods this fixes the issue of having to document the intent of the broken class.

Also, since the get_labels and read_examples_from_file methods are ought to be implemented. Static method definitions are unchanged even after inheritance, which means that it can be overridden, similar to other class methods.

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


GPT2Tokenizer strips spaces surrounding special tokens

## Environment info

- `transformers` version: 3.3.1
- Platform: Darwin-19.6.0-x86_64-i386-64bit
- Python version: 3.7.9
- PyTorch version (GPU?): 1.6.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help

@mfuntowicz, this seems to be an issue related to tokenization. So I hope you are the right person to ping here.

## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Minimal working example:

```py
import os
os.makedirs('models/', exist_ok=True)

from transformers import GPT2Tokenizer
from tokenizers import ByteLevelBPETokenizer

open('train.txt', 'w').write('Training data including a <special> token.')

special_tokens = ['<special>']

bpe_tokenizer = ByteLevelBPETokenizer()
bpe_tokenizer.train(
    files=['train.txt'],
    special_tokens=special_tokens
)
bpe_tokenizer.save_model('models/')

gpt2_tokenizer = GPT2Tokenizer.from_pretrained(
    'models/',
    additional_special_tokens=special_tokens,
)
```

When encoding below text, the two tokenizers yield different outputs:

```py
>>> text = 'A <special> token.'
>>> bpe_tokenizer.encode(text).tokens
['A', 'Ġ', '<special>', 'Ġ', 't', 'o', 'k', 'e', 'n', '.']
>>> gpt2_tokenizer.tokenize(text)
['A', '<special>', 't', 'o', 'k', 'e', 'n', '.'] # <----- Note the missing space (`Ġ`) around `<special>`
```

## Expected behavior

I would expect that both tokenizers give the same output when encoding the sentence. Furthermore, because `GPT2Tokenizer` seems to remove the spaces surrounding the special token, the decode(encode()) does not return the original string.

```py
assert bpe_tokenizer.encode(text).tokens == ['A', 'Ġ', '<special>', 'Ġ', 't', 'o', 'k', 'e', 'n', '.']
assert gpt2_tokenizer.tokenize(text) == ['A', 'Ġ', '<special>', 'Ġ', 't', 'o', 'k', 'e', 'n', '.']
```

It is possible that I misunderstand the `GPT2Tokenizer` API. Please advise if I should pass `special_tokens` in a different way. Thank you in advance.


example for passage re-ranking using bert-multilingual-passage-reranking-msmarco

can you give me a working example this

`from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("amberoad/bert-multilingual-passage-reranking-msmarco")

model = AutoModelForSequenceClassification.from_pretrained("amberoad/bert-multilingual-passage-reranking-msmarco")
`


Create README.md

Related to pretrained model cards

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


[EncoderDecoder] google/roberta2roberta_L-24_wikisplit

It seems like `google/roberta2roberta_L-24_wikisplit` should be pre-processed differently than originally thought:

```python
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("google/roberta2roberta_L-24_wikisplit")
model = AutoModelForSeq2SeqLM.from_pretrained("google/roberta2roberta_L-24_wikisplit")

long_sentence = """Due to the hurricane, Lobsterfest has been canceled, making Bob very happy about it and he decides to open Bob 's Burgers for customers who were planning on going to Lobsterfest."""

input_ids = tokenizer(long_sentence, return_tensors="pt").input_ids
output_ids = model.generate(input_ids)[0]
print(tokenizer.decode(output_ids, skip_special_tokens=True))
# should output
# Due Due hurricane, Lobsterfest has been canceled, making Bob very happy about it. He decides to open B
# ob's Burgers for customers who were planning on going to Lobsterfest.com.
```

yields a weird word duplication bug, whereas:

```python
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("google/roberta2roberta_L-24_wikisplit")
model = AutoModelForSeq2SeqLM.from_pretrained("google/roberta2roberta_L-24_wikisplit")

long_sentence = """Due to the hurricane, Lobsterfest has been canceled, making Bob very happy about it and he decides to open Bob 's Burgers for customers who were planning on going to Lobsterfest."""

input_ids = tokenizer(tokenizer.bos_token + long_sentence + tokenizer.eos_token, return_tensors="pt").input_ids
output_ids = model.generate(input_ids)[0]
print(tokenizer.decode(output_ids, skip_special_tokens=True))
# should output
# Due to the hurricane, Lobsterfest has been canceled, making Bob very happy about it. He decides to open Bob's Burgers for customers who were planning on going to Lobsterfest.
```

yields good results.
Notice the difference of
`input_ids = tokenizer(tokenizer.bos_token + long_sentence + tokenizer.eos_token, return_tensors="pt").input_ids` between code sample 1 and 2.

@patrickvonplaten

Wait for author's (@shashiongithub) answer before changing code example.


GPT2Tokenizer.add_tokens() didnt change tokenizer.vocab_size

follow the instruction in add_tokens()

code:

special_tokens = ['<pad>', '<go_r>', '<unk>', '<go_b>', '<go_a>', '<go_u>', ]
num_added_toks = tokenizer.add_tokens(special_tokens)
print('We have added', num_added_toks, 'tokens')

it can successfully encode and decode the newly added tokens, but the GPT2Tokenizer.vocab_size is still 50257.


Bert2bert EncoderDecoderModel from Huggingface is generating a zero tensor for any input

I am using Bert2Bert EncoderDecoderModel from Huggingface for sentence simplification. But my model is generating a zero tensor of the same length regardless of the input. Could someone help me what is wrong in my project. The GitHub link of my project is this https://github.com/Aakash12980/Sentence-Simplification-using-Transformer. Please help me through this.

My decode module is like this:
`
@task.command()
@click.option('--src_test', default="./drive/My Drive/Mini Project/dataset/src_test.txt", help="test source file path")
@click.option('--tgt_test', default="./drive/My Drive/Mini Project/dataset/tgt_test.txt", help="test target file path")
@click.option('--best_model', default="./drive/My Drive/Mini Project/best_model/model.pt", help="best model file path")

def test(**kwargs):

    print("Testing Model module executing...")
    logging.info(f"Test module invoked.")

    src_test = open_file(kwargs['src_test'])
    tgt_test = open_file(kwargs['tgt_test'])
    len_data = len(src_test)

    test_dataset = WikiDataset(src_test, tgt_test)

    test_dl = DataLoader(test_dataset, batch_size=TRAIN_BATCH_SIZE, collate_fn=collate_fn, shuffle=True)

    _,_,_ = load_checkpt(kwargs["best_model"])
    print("Model loaded...")
    model.to(device)
    model.eval()

    test_start_time = time.time()
    epoch_test_loss = evaluate(test_dl, 0)
    epoch_test_loss = epoch_test_loss/len_data
    print(f'avg. test loss: {epoch_test_loss:.5f} | time elapsed: {time.time() - test_start_time}')
    logging.info(f'avg. test loss: {epoch_test_loss:.5f} | time elapsed: {time.time() - test_start_time}')
    print("Test Complete!")
 `
and the output its generates is this:

`input: tensor([[  101,  1188,  1108,  1272, 11310,  1125,  1136,  1151,  2548,  1106,
          2732, 20386,  2692,   117,  2693,  1117, 16975,   117,  1105,  1103,
          8183,  1571,  1105,  8183,  1545,  3784,   118,  8581,  1125,  3335,
          1136,  1678,  2629,   119,   102]])
output: tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0])
input: tensor([[  101,  1109,  4665,  1108,  6497,  1173,  1112,  1126,   169,   169,
          6548, 21718, 24226, 25285,   112,   112,   117,  1112,  1103,  1586,
          1108,   169,   169, 16314,  1105,  4489,  1200,  1190,  1103,  6188,
           117,  4411,  1126,  5340,  1895,  4252, 20473,  4626,  2116,  1104,
          9494,   112,   112,   119,   102]])
output: tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0])
input: tensor([[  101, 13030,  1643,  2883,  1144,  3567,  8593,  9648,  1958,   117,
          1259,  1210,  4748,  1453,  4278,   117,  1160,  1635,  8793,   112,
          1635,  4278,  1105,   170,  6068,  1635,  1509,   119,   102]])
output: tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0])`


[testing] slow tests should be marked as slow

As discussed in https://github.com/huggingface/transformers/issues/7885 this is an effort to make the test suite manageable execution time-wise, as the number of tests is growing and it takes much longer to complete the tests on CI.

* [x] document and set a standard for when a test needs to be marked as `@slow`
* [x] Marks slow the following tests:
- tests/test_tokenization_auto.py::AutoTokenizerTest::test_tokenizer_from_pretrained (23s)
- tests/test_pipelines.py::MonoColumnInputTestCase::test_torch_translation (17s)
- tests/test_modeling_marian.py::TestMarian_en_ROMANCE::test_pipeline (35s)

Fixes: https://github.com/huggingface/transformers/issues/7885

 @LysandreJik, @sgugger, @sshleifer


AdamW

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

Using AdamW of transformers 2.5.1 or 2.6.0, I got the following:

 /home/user/anaconda3/lib/python3.7/site-packages/transformers/optimization.py:155: UserWarning: This overload of add_ is deprecated:
        add_(Number alpha, Tensor other)
Consider using one of the following signatures instead:
        add_(Tensor other, *, Number alpha) (Triggered internally at  /opt/conda/conda-bld/pytorch_1595629403081/work/torch/csrc/utils/python_arg_parser.cpp:766.)
  exp_avg.mul_(beta1).add_(1.0 - beta1, grad)

- `transformers` version:
- Platform: linux
- Python version: 3.7.4
- PyTorch version (GPU?): 1.6.0
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 blenderbot: @mariamabarham
 Bart: @sshleifer
 Marian: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1.
2.
3.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->


pip install transformers by default install 2.5.1

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

https://huggingface.co/transformers/installation.html

Running '''pip install transformers''' by default installs 2.5.1
Could you also specify TensorFlow CPU or GPU version should be installed

- `transformers` version:
- Platform:
- Python version: 3.7.4
- PyTorch version (GPU?): yes
- Tensorflow version (GPU?): 2.2.0
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?:

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 blenderbot: @mariamabarham
 Bart: @sshleifer
 Marian: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1.
2.
3.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->


Issue with XLM-R for multiple-choice questions

Hi there,

I am not able to reasonable numbers with XLM models ("xlm-roberta-base" "xlm-roberta-large") when I test them on [multiple choice questions](https://github.com/huggingface/transformers/tree/master/examples/multiple-choice).

I suspect that it's related to issue #7774.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.3.1
- Platform: Ubuntu
- Python version: 3.7
- PyTorch version (GPU?): 3.6.0 (yes)
- Tensorflow version (GPU?):
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?:

### Who can help
@LysandreJik  @sgugger @VictorSanh

## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

Here is the script you can try,  based on [these instructions](https://github.com/huggingface/transformers/tree/master/examples/multiple-choice):
```
export SWAG_DIR=/path/to/swag_data_dir
python ./examples/multiple-choice/run_multiple_choice.py \
--task_name swag \
--model_name_or_path xlm-roberta-base \
--do_train \
--do_eval \
--data_dir $SWAG_DIR \
--learning_rate 5e-5 \
--num_train_epochs 3 \
--max_seq_length 80 \
--output_dir models_bert/swag_base \
--per_gpu_eval_batch_size=16 \
--per_device_train_batch_size=16 \
--gradient_accumulation_steps 2 \
--overwrite_output
```

It should give you under 30%  -- near random.


[RAG] Propagating of n_docs as parameter to all RagModel's related functions

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #7874

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [X] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [X] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->
@patrickvonplaten


[wip] improved github actions workflow

This is a WIP work on https://github.com/huggingface/transformers/issues/7887


from_pretrained incompatible with the models being downloaded

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.3.0
- Platform: Linux
- Python version: 3.7.7

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 -->
@LysandreJik

## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

```
import logging
logging.basicConfig(level=logging.DEBUG)
from transformers import TFAlbertModel
model = TFAlbertModel.from_pretrained('albert-xxlarge-v2')
```

**Problem:**
The model does not load the weights. I assume due to some incompatibility. The output:
```
INFO:transformers.modeling_tf_utils:Layers of TFAlbertModel not initialized from pretrained model: ['pooler', 'encoder', 'embeddings']
INFO:transformers.modeling_tf_utils:Layers from pretrained model not used in TFAlbertModel: ['albert', 'predictions']
```

I suppose the model from https://s3.amazonaws.com/models.huggingface.co/bert/albert-xxlarge-v2-tf_model.h5 has been changed and is incompatible now. I cannot find the older version of the model anywhere.

## To reproduce

Steps to reproduce the behavior:

(See script above)
1. (Assume) Clean model cache
2. Enable full logging
3. Run TFAlbertModel.from_pretrained('albert-xxlarge-v2')

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

It should load the weights correctly (or at least give an error/warning), not simply fail silently!

<!-- A clear and concise description of what you would expect to happen. -->

---

## Question

Is there a way to access older Albert models (for `transformers==2.3.0`)?
Those from https://s3.amazonaws.com/models.huggingface.co/bert/albert-xxlarge-v2-tf_model.h5 seem to be incompatible.

I really cannot modify the code now or upgrade `transformers` version, but it would totally save me if someone had available the older version of the model. Would it be possible to share (at least all Albert models)?


[tests] fix slow bart cnn test, faster marian tests

+ bart slow cnn test needs to not consider special tokens
+ Marian integration tests run `setupClass`, which downloads a tokenizer, even if they are skipped. By moving that logic to skipped property, we can save 40s of useless tokenizer downloads.

cc @stas00


Github actions: more readable/informative output

Help wanted

Extra attention is needed, help appreciated

Tests

Related to tests

### TLDR
The goal of this Issue is to make the output of https://github.com/huggingface/transformers/runs/1269744369?check_suite_focus=true more readable, without changing the fact that the job failed.

#### Current Workflow:

I randomly remember to check github actions every 3 days, scroll a long way to see what broke, figure out who to assign (sometimes easy/sometimes hard), and then make issues or slack that person. The goal of this PR is to try to make that process easier by attacking the first two steps, but ideas for automating parts of the second two steps are welcome.

#### Goals:
- user can click on github actions and see what broke quickly.
- (Optional) user can see 50 slowest tests
- (HARD/Optional) user can subscribe to slack message/email that pings them when action breaks (Related: https://github.community/t/subscribing-to-actions-notifications/135328)

#### How to attack this PR
I would make a new github action file that runs only on your branch and makes the output/artifacts files you desire. Then, once it is working, apply similar logic to `.github/workflows/self-scheduled.yml`.  Once PR is merged, send follow up PR applying same logic to `.github/workflows/self-push.yml`


Question answering example errors with BrokenPipeError: [Errno 32] Broken pipe

## Environment info
- `transformers` version: 3.3.1
- Platform: Windows-10-10.0.18362-SP0
- Python version: 3.7.1
- PyTorch version (GPU?): 1.4.0+cpu (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: No
### Who can help
No option for question answering in your list

## Information

question-answering pipeline example was used.

The problem arises when using:
the official example scripts

The tasks I am working on is:
question-answering

## To reproduce

Steps to reproduce the behavior:

1. Run
```

from transformers import pipeline

# From https://huggingface.co/transformers/usage.html

nlp = pipeline("question-answering")

context = r"""
Extractive Question Answering is the task of extracting an answer from a text given a question. An example of a
question answering dataset is the SQuAD dataset, which is entirely based on that task. If you would like to fine-tune
a model on a SQuAD task, you may leverage the `run_squad.py`.
"""

print(nlp(question="What is extractive question answering?", context=context))
print(nlp(question="What is a good example of a question answering dataset?", context=context))
```

2. Get error:
```

I1018 21:00:26.369285 15020 filelock.py:274] Lock 2310999501064 acquired on C:\Users\User/.cache\torch\transformers\c2341a51039a311cb3c7dc71b3d21970e6a127876f067f379f8bcd77ef870389.6a09face0659d64f93c9919f323e2ad4543ca9af5d2417b1bfb1a36f2f6b94a4.lock
Downloading: 100%|███████████████████████████████████████████████████████████████████████| 473/473 [00:00<00:00, 118kB/s]
I1018 21:00:27.404981 15020 filelock.py:318] Lock 2310999501064 released on C:\Users\User/.cache\torch\transformers\c2341a51039a311cb3c7dc71b3d21970e6a127876f067f379f8bcd77ef870389.6a09face0659d64f93c9919f323e2ad4543ca9af5d2417b1bfb1a36f2f6b94a4.lock
I1018 21:00:28.506618 15020 filelock.py:274] Lock 2310999482320 acquired on C:\Users\User/.cache\torch\transformers\cee054f6aafe5e2cf816d2228704e326446785f940f5451a5b26033516a4ac3d.e13dbb970cb325137104fb2e5f36fe865f27746c6b526f6352861b1980eb80b1.lock
Downloading: 100%|█████████████████████████████████████████████████████████████████████| 213k/213k [00:00<00:00, 299kB/s]
I1018 21:00:30.234217 15020 filelock.py:318] Lock 2310999482320 released on C:\Users\User/.cache\torch\transformers\cee054f6aafe5e2cf816d2228704e326446785f940f5451a5b26033516a4ac3d.e13dbb970cb325137104fb2e5f36fe865f27746c6b526f6352861b1980eb80b1.lock
I1018 21:00:31.294145 15020 filelock.py:274] Lock 2310999651048 acquired on C:\Users\User/.cache\torch\transformers\414816afc2ab8922d082f893dbf90bcb9a43f09838039249c6c8ca3e8b77921f.455d944f3d1572ab55ed579849f751cf37f303e3388980a42d94f7cd57a4e331.lock
Downloading: 100%|██████████████████████████████████████████████████████████████████████| 230/230 [00:00<00:00, 51.5kB/s]
I1018 21:00:32.335124 15020 filelock.py:318] Lock 2310999651048 released on C:\Users\User/.cache\torch\transformers\414816afc2ab8922d082f893dbf90bcb9a43f09838039249c6c8ca3e8b77921f.455d944f3d1572ab55ed579849f751cf37f303e3388980a42d94f7cd57a4e331.lock
I1018 21:00:33.984491 15020 filelock.py:274] Lock 2310999583152 acquired on C:\Users\User/.cache\torch\transformers\3efcb155a9475fe6b9318b8a8d5278bce1972d30291f97f2a8faeb50d02acabc.087b9fac49619019e540876a2d8ecb497884246b5aa8c9e8b7a0292cfbbe7c52.lock
Downloading: 100%|████████████████████████████████████████████████████████████████████| 261M/261M [00:42<00:00, 6.14MB/s]
I1018 21:01:16.665997 15020 filelock.py:318] Lock 2310999583152 released on C:\Users\User/.cache\torch\transformers\3efcb155a9475fe6b9318b8a8d5278bce1972d30291f97f2a8faeb50d02acabc.087b9fac49619019e540876a2d8ecb497884246b5aa8c9e8b7a0292cfbbe7c52.lock
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "E:\WPy-3710\python-3.7.1.amd64\lib\multiprocessing\spawn.py", line 105, in spawn_main
Traceback (most recent call last):
  File "e:\Work\Python\transformers_question_answering.py", line 11, in <module>
        exitcode = _main(fd)print(nlp(question="What is extractive question answering?", context=context))

  File "E:\WPy-3710\python-3.7.1.amd64\lib\multiprocessing\spawn.py", line 114, in _main
  File "E:\WPy-3710\python-3.7.1.amd64\lib\site-packages\transformers\pipelines.py", line 1691, in __call__
    prepare(preparation_data)
  File "E:\WPy-3710\python-3.7.1.amd64\lib\multiprocessing\spawn.py", line 225, in prepare
    for example in examples
_fixup_main_from_path(data['init_main_from_path'])  File "E:\WPy-3710\python-3.7.1.amd64\lib\site-packages\transformers\pipelines.py", line 1691, in <listcomp>

  File "E:\WPy-3710\python-3.7.1.amd64\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
    run_name="__mp_main__")
  File "E:\WPy-3710\python-3.7.1.amd64\lib\runpy.py", line 263, in run_path
    for example in examples
  File "E:\WPy-3710\python-3.7.1.amd64\lib\site-packages\transformers\data\processors\squad.py", line 345, in squad_convert_examples_to_features
    pkg_name=pkg_name, script_name=fname)
  File "E:\WPy-3710\python-3.7.1.amd64\lib\runpy.py", line 96, in _run_module_code
    with Pool(threads, initializer=squad_convert_example_to_features_init, initargs=(tokenizer,)) as p:
      File "E:\WPy-3710\python-3.7.1.amd64\lib\multiprocessing\context.py", line 119, in Pool
mod_name, mod_spec, pkg_name, script_name)
  File "E:\WPy-3710\python-3.7.1.amd64\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "e:\Work\Python\transformers_question_answering.py", line 11, in <module>
    print(nlp(question="What is extractive question answering?", context=context))
  File "E:\WPy-3710\python-3.7.1.amd64\lib\site-packages\transformers\pipelines.py", line 1691, in __call__
    for example in examples
  File "E:\WPy-3710\python-3.7.1.amd64\lib\site-packages\transformers\pipelines.py", line 1691, in <listcomp>
    for example in examples
  File "E:\WPy-3710\python-3.7.1.amd64\lib\site-packages\transformers\data\processors\squad.py", line 345, in squad_convert_examples_to_features
    with Pool(threads, initializer=squad_convert_example_to_features_init, initargs=(tokenizer,)) as p:
  File "E:\WPy-3710\python-3.7.1.amd64\lib\multiprocessing\context.py", line 119, in Pool
        context=self.get_context())context=self.get_context())

  File "E:\WPy-3710\python-3.7.1.amd64\lib\multiprocessing\pool.py", line 177, in __init__
  File "E:\WPy-3710\python-3.7.1.amd64\lib\multiprocessing\pool.py", line 177, in __init__
        self._repopulate_pool()self._repopulate_pool()

  File "E:\WPy-3710\python-3.7.1.amd64\lib\multiprocessing\pool.py", line 238, in _repopulate_pool
  File "E:\WPy-3710\python-3.7.1.amd64\lib\multiprocessing\pool.py", line 238, in _repopulate_pool
    self._wrap_exception)
      File "E:\WPy-3710\python-3.7.1.amd64\lib\multiprocessing\pool.py", line 257, in _repopulate_pool_static
self._wrap_exception)
  File "E:\WPy-3710\python-3.7.1.amd64\lib\multiprocessing\pool.py", line 257, in _repopulate_pool_static
    w.start()
      File "E:\WPy-3710\python-3.7.1.amd64\lib\multiprocessing\process.py", line 112, in start
w.start()
  File "E:\WPy-3710\python-3.7.1.amd64\lib\multiprocessing\process.py", line 112, in start
        self._popen = self._Popen(self)self._popen = self._Popen(self)

  File "E:\WPy-3710\python-3.7.1.amd64\lib\multiprocessing\context.py", line 322, in _Popen
  File "E:\WPy-3710\python-3.7.1.amd64\lib\multiprocessing\context.py", line 322, in _Popen
    return Popen(process_obj)
  File "E:\WPy-3710\python-3.7.1.amd64\lib\multiprocessing\popen_spawn_win32.py", line 65, in __init__
    return Popen(process_obj)
  File "E:\WPy-3710\python-3.7.1.amd64\lib\multiprocessing\popen_spawn_win32.py", line 33, in __init__
        prep_data = spawn.get_preparation_data(process_obj._name)reduction.dump(process_obj, to_child)

  File "E:\WPy-3710\python-3.7.1.amd64\lib\multiprocessing\spawn.py", line 143, in get_preparation_data
  File "E:\WPy-3710\python-3.7.1.amd64\lib\multiprocessing\reduction.py", line 60, in dump
    _check_not_importing_main()
  File "E:\WPy-3710\python-3.7.1.amd64\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main
    is not going to be frozen to produce an executable.''')
RuntimeError:
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.ForkingPickler(file, protocol).dump(obj)

BrokenPipeError: [Errno 32] Broken pipe
```

## Expected behavior

Output:

```
{'score': 0.622232091629833, 'start': 34, 'end': 96, 'answer': 'the task of extracting an answer from a text given a question.'}
{'score': 0.5115299158662765, 'start': 147, 'end': 161, 'answer': 'SQuAD dataset,'}
```