ModelUtilsTest.test_model_from_pretrained failiing on CUDA

Seems as though an `architecture` key is being added.

Not sure who to assign this to @LysandreJik.
```python
__________________ ModelUtilsTest.test_model_from_pretrained ___________________
[gw0] linux -- Python 3.7.6 /home/hf/actions-runner_transformers/_work/transformers/transformers/.env/bin/python

self = <tests.test_modeling_common.ModelUtilsTest testMethod=test_model_from_pretrained>

    @slow
    def test_model_from_pretrained(self):
        for model_name in BERT_PRETRAINED_MODEL_ARCHIVE_LIST[:1]:
            config = BertConfig.from_pretrained(model_name)
            self.assertIsNotNone(config)
            self.assertIsInstance(config, PretrainedConfig)

            model = BertModel.from_pretrained(model_name)
            model, loading_info = BertModel.from_pretrained(model_name, output_loading_info=True)
            self.assertIsNotNone(model)
            self.assertIsInstance(model, PreTrainedModel)
            for value in loading_info.values():
                self.assertEqual(len(value), 0)

            config = BertConfig.from_pretrained(model_name, output_attentions=True, output_hidden_states=True)
            model = BertModel.from_pretrained(model_name, output_attentions=True, output_hidden_states=True)
            self.assertEqual(model.config.output_hidden_states, True)
>           self.assertEqual(model.config, config)
E           AssertionError: BertConfig {
E             "_name_or_path": "bert-base-uncased",
E             "a[518 chars]22
E           }
E            != BertConfig {
E             "architectures": [
E               "BertForMaskedLM"
E            [478 chars]22
E           }

tests/test_modeling_common.py:1092: AssertionError

```

https://github.com/huggingface/transformers/runs/1303479917?check_suite_focus=true


DataCollatorIntegrationTest::test_nsp failing on GPU

@sgugger I believe you are the right person to tag?

```python
=================================== FAILURES ===================================
_____________________ DataCollatorIntegrationTest.test_nsp _____________________
[gw0] linux -- Python 3.7.6 /home/hf/actions-runner_transformers/_work/transformers/transformers/.env/bin/python

self = <tests.test_data_collator.DataCollatorIntegrationTest testMethod=test_nsp>

    @slow
    def test_nsp(self):
        tokenizer = AutoTokenizer.from_pretrained("bert-base-cased")
        data_collator = DataCollatorForNextSentencePrediction(tokenizer)

        dataset = TextDatasetForNextSentencePrediction(tokenizer, file_path=PATH_SAMPLE_TEXT, block_size=512)
        examples = [dataset[i] for i in range(len(dataset))]
        batch = data_collator(examples)
        self.assertIsInstance(batch, dict)

        # Since there are randomly generated false samples, the total number of samples is not fixed.
        total_samples = batch["input_ids"].shape[0]
        self.assertEqual(batch["input_ids"].shape, torch.Size((total_samples, 512)))
        self.assertEqual(batch["token_type_ids"].shape, torch.Size((total_samples, 512)))
>       self.assertEqual(batch["masked_lm_labels"].shape, torch.Size((total_samples, 512)))
E       KeyError: 'masked_lm_labels'
```

https://github.com/huggingface/transformers/runs/1303479917?check_suite_focus=true


[cleanup] pegasus,marian,mbart pytorch tests

Tests

Related to tests

+ Cleans up 3 pytorch test files
+ Faster (num_beams=2) pegasus integration test.


Commit 121dd43 changes DialoGPT generation behavior

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.3.1
- Platform: Linux-4.4.0-127-generic-x86_64-with-debian-stretch-sid
- Python version: 3.7.3
- PyTorch version (GPU?): 1.6.0+cu101 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: yes (1 TITAN-XP)
- Using distributed or parallel set-up in script?: no

### Who can help
@cccntu @patrickvonplaten @LysandreJik

## Information

Model I am using (Bert, XLNet ...): DialoGPT-large

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Checkout [121dd43](https://github.com/huggingface/transformers/commit/121dd43).

2. Run the DialoGPT "How to use" code given [here](https://huggingface.co/microsoft/DialoGPT-medium), but change `DialoGPT-medium` to `DialoGPT-large`:
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

tokenizer = AutoTokenizer.from_pretrained("microsoft/DialoGPT-large")
model = AutoModelForCausalLM.from_pretrained("microsoft/DialoGPT-large")

# Let's chat for 5 lines
for step in range(5):
    # encode the new user input, add the eos_token and return a tensor in Pytorch
    new_user_input_ids = tokenizer.encode(input(">> User:") + tokenizer.eos_token, return_tensors='pt')

    # append the new user input tokens to the chat history
    bot_input_ids = torch.cat([chat_history_ids, new_user_input_ids], dim=-1) if step > 0 else new_user_input_ids

    # generated a response while limiting the total chat history to 1000 tokens,
    chat_history_ids = model.generate(bot_input_ids, max_length=1000, pad_token_id=tokenizer.eos_token_id)

    # pretty print last ouput tokens from bot
    print("DialoGPT: {}".format(tokenizer.decode(chat_history_ids[:, bot_input_ids.shape[-1]:][0], skip_special_tokens=True)))
```

3. For the user's first utterance, type "Hello, how are you?". I get this output:
```
>> User:Hello, how are you?
DialoGPT: 're you a fan of the show?
```

Note: this problem is still present in the current version of master (`5148f43`).

## Expected behavior

With the previous commit, `0c64b18`, I get this output:
```
>> User:Hello, how are you?
DialoGPT: I'm good, you?
```

## Possible cause
The issue seems to be related to the `<|endoftext|>` token, which is used at the end of every utterance. This is being regarded as a padding token, and thus it's attention-masked, which also seems to affect the position ids.


[fix] FSMT slow test uses lists instead of torch tensors

`test_match_encode_decode` is failing in test_tf mode because it depends on torch. This removes the dependency.

cc @stas00


Getting Hosted inference API working?

model card

Related to pretrained model cards

Trying to get Hosted inference API to work. Was following https://gist.github.com/julien-c/857ba86a6c6a895ecd90e7f7cab48046 ... is below the correct YAML syntax?

pipeline:
-fill-mask
widget:
-text: "København er [mask] i Danmark."

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


BlenderbotSmallTokenizer throws tuple index out of range error for stopword

Using transformers==3.4.0

Script used:
```

from transformers import BlenderbotSmallTokenizer, BlenderbotForConditionalGeneration
mname = 'facebook/blenderbot-90M'
tokenizer = BlenderbotSmallTokenizer.from_pretrained(mname)

sentence = "."
tokenizer(sentence)['input_ids']
```

This throws `IndexError: tuple index out of range`


[BUG] Unexpected overflowing_tokens in tokenizer.encode_plus

### Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.4.0
- Platform: Linux
- Python version: 3.7
- PyTorch version (GPU?): 1.3.1
- Tensorflow version (GPU?):
- Using GPU in script?: True
- Using distributed or parallel set-up in script?:

### Who can help

tokenizers: @mfuntowicz

## Information

When I am using BERT tokenizer, I get unexpected `overflowing_tokens`. Here is a example code to reproduce:

## To reproduce

```
import torch
import transformers
from transformers import AutoTokenizer
import pdb

tokenizer = AutoTokenizer.from_pretrained('bert-base-multilingual-cased')

subtoken_ids_sentence = [x for x in range(1000,1050)]

nr_sentence_parts += 1
encoded_inputs = tokenizer.encode_plus(subtoken_ids_sentence,
                                            max_length=40,
                                            stride=20,
                                            return_overflowing_tokens=True,
                                            truncation=True,
                                            )

print(encoded_inputs['overflowing_tokens'])
```

The output is: `[1029, 1030, 1031, 1032, 1033, 1034, 1035, 1036, 1037, 1038, 1039, 1040, 1041, 1042, 1043, 1044, 1045, 1046, 1047, 1048, 1049, 1048, 1047, 1046, 1045, 1044, 1043, 1042, 1041, 1040, 1039, 1038]`

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior
The expected behavior I want is:
`[1018, 1019, 1020, 1021, 1022, 1023, 1024, 1025, 1026, 1027, 1028, 1029, 1030, 1031, 1032, 1033, 1034, 1035, 1036, 1037, 1038, 1039, 1040, 1041, 1042, 1043, 1044, 1045, 1046, 1047, 1048, 1049]`
The current output contains `[1029, 1030, 1031, 1032, 1033, 1034, 1035, 1036, 1037, 1038, 1039, 1040, 1041, 1042, 1043, 1044, 1045, 1046, 1047, 1048, 1049]` and an additional reversed Tensor of `[1048, 1047, 1046, 1045, 1044, 1043, 1042, 1041, 1040, 1039, 1038]`, which I think is wrong.

When I dig into the code, I find that:
https://github.com/huggingface/transformers/blob/6b4c617666fd26646d44d54f0c45dfe1332b12ca/src/transformers/tokenization_utils_base.py#L2556-L2564

I wonder why there is a for loop in it and I think I need  `truncation_strategy = TruncationStrategy.ONLY_FIRST`. However, I failed to turn the truncation_stractegy to `only_first` because the code here turn the truncation strategy to `longest_first`.

https://github.com/huggingface/transformers/blob/6b4c617666fd26646d44d54f0c45dfe1332b12ca/src/transformers/tokenization_utils_base.py#L1750-L1759

Can you give me any help?
<!-- A clear and concise description of what you would expect to happen. -->


(Load dataset failure) ConnectionError: Couldn’t reach https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/cnn_dailymail/cnn_dailymail.py

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers
     -->

## Details
<!-- Description of your issue -->
Hey, I want to load the cnn-dailymail dataset for fine-tune.
I write the code like this
from datasets import load_dataset

test_dataset = load_dataset(“cnn_dailymail”, “3.0.0”, split=“train”)

And I got the following errors.

Traceback (most recent call last):
File “test.py”, line 7, in
test_dataset = load_dataset(“cnn_dailymail”, “3.0.0”, split=“test”)
File “C:\Users\666666\AppData\Local\Programs\Python\Python38\lib\site-packages\datasets\load.py”, line 589, in load_dataset
module_path, hash = prepare_module(
File “C:\Users\666666\AppData\Local\Programs\Python\Python38\lib\site-packages\datasets\load.py”, line 268, in prepare_module
local_path = cached_path(file_path, download_config=download_config)
File “C:\Users\666666\AppData\Local\Programs\Python\Python38\lib\site-packages\datasets\utils\file_utils.py”, line 300, in cached_path
output_path = get_from_cache(
File “C:\Users\666666\AppData\Local\Programs\Python\Python38\lib\site-packages\datasets\utils\file_utils.py”, line 475, in get_from_cache
raise ConnectionError(“Couldn’t reach {}”.format(url))
ConnectionError: Couldn’t reach https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/cnn_dailymail/cnn_dailymail.py

How can I fix this ?
<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:


[Model Card] new cross lingual sentence model for German and English

Related to pretrained model cards

- add new model card
- adapted model cards of our other sentence embeddings


Create README.md

Related to pretrained model cards

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


T5 on multiple tasks

Dear Huggingface team,
I am looking for a code to pretrain T5 on multiple tasks, the best I could find is the code released with wrapper around huggingface T5 model in the original author's repo:
https://github.com/google-research/text-to-text-transfer-transformer/blob/master/t5/models/hf_model.py

On line 308-310, they create tensorflow datasets I think, and pass it to huggingface model, I was wondering if I could ask for your help, if one wants to add data parallelism to this code to make it efficient in pytorch, could you help me how I can do it in pytorch? thanks a lot, and I appreciate your help.


Tiny TF Bart fixes

Tiny TF Bart fixes


[test] tests/test_modeling_deberta.py breaks on pytorch-nightly

pytorch-nightly and pytorch-1.7-candidate break these:
```
FAILED tests/test_modeling_deberta.py::DebertaModelTest::test_deberta_model - RuntimeError: diff_view_meta->output_nr_ == 0 INTERNAL ASSERT FAILED...
FAILED tests/test_modeling_deberta.py::DebertaModelTest::test_feed_forward_chunking - RuntimeError: diff_view_meta->output_nr_ == 0 INTERNAL ASSER...
FAILED tests/test_modeling_deberta.py::DebertaModelTest::test_for_sequence_classification - RuntimeError: diff_view_meta->output_nr_ == 0 INTERNAL...
FAILED tests/test_modeling_deberta.py::DebertaModelTest::test_resize_tokens_embeddings - RuntimeError: diff_view_meta->output_nr_ == 0 INTERNAL AS...
```

looks like a bug in pytorch, right?

```
RuntimeError: diff_view_meta->output_nr_ == 0 INTERNAL ASSERT FAILED at "/opt/conda/conda- \
bld/pytorch_1603436966316/work/torch/csrc/autograd/variable.cpp":363, please report a bug to PyTorch.
```

log:
```
================================================================ test session starts ================================================================
platform linux -- Python 3.8.5, pytest-6.1.1, py-1.9.0, pluggy-0.13.1 -- /home/stas/anaconda3/envs/main-38/bin/python
cachedir: .pytest_cache
rootdir: /mnt/nvme1/code/huggingface/transformers-master
plugins: typeguard-2.10.0, forked-1.3.0, xdist-2.1.0, instafail-0.4.2
collecting ... 2020-10-24 09:21:07.169605: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
collected 1 item

tests/test_modeling_deberta.py::DebertaModelTest::test_deberta_model FAILED
________________________________________________________ DebertaModelTest.test_deberta_model ________________________________________________________

self = <tests.test_modeling_deberta.DebertaModelTest testMethod=test_deberta_model>

    def test_deberta_model(self):
        config_and_inputs = self.model_tester.prepare_config_and_inputs()
>       self.model_tester.create_and_check_deberta_model(*config_and_inputs)

tests/test_modeling_deberta.py:210:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/test_modeling_deberta.py:159: in create_and_check_deberta_model
    sequence_output = model(input_ids, attention_mask=input_mask, token_type_ids=token_type_ids)[0]
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/modules/module.py:744: in _call_impl
    result = self.forward(*input, **kwargs)
src/transformers/modeling_deberta.py:891: in forward
    encoder_outputs = self.encoder(
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/modules/module.py:744: in _call_impl
    result = self.forward(*input, **kwargs)
src/transformers/modeling_deberta.py:401: in forward
    hidden_states = layer_module(
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/modules/module.py:744: in _call_impl
    result = self.forward(*input, **kwargs)
src/transformers/modeling_deberta.py:324: in forward
    attention_output = self.attention(
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/modules/module.py:744: in _call_impl
    result = self.forward(*input, **kwargs)
src/transformers/modeling_deberta.py:257: in forward
    self_output = self.self(
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/modules/module.py:744: in _call_impl
    result = self.forward(*input, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = DisentangledSelfAttention(
  (in_proj): Linear(in_features=32, out_features=96, bias=False)
  (dropout): StableDropout()
)
hidden_states = tensor([[[-1.5951, -1.0046,  0.5641,  ..., -0.4472,  0.0159, -0.1435],
         [-1.3476, -0.1559, -1.3866,  ...,  1.2...        [ 0.5269, -0.0601, -0.4018,  ..., -0.1616,  0.5335, -0.8894]]],
       device='cuda:0', grad_fn=<MulBackward0>)
attention_mask = tensor([[[[1, 1, 1, 1, 0, 0, 0],
          [1, 1, 1, 1, 0, 0, 0],
          [1, 1, 1, 1, 0, 0, 0],
          [1, 1, 1,..., 1, 1, 1, 1],
          [0, 0, 0, 1, 1, 1, 1],
          [0, 0, 0, 1, 1, 1, 1]]]], device='cuda:0', dtype=torch.uint8)
return_att = False, query_states = None, relative_pos = None, rel_embeddings = None

    def forward(
        self,
        hidden_states,
        attention_mask,
        return_att=False,
        query_states=None,
        relative_pos=None,
        rel_embeddings=None,
    ):
        """Call the module

        Args:
            hidden_states (:obj:`torch.FloatTensor`):
                Input states to the module usally the output from previous layer, it will be the Q,K and V in `Attention(Q,K,V)`

            attention_mask (:obj:`torch.ByteTensor`):
                An attention mask matrix of shape [`B`, `N`, `N`] where `B` is the batch size, `N` is the maxium sequence length in which element [i,j] = `1` means the `i` th token in the input can attend to the `j` th token.

            return_att (:obj:`bool`, optional):
                Whether return the attention maxitrix.

            query_states (:obj:`torch.FloatTensor`, optional):
                The `Q` state in `Attention(Q,K,V)`.

            relative_pos (:obj:`torch.LongTensor`):
                The relative position encoding between the tokens in the sequence. It's of shape [`B`, `N`, `N`] with values ranging in [`-max_relative_positions`, `max_relative_positions`].

            rel_embeddings (:obj:`torch.FloatTensor`):
                The embedding of relative distances. It's a tensor of shape [:math:`2 \\times \\text{max_relative_positions}`, `hidden_size`].

        """
        if query_states is None:
            qp = self.in_proj(hidden_states)  # .split(self.all_head_size, dim=-1)
            query_layer, key_layer, value_layer = self.transpose_for_scores(qp).chunk(3, dim=-1)
        else:

            def linear(w, b, x):
                if b is not None:
                    return torch.matmul(x, w.t()) + b.t()
                else:
                    return torch.matmul(x, w.t())  # + b.t()

            ws = self.in_proj.weight.chunk(self.num_attention_heads * 3, dim=0)
            qkvw = [torch.cat([ws[i * 3 + k] for i in range(self.num_attention_heads)], dim=0) for k in range(3)]
            qkvb = [None] * 3

            q = linear(qkvw[0], qkvb[0], query_states)
            k, v = [linear(qkvw[i], qkvb[i], hidden_states) for i in range(1, 3)]
            query_layer, key_layer, value_layer = [self.transpose_for_scores(x) for x in [q, k, v]]

        query_layer += self.transpose_for_scores(self.q_bias[None, None, :])
>       value_layer += self.transpose_for_scores(self.v_bias[None, None, :])
E       RuntimeError: diff_view_meta->output_nr_ == 0 INTERNAL ASSERT FAILED at "/opt/conda/conda-bld/pytorch_1603436966316/work/torch/csrc/autograd/variable.cpp":363, please report a bug to PyTorch.

src/transformers/modeling_deberta.py:575: RuntimeError

============================================================== short test summary info ==============================================================
FAILED tests/test_modeling_deberta.py::DebertaModelTest::test_deberta_model - RuntimeError: diff_view_meta->output_nr_ == 0 INTERNAL ASSERT FAILED...
=========================================================== 1 failed, 5 warnings in 4.22s ===========================================================
```

## Environment info
```
- `transformers` version: 3.4.0
- Platform: Linux-5.4.0-52-generic-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyTorch version (GPU?): 1.8.0.dev20201023 (True)
- Tensorflow version (GPU?): 2.3.1 (True)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
```

for pytorch-1.7 candidate I used:

https://download.pytorch.org/whl/test/cu102/torch-1.7.0-cp38-cp38-linux_x86_64.whl

----------------------

**update**: The release team https://github.com/pytorch/pytorch/issues/45592 has been notified of this issue on pytorch slack. Waiting to hear back from them.


[bart] SinusoidalPositionalEmbedding breaks under pytorch-nightly

pytorch-nightly breaks these:
```
FAILED tests/test_modeling_bart.py::TestSinusoidalPositionalEmbeddings::test_odd_embed_dim - RuntimeError: a view of a leaf Variable that requires...
FAILED tests/test_modeling_bart.py::TestSinusoidalPositionalEmbeddings::test_positional_emb_cache_logic - RuntimeError: a view of a leaf Variable ...
FAILED tests/test_modeling_bart.py::TestSinusoidalPositionalEmbeddings::test_positional_emb_weights_against_marian - RuntimeError: a view of a lea...
F
```

```
================================================================ test session starts ================================================================
platform linux -- Python 3.8.5, pytest-6.1.1, py-1.9.0, pluggy-0.13.1 -- /home/stas/anaconda3/envs/main-38/bin/python
cachedir: .pytest_cache
rootdir: /mnt/nvme1/code/huggingface/transformers-master
plugins: typeguard-2.10.0, forked-1.3.0, xdist-2.1.0, instafail-0.4.2
collecting ... 2020-10-24 09:14:35.276431: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
collected 1 item

tests/test_modeling_bart.py::TestSinusoidalPositionalEmbeddings::test_odd_embed_dim FAILED
_______________________________________________ TestSinusoidalPositionalEmbeddings.test_odd_embed_dim _______________________________________________

self = <tests.test_modeling_bart.TestSinusoidalPositionalEmbeddings testMethod=test_odd_embed_dim>

    def test_odd_embed_dim(self):
        with self.assertRaises(NotImplementedError):
            SinusoidalPositionalEmbedding(num_positions=4, embedding_dim=5, padding_idx=0).to(torch_device)

        # odd num_positions is allowed
>       SinusoidalPositionalEmbedding(num_positions=5, embedding_dim=4, padding_idx=0).to(torch_device)

tests/test_modeling_bart.py:627:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
src/transformers/modeling_bart.py:1331: in __init__
    self.weight = self._init_weight(self.weight)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

out = Parameter containing:
tensor([[ 0.1368,  2.6925,  1.3918,  0.7332],
        [-1.2813, -0.3071,  1.0553, -0.4325],
    ...6445],
        [-1.6619, -0.2872,  0.6869,  0.6489],
        [-1.5226,  0.1161, -0.2026,  0.1853]], requires_grad=True)

    @staticmethod
    def _init_weight(out: nn.Parameter):
        """Identical to the XLM create_sinusoidal_embeddings except features are not interleaved.
        The cos features are in the 2nd half of the vector. [dim // 2:]
        """
        n_pos, dim = out.shape
        position_enc = np.array(
            [[pos / np.power(10000, 2 * (j // 2) / dim) for j in range(dim)] for pos in range(n_pos)]
        )
>       out[:, 0 : dim // 2] = torch.FloatTensor(np.sin(position_enc[:, 0::2]))  # This line breaks for odd n_pos
E       RuntimeError: a view of a leaf Variable that requires grad is being used in an in-place operation.

src/transformers/modeling_bart.py:1342: RuntimeError

============================================================== short test summary info ==============================================================
FAILED tests/test_modeling_bart.py::TestSinusoidalPositionalEmbeddings::test_odd_embed_dim - RuntimeError: a view of a leaf Variable that requires...
=========================================================== 1 failed, 3 warnings in 3.01s ===========================================================

```

## Environment info
```
- `transformers` version: 3.4.0
- Platform: Linux-5.4.0-52-generic-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyTorch version (GPU?): 1.8.0.dev20201023 (True)
- Tensorflow version (GPU?): 2.3.1 (True)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
```


sentencepiece 0.1.94 causing segmentation fault

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.4.0 and 3.3.1
- Platform: Linux/Sagemaker
- Python version: 3.7
- PyTorch version (GPU?): 1.4
- Tensorflow version (GPU?):
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help

## Information

## To reproduce

Steps to reproduce the behavior:

1. `pip install transformers[torch]`
2. `from transformers.trainer import TrainingArguments, Trainer`
`import torch`
3. `torch.tensor([1,2,3])`

`transformers` 3.3.1 seg faults at step 3, `transformers` 3.4 seg faults at step 2.

## Expected behavior

No segmentation fault


Colab can't import trim_batch for T5, anything changed in transformers.tokenization_utils?

from transformers.tokenization_utils import trim_batch

ImportError: cannot import name 'trim_batch'

Any solutions? Thanks a lot.


tutorial document

In the Translation section in https://huggingface.co/transformers/task_summary.html
"
Here is an example of doing translation using a model and a tokenizer. The process is the following:
1.Instantiate a tokenizer and a model from the checkpoint name. Summarization is usually done using an encoder-decoder model, such as Bart or T5.
2.Define the article that should be summarizaed.
3.Add the T5 specific prefix “translate English to German: “
4.Use the PreTrainedModel.generate() method to perform the translation.
"
1 and 2 seemed copies from Summarization section and not be modified accordingly.


Create README.md

Related to pretrained model cards

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


Mlflow integration callback

# What does this PR do?

This PR adds Trainer integration with [MLflow](https://mlflow.org/).

It is implemented in roughly the same way as other integration callbacks (CometML, wandb) and gets added to the list of Trainer callbacks automatically when mlflow is installed. All the mlflow parameters are configured with env variables, as described in the library documentation. This PR adds an additional environment variable, `HF_MLFLOW_LOG_ARTIFACTS`, which controls whether to use mlflow artifact logging facility to save artifacts generated after training (it doesn't make much sense if mlflow is used locally).

Fixes #7698

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

@sgugger


Create README.md

Related to pretrained model cards


weird output shape when fine-tuning TFDistilBertForSequenceClassification

I'm trying to fine-tune `TFDistilBertForSequenceClassification` for multi-class classification (100 classes) on a custom dataset following the tutorial at https://huggingface.co/transformers/custom_datasets.html.

I'm following the workflow for fine-tuning in native tensorflow, i.e.:

```
from transformers import TFDistilBertForSequenceClassification
model = TFDistilBertForSequenceClassification.from_pretrained('distilbert-base-uncased')
optimizer = tf.keras.optimizers.Adam(learning_rate=5e-5)
model.compile(optimizer=optimizer, loss=model.compute_loss) # can also use any keras loss fn
model.fit(train_dataset.shuffle(1000).batch(16), epochs=3, batch_size=16)
```

Everything seems to go fine during fine-tuning, but when I try to predict on the test dataset (2000 samples) using `model.predict(test_dataset)`, I get an output with weird shape.

That is, instead of getting an output of shape (1, 2000, 100), I get one with shape (1, 1024000, 100), where 1024000 happens to be number of test examples (2000) times the sequence length (512).

Any hint on what's going on here? Sorry if it's a naïve mistake on my side, I'm new to tf.


[doc prepare_seq2seq_batch] fix docs

The `prepare_seq2seq_batch` batch returns `[input_ids, attention_mask, labels]` and not `[input_ids, attention_mask, decoder_input_ids]`. This PR fixes the docs accordingly.

@sshleifer


Add model_cards for DynaBERT

Related to pretrained model cards

Add model_cards for DynaBERT_MNLI and DynaBERT_SST-2.


Problem and solution:

This happened several times as of recent. Some env gets messed up and I end up with most tests failing with:
```____________________________________________________ ERROR collecting tests/test_benchmark_tf.py ____________________________________________________
tests/test_benchmark_tf.py:6: in <module>
    from transformers import AutoConfig, is_tf_available
src/transformers/__init__.py:22: in <module>
    from .integrations import (  # isort:skip
src/transformers/integrations.py:58: in <module>
    from .file_utils import is_torch_tpu_available
src/transformers/file_utils.py:59: in <module>
    import tensorflow as tf
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/tensorflow/__init__.py:41: in <module>
    from tensorflow.python.tools import module_util as _module_util
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/tensorflow/python/__init__.py:84: in <module>
    from tensorflow.python import keras
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/tensorflow/python/keras/__init__.py:27: in <module>
    from tensorflow.python.keras import models
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/tensorflow/python/keras/models.py:24: in <module>
    from tensorflow.python.keras import metrics as metrics_module
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/tensorflow/python/keras/metrics.py:37: in <module>
    from tensorflow.python.keras.engine import base_layer
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py:51: in <module>
    from tensorflow.python.keras import initializers
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/tensorflow/python/keras/initializers/__init__.py:127: in <module>
    populate_deserializable_objects()
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/tensorflow/python/keras/initializers/__init__.py:85: in populate_deserializable_objects
    generic_utils.populate_dict_with_module_objects(
E   AttributeError: module 'tensorflow.python.keras.utils.generic_utils' has no attribute 'populate_dict_with_module_objects'
collected 0 items / 1 error
```

I think it's conda installing a broken tensorflow, I'm not 100% sure - see the solution in the next comment.


src->transformers->generation_tf_util.py ->_generate_beam_search->outputs = self(**model_inputs)    why self ?There is not a function?

Traceback (most recent call last):
  File "D:\Program Files\JetBrains\PyCharm 2018.3.2\helpers\pydev\pydevd.py", line 1741, in <module>
    main()
  File "D:\Program Files\JetBrains\PyCharm 2018.3.2\helpers\pydev\pydevd.py", line 1735, in main
    globals = debugger.run(setup['file'], None, None, is_module)
  File "D:\Program Files\JetBrains\PyCharm 2018.3.2\helpers\pydev\pydevd.py", line 1135, in run
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "D:\Program Files\JetBrains\PyCharm 2018.3.2\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "D:/Github_project/transformers-master/src/transformers/generation_tf_utils.py", line 1128, in <module>
    use_cache = None,
  File "D:/Github_project/transformers-master/src/transformers/generation_tf_utils.py", line 625, in _generate_beam_search
    outputs = self(**model_inputs)  # (batch_size * num_beams, cur_len, vocab_size)
TypeError: 'TFGenerationMixin' object is not callable


Doc styling

# What does this PR do?

Add a script that does some styling on the doc files and docstrings.


TextDataset bug with big files

- `transformers` version: 3.0.2
- Platform: Linux-4.4.0-137-generic-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyTorch version (GPU?): 1.6.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help

@mfuntowicz

## Information

The model I am using (Bert, XLNet ...): XLMRobertaTokenizer

The problem arises when using:
* [ x ] my own modified script: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Run the following code
 ```Python
from transformers import XLMRobertaTokenizer, TextDataset
max_pos = 4096
tokenizer = XLMRobertaTokenizer.from_pretrained("xlm-roberta-base", model_max_length=max_pos)
tokenizer.model_max_length = max_pos
tokenizer.init_kwargs["model_max_length"] = max_pos
train_datapath = "path/to/train.raw"
train_dataset = TextDataset(
    tokenizer=tokenizer,
    file_path=train_datapath,
    block_size=tokenizer.max_len
)
```

## Error Messages

```
terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
Aborted
```

## Expected behavior

My file `train.raw` contains 349912098 words with 2,3G. When I try with a small dataset (262703214 words and 251M), it works fine. I have modified this line in the TextDataset class https://github.com/huggingface/transformers/blob/a16e568f22a4d07813ba76343309ec20096115a5/src/transformers/data/datasets/language_modeling.py#L68 to understand where is the problem. It happens in the `tokenizer.tokenize(text)` part. I have changed it not to tokenize the entire text directly but to process chunks of the text each time, concatenating the result in a final list. Although memory hungry, this method works fine (and memory is not my problem).

**Note:** I have executed this script in a machine with 504G of RAM, and the script used approx. 36G when it died.


Ci test tf super slow

Enable the slow TF suite on GPU. Below are the current failing tests.

Investigating whether they're actually failing or if it's for some other reason.

- [x] ~FAILED tests/test_modeling_marian.py::ModelManagementTests::test_model_names~
- [x] ~FAILED tests/test_modeling_prophetnet.py::ProphetNetModelIntegrationTest::test_cnndm_inference~
- [x] ~FAILED tests/test_modeling_prophetnet.py::ProphetNetModelIntegrationTest::test_pretrained_checkpoint_hidden_states~
- [x] ~FAILED tests/test_modeling_prophetnet.py::ProphetNetModelIntegrationTest::test_question_gen_inference~
- [x] ~FAILED tests/test_modeling_roberta.py::RobertaModelIntegrationTest::test_inference_classification_head~
- [x] ~FAILED tests/test_modeling_roberta.py::RobertaModelIntegrationTest::test_inference_masked_lm~
- [x] ~FAILED tests/test_modeling_roberta.py::RobertaModelIntegrationTest::test_inference_no_head~
- [x] ~FAILED tests/test_modeling_squeezebert.py::SqueezeBertModelIntegrationTest::test_inference_classification_head~
- [x] ~FAILED tests/test_modeling_tf_albert.py::TFAlbertModelTest::test_saved_model_with_attentions_output~
- [x] ~FAILED tests/test_modeling_tf_albert.py::TFAlbertModelTest::test_saved_model_with_hidden_states_output~
- [x] ~FAILED tests/test_modeling_tf_camembert.py::TFCamembertModelIntegrationTest::test_output_embeds_base_model~
- [x] ~FAILED tests/test_modeling_tf_electra.py::TFElectraModelTest::test_saved_model_with_attentions_output~
- [x] ~FAILED tests/test_modeling_tf_electra.py::TFElectraModelTest::test_saved_model_with_hidden_states_output~
- [x] ~FAILED tests/test_modeling_tf_flaubert.py::TFFlaubertModelTest::test_saved_model_with_attentions_output~
- [x] ~FAILED tests/test_modeling_tf_flaubert.py::TFFlaubertModelTest::test_saved_model_with_hidden_states_output~
- [x] ~FAILED tests/test_modeling_tf_funnel.py::TFFunnelModelTest::test_saved_model_with_attentions_output~
- [x] ~FAILED tests/test_modeling_tf_funnel.py::TFFunnelModelTest::test_saved_model_with_hidden_states_output~
- [x] ~FAILED tests/test_modeling_tf_funnel.py::TFFunnelBaseModelTest::test_saved_model_with_attentions_output~
- [x] ~FAILED tests/test_modeling_tf_funnel.py::TFFunnelBaseModelTest::test_saved_model_with_hidden_states_output~
- [x] FAILED tests/test_modeling_tf_longformer.py::TFLongformerModelTest::test_saved_model_with_attentions_output
- [x] ~FAILED tests/test_modeling_tf_lxmert.py::TFLxmertModelTest::test_saved_model_with_attentions_output~
- [x] ~FAILED tests/test_modeling_tf_lxmert.py::TFLxmertModelTest::test_saved_model_with_hidden_states_output~
- [x] ~FAILED tests/test_modeling_tf_mobilebert.py::TFMobileBertModelTest::test_model_from_pretrained~
- [x] FAILED tests/test_modeling_tf_t5.py::TFT5ModelTest::test_saved_model_with_attentions_output
- [x] FAILED tests/test_modeling_tf_t5.py::TFT5ModelTest::test_saved_model_with_hidden_states_output
- [x] ~FAILED tests/test_modeling_tf_xlm_roberta.py::TFFlaubertModelIntegrationTest::test_output_embeds_base_model~
- [x] ~FAILED tests/test_modeling_tf_xlnet.py::TFXLNetModelLanguageGenerationTest::test_lm_generate_xlnet_base_cased~
- [x] ~FAILED tests/test_modeling_transfo_xl.py::TransfoXLModelLanguageGenerationTest::test_lm_generate_transfo_xl_wt103~
- [x] ~FAILED tests/test_modeling_xlm_prophetnet.py::XLMProphetNetModelIntegrationTest::test_ntg_hidden_states~
- [x] ~FAILED tests/test_modeling_xlm_prophetnet.py::XLMProphetNetModelIntegrationTest::test_pretrained_checkpoint_hidden_states~
- [x] ~FAILED tests/test_modeling_xlm_prophetnet.py::XLMProphetNetModelIntegrationTest::test_xprophetnet_ntg_inference~
- [x] ~FAILED tests/test_modeling_xlm_roberta.py::XLMRobertaModelIntegrationTest::test_xlm_roberta_base~
- [x] ~FAILED tests/test_modeling_xlm_roberta.py::XLMRobertaModelIntegrationTest::test_xlm_roberta_large~
- [x] FAILED tests/test_pipelines.py::PipelineCommonTests::test_tf_defaults - Value...
- [x] ~FAILED tests/test_tokenization_fsmt.py::FSMTTokenizationTest::test_match_encode_decode~

And while we're at it here are the remaining failing tests in PyTorch slow multi-gpu tests:

- [x] FAILED tests/test_data_collator.py::DataCollatorIntegrationTest::test_nsp - K...
- [x] FAILED tests/test_modeling_common.py::ModelUtilsTest::test_model_from_pretrained
- [x] ~FAILED tests/test_modeling_rag.py::RagModelIntegrationTests::test_rag_sequence_generate_batch~
- [x] ~FAILED tests/test_modeling_rag.py::RagModelIntegrationTests::test_rag_sequence_generate_beam~
- [x] ~FAILED tests/test_modeling_rag.py::RagModelIntegrationTests::test_rag_token_generate_batch~
- [x] ~FAILED tests/test_modeling_rag.py::RagModelIntegrationTests::test_rag_token_generate_beam~
- [x] ~FAILED tests/test_modeling_rag.py::RagModelIntegrationTests::test_rag_token_inference~
- [x] FAILED tests/test_pipelines.py::MonoColumnInputTestCase::test_integration_torch_conversation
- [x] FAILED tests/test_pipelines.py::MonoColumnInputTestCase::test_integration_torch_conversation_truncated_history
- [x] FAILED tests/test_pipelines.py::DialoguePipelineTests::test_torch_conversation
- [x] FAILED tests/test_pipelines.py::PipelineCommonTests::test_pt_defaults - Value...

The RAG integration tests seem to happen because of OOM errors, deactivated them in a multi-gpu setup as the GPUs have less memory.

Status:

Done! Waiting for the green tests to merge.


[tokenizers] Fixing #8001 - Adding tests on tokenizers serialization

# What does this PR do?

Fixes #8001

Now the tokenizers classes have to send all the keyword arguments of the `__init__` up to the base class of the tokenizer (by `super().__init__`) were they are stored in `init_kwargs` for serialized saving/reloading with `save_pretrained/from_pretrained`.

Adding a test on tokenizers serialization that all the keyword arguments of the `__init__` are found in the saved `init_kwargs` to avoid forgetting to send some arguments up in future (and current) tokenizers.

Make T5 tokenizer serialization more robust.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


Differences between facebook/bart-base and facebook/bart-large?

# ❓ Questions & Help

Is there some more difference between `facebook/bart-base` and `facebook/bart-large` (other than dimensions, heads and layers)?

## Who can help

@sshleifer @WiseDoge

## Environment info

- transformers version: 3.3.1
- Python version: 3.6.12
- PyTorch version (GPU?): 1.4.0 GPU-version

## Command:

I'm using the seq2seq/finetune.py script to finetune both BARTs.

```
python finetune.py \
--data_dir=${DATA_DIR} \
--learning_rate=3e-5 \
--num_train_epochs 5 \
--task summarization \
--model_name_or_path=${MODEL} \
--train_batch_size=4 \
--eval_batch_size=4 \
--gpus 1 \
--output_dir=$OUTPUT_DIR \
--max_source_length=256 \
--max_target_length=256 \
--val_max_target_length=256 \
--test_max_target_length=256 \
--eval_max_gen_length=256 \
--do_train --do_predict \
--eval_beams 5
```

${MODEL} model can be `facebook/bart-base` or `facebook/bart-large`

## Details

When I finetune facebook/bart-base, it works well:

```
"input_ids": " <s> ( report :ARG1 ( station :ARG1 ( troop :mod ( country :wiki Russia :name ( name :op1 Russia ) ) :ARG0-of ( withdraw :ARG2 ( country :quant 3 :location ( sea :wiki Baltic_Sea :name ( name :op1 Baltic :op2 Sea ) ) ) ) ) :ARG2 ( and :op1 ( state :wiki - :name ( name :op1 Jalininggele ) :location country ) :op2 ( state :wiki - :name ( name :op1 Simolingsike ) ) :op3 ( city :wiki - :name ( name :op1 Yelinia ) :location ( relative-position :op1 ( city :wiki Moscow :name ( name :op1 Moscow ) ) :quant ( distance-quantity :quant 300 :unit ( kilometer ) ) ) ) ) :mod ( respective ) ) )</s><pad><pad><pad>",
        "labels": "<s> It is reported that the Russian troops that withdrew from the three Baltic Sea countries will be stationed respectively in the Russian state of Jalininggele, the state of Simolingsike and Yelinia city which is 300 kilometers away from Moscow.</s>",
        "decoder_input_ids": "</s><s> It is reported that the Russian troops that withdrew from the three Baltic Sea countries will be stationed respectively in the Russian state of Jalininggele, the state of Simolingsike and Yelinia city which is 300 kilometers away from Moscow.",
        "generated_ids": "</s><s> Russian troops reported to be stationed in the 3 Baltic Sea countries of Jalininggele, Simolingsike and Yelinia 300 kilometers (110 miles) from Moscow.</s><pad><pad><pad><pad><pad><pad><pad>"
```

When I finetune facebook/bart-large, it did not generate a reasonable output:

```
"input_ids": "<s> ( report :ARG1 ( station :ARG1 ( troop :mod ( country :wiki Russia :name ( name :op1 Russia ) ) :ARG0-of ( withdraw :ARG2 ( country :quant 3 :location ( sea :wiki Baltic_Sea :name ( name :op1 Baltic :op2 Sea ) ) ) ) ) :ARG2 ( and :op1 ( state :wiki - :name ( name :op1 Jalininggele ) :location country ) :op2 ( state :wiki - :name ( name :op1 Simolingsike ) ) :op3 ( city :wiki - :name ( name :op1 Yelinia ) :location ( relative-position :op1 ( city :wiki Moscow :name ( name :op1 Moscow ) ) :quant ( distance-quantity :quant 300 :unit ( kilometer ) ) ) ) ) :mod ( respective ) ) )</s><pad><pad><pad>",
        "labels": "<s> It is reported that the Russian troops that withdrew from the three Baltic Sea countries will be stationed respectively in the Russian state of Jalininggele, the state of Simolingsike and Yelinia city which is 300 kilometers away from Moscow.</s>",
        "decoder_input_ids": "</s><s> It is reported that the Russian troops that withdrew from the three Baltic Sea countries will be stationed respectively in the Russian state of Jalininggele, the state of Simolingsike and Yelinia city which is 300 kilometers away from Moscow.",
        "generated_ids": "</s><s><s><s><s><s><s><s><s><s><s> ... <s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s></s>"
```

I'm using the same code, but only `facebook/bart-base` model works. In a previous transformer version, both worked, but not in this one (3.3.1).


german medbert readme

Related to pretrained model cards

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


Create model card for bert-italian-cased-finetuned-pos

Related to pretrained model cards


do_lower_case not saved/loaded correctly for Tokenizers

## Environment info
- `transformers` version: 3.4.0
- Platform: Linux-5.4.0-52-generic-x86_64-with-debian-buster-sid
- Python version: 3.7.6
- PyTorch version (GPU?): 1.5.1 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
@mfuntowicz

## Information

The `do_lower_case` property of BertTokenizer is not correctly restored after saving / loading.

## To reproduce

```python
from transformers import BertTokenizer

tokenizer = BertTokenizer.from_pretrained("bert-base-cased")
print(tokenizer.do_lower_case)

tokenizer.save_pretrained("debug_tokenizer")
tokenizer_loaded = BertTokenizer.from_pretrained("debug_tokenizer")
print(tokenizer_loaded.do_lower_case)

```
returns
```
False
True
```
## Expected behavior
Same object attributes after saving / loading


How to load tokenizer for models without vocab.txt?

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers
     -->

## Details
<!-- Description of your issue -->
I want to use xlm-roberta-large model, but "https://huggingface.co/"just give the file named "xlm-roberta-large-tokenizer.json", and have no "vocab.txt", so how to use the package “XLMRobertaTokenizer” to load the the file  "xlm-roberta-large-tokenizer.json"?
<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:


Add model cards for DynaBERT

Related to pretrained model cards

# What does this PR do?
Add model cards for DynaBERT.


update version for scipy

# What does this PR do?
 updating version requirement for scipy in 'examples\\distillation\\requirements.txt'.
fix [https://github.com/huggingface/transformers/issues/7967](url)


Create README.md

Related to pretrained model cards

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)
Add model card about DynaBERT in HuggingFace.io.

## Before submitting
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


Added model cards for Tagalog ELECTRA models

Related to pretrained model cards

# What does this PR do?
Added model cards for eight ELECTRA Tagalog models (four generators and four discriminators).


[CI] generate separate report files as artifacts

This PR solves https://github.com/huggingface/transformers/issues/7887 to produce easier to use reports on CIs.

* [x] adds an optional `--make_reports=id` to `pytest`- e.g. `--make_reports=examples`. It then uses that id to generate `report_{id}_{reports}.txt` - this was needed since some jobs like on scheduled jobs have multiple pytest runs, so a unique string is required. W/o this new flag everything remains as is - i.e. no reports get generated
* [x] the generated reports are all saved under `reports` to simplify the upload and are at the moment (assuming `id` was `tests`):

   - report_tests_durations.txt
   - report_tests_errors.txt
   - report_tests_failures.txt
   - report_tests_passes.txt
   - report_tests_short_summary.txt
   - report_tests_stats.txt
   - report_tests_warnings.txt

   We no longer need any `pytests` flags to generate these - e.g. no need for `-rA` or `-durations=` - they are all done internally.

   The code itself is a bit of a hack, that borrows a lot of `pytest` internals - but that's a start - I will see if I can find a public API to accomplish the same later if this new functionality catches on. Actually, it's pretty safe since it calls the same report functions `pytest` uses, so it's unlikely to break.

* [x] added the reporting to:
   - CirlcleCI `run_examples_torch` and `run_tests_torch` jobs
   - GitHub workflow `run_all_tests_torch_and_tf_gpu` job. (this one generates 3 (!) groups of reports)

Once these are tested on `master` and the results are satisfactory, I will add this new functionality to the rest of the jobs.

**This is what you want to review**:
- the latest [report](https://app.circleci.com/pipelines/github/huggingface/transformers/14586/workflows/21a114bc-c65b-4b62-b747-a0056923479a/jobs/106843)
- the corresponding [artifacts](https://app.circleci.com/pipelines/github/huggingface/transformers/14586/workflows/21a114bc-c65b-4b62-b747-a0056923479a/jobs/106843/artifacts)

Fixes: #7887

@sshleifer, @sgugger, @LysandreJik


BertTokenizer's add_token won't add token

`add_token` actually won't add token. Please refer to the code below:
```
from transformers import BertTokenizer
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased', do_lower_case=True)
tokenizer.vocab_size
>>30522

tokenizer.add_tokens(new_tokens=['[SUBJ]', '[OBJ]'], special_tokens=True)
>>2

tokenizer.vocab['[OBJ]']
>> KeyError: '[OBJ]'

tokenizer.vocab_size
>>30522 # not changed

tokenizer.tokenize('[OBJ]')
>>['[', 'ob', '##j', ']'] # expected: '[OBJ]'
```


[docs] [testing] distributed training

We figured out how to support distributed training with `pytest`, this is a preliminary doc snippet to help those in need to find the current implementation. I'm sure it will evolve as we  have more tests with varying needs, but for now that's all we have.

@sgugger


update zero shot default widget example

Related to pretrained model cards

# What does this PR do?

Just changing bart's zero shot widget example.


[Reformer] remove reformer pad_token_id

# What does this PR do?

The `crime-and-punishment` tokenizer actually does not have a `pad_token_id` - check with this [notebook](https://colab.research.google.com/github/google/trax/blob/master/trax/models/reformer/text_generation.ipynb#scrollTo=iDgvKNa_DDIq). Since this is our only tokenizer for Reformer, we should remove the `pad_token` completely from the Reformer Tokenizer script (otherwise `tokenizer.pad_token_id` get's an id >= tokenizer.max_len`).

Since `crime-and-punishment` runs on causal attention, any token can be set to the padding token during inference.
Thus before padding one should do `tokenizer.pad_token = tokenizer.eos_token`.

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #7929

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


Handling longformer model_type

Updating the run_squad training script to handle the "longformer" `model_type`. The longformer is trained in the same was as RoBERTa, so I've added the "longformer" `model_type` (that's the right hugginface name for the LongFormer model, right?) everywhere there was a "roberta" `model_type` reference. The longformer (like RoBERTa) doesn't use `token_type_ids` (as I understand from looking at the [longformer notebook](https://github.com/patil-suraj/Notebooks/blob/master/longformer_qa_training.ipynb), which is what gets updated after this change.

This fix might be related to [this issue](https://github.com/huggingface/transformers/issues/7249) with SQuAD training when using run_squad.py

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


[gh ci] less output ( --durations=50)

Way too much output in [this](https://github.com/huggingface/transformers/pull/7989)
This will make it slightly better.
cc @stas00


[Good first issue] Documentation links in older docs versions

# 🚀 Feature request

This is a documentation request in order to make it easier to find corresponding examples in the documentation.

Good first issue if you want to get acquainted with the docs and how to build docs using Sphinx!

## Current issue

Here's the issue: currently, if one goes to an older documentation version to check the "examples" page, for example, [v2.6.0](https://huggingface.co/transformers/v2.6.0/examples.html), all links point towards the `master` branch.

For example, the link towards `run_tf_glue.py` is the following: https://github.com/huggingface/transformers/blob/master/examples/run_tf_glue.py

As this points towards the `master` branch, it is prone to breaking as files can (and probably will) be moved around as versions come out. It is the case for this example, as the `run_tf_glue.py` script is not in `examples/` anymore, but in `examples/text-classification/`.

I think we need a way to ensure that all links point toward their appropriate version, and the easiest would be to point to a given tag. Since we're looking at the version `v2.6.0`, it makes sense to point towards the tag v2.6.0: https://github.com/huggingface/transformers/blob/v2.6.0/examples/run_tf_glue.py

This way links get frozen in time and redirect to actual files corresponding to their description and behaviour as stated in the docs.

## Resolution

I believe the easiest change would be to use sphinx variables in order to do this. Probably either [rst_epilog](https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-rst_epilog) or [rst_prolog](https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-rst_prolog) could be useful here.

Some useful links: [rst_epilog](https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-rst_epilog), [rst_prolog](https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-rst_prolog)


TFMarian, TFMbart, TFPegasus, TFBlenderbot

### Notes:
- add `TFSinusoidalPositionalEmbeddings`.
- Code structure identical to the corresponding pytorch code -- same classes, implementations differ only slightly.
- Integration tests and common tests, rst updates, for all 4 children. All 4 children run the same common tests as TFBart and at least 1 integration test.
- For pegasus, generations are not identical to PT because Linear layers are slightly different in tf/pt. For Marian, generations are identical.
- Loading will generate 0 warnings.


T5 Decoder Inputs

# ❓ Questions & Help
Just confirming that my data preprocessing is perfect for T5. I added a print statement in `T5ConditionalGeneration` for the `decoder_input_ids` and `decoder_attention_mask` just before they're passed to the decoder. Which of these is right?

```
# pad_token prepended, eos_token is not in the sequence
decoder_input_ids: tensor([[0, 2018, 55, 0, 0, 0, 0]])
decoder_attention_mask: tensor([[1, 1, 1, 0, 0, 0, 0]])

# pad_token prepended, eos_token unmasked in attention
decoder_input_ids: tensor([[0, 2018, 55, 1, 0, 0, 0]])
decoder_attention_mask: tensor([[1, 1, 1, 1, 0, 0, 0]])

# pad_token prepended, eos_token masked in attention
decoder_input_ids: tensor([[0, 2018, 55, 1, 0, 0, 0]])
decoder_attention_mask: tensor([[1, 1, 1, 0, 0, 0, 0]])
```


[setup] require torch>=1.4

I run the non-slow test suite on lower torch versions:

* torch-1.2 and below is definitely a no go - a gazillion of errors in the test suite.

* torch-1.3+ is mostly OK, but:
```
FAILED tests/test_modeling_bart.py::BartHeadTests::test_generate_fp16 - RuntimeError: "argmax_cuda" not implemented for 'Half'
FAILED tests/test_modeling_funnel.py::FunnelModelIntegrationTest::test_inference_tiny_model - OSError: Unable to load weights from pytorch checkpoi...
FAILED tests/test_modeling_gpt2.py::GPT2ModelTest::test_model_outputs_equivalence - RuntimeError: Expected object of scalar type Float but got scal...
FAILED tests/test_modeling_lxmert.py::LxmertModelTest::test_lxmert_pretraining - RuntimeError: Expected object of scalar type Float but got scalar ...
FAILED tests/test_modeling_openai.py::OpenAIGPTModelTest::test_model_outputs_equivalence - RuntimeError: Expected object of scalar type Float but g...
FAILED tests/test_modeling_reformer.py::ReformerLocalAttnModelTest::test_reformer_model_fp16_generate - RuntimeError: "argmax_cuda" not implemented...
FAILED tests/test_modeling_reformer.py::ReformerLSHAttnModelTest::test_reformer_model_fp16_forward - RuntimeError: "argmax_cuda" not implemented fo...
FAILED tests/test_modeling_reformer.py::ReformerLSHAttnModelTest::test_reformer_model_fp16_generate - RuntimeError: "argmax_cuda" not implemented f...
```
which could be fixed in the core if desired, but it won't pass as is right now.

* torch-1.4 - has mostly serialization issues (files saved with new pytorch can't be read by the old one)

Hence changing to `torch>=1.4`

@sgugger, @LysandreJik


Reload checkpoint

# What does this PR do?

This PR fixes a few bugs linked to resuming training from a checkpoint, mainly:
- the progress was not properly displayed (beginning at 0 instead of the step from the checkpoint)
- reloading the optimizer state and scheduler state on TPU was causing an error

Tested on TPU, single-GPU and multi-GPU env.

Fixes #4963
Fixes #7976


add zero shot pipeline tags & examples

Related to pretrained model cards

# What does this PR do?

Adds the zero shot pipeline tag as well as default examples for a selection of pre-trained MNLI models. cc @patil-suraj


[s2s test] examples/seq2seq/test_finetune_trainer.py::TestFinetuneTrainer::test_finetune_trainer_slow fails on GPU

This works (cpu / any pytorch):
```
CUDA_VISIBLE_DEVICES="" RUN_SLOW=1 pytest -sv examples/seq2seq/test_finetune_trainer.py::TestFinetuneTrainer::test_finetune_trainer_slow
```
This fails torch-1.5/gpu or 1.6, or nightly:
```
CUDA_VISIBLE_DEVICES="0" RUN_SLOW=1 pytest -sv examples/seq2seq/test_finetune_trainer.py::TestFinetuneTrainer::test_finetune_trainer_slow
```
Same with pytorch-nightly. same with py37 and py38.

Error:
```
{'eval_loss': 5223.2333984375, 'eval_bleu': 0.0, 'eval_gen_len': 1.0, 'epoch': 1.0}
{'eval_loss': 5064.154296875, 'eval_bleu': 0.0, 'eval_gen_len': 1.0, 'epoch': 2.0}
{'eval_loss': 4966.837890625, 'eval_bleu': 0.0, 'eval_gen_len': 3.8, 'epoch': 3.0}
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 6/6 [00:03<00:00,  1.55it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00,  4.69it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:01<00:00,  3.04it/s]FAILED

====================================================================== FAILURES ======================================================================
___________________________________________________ TestFinetuneTrainer.test_finetune_trainer_slow ___________________________________________________

self = <seq2seq.test_finetune_trainer.TestFinetuneTrainer testMethod=test_finetune_trainer_slow>

100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:01<00:00,  2.78it/s]
    @slow
    def test_finetune_trainer_slow(self):
        # There is a missing call to __init__process_group somewhere
        output_dir = self.run_trainer(eval_steps=2, max_len="128", model_name=MARIAN_MODEL, num_train_epochs=3)

        # Check metrics
        logs = TrainerState.load_from_json(os.path.join(output_dir, "trainer_state.json")).log_history
        eval_metrics = [log for log in logs if "eval_loss" in log.keys()]
        first_step_stats = eval_metrics[0]
        last_step_stats = eval_metrics[-1]

>       assert first_step_stats["eval_bleu"] < last_step_stats["eval_bleu"]  # model learned nothing
E       AssertionError: assert 0.0 < 0.0

examples/seq2seq/test_finetune_trainer.py:36: AssertionError
```
env:
```
- `transformers` version: 3.4.0
- Platform: Linux-4.15.0-118-generic-x86_64-with-debian-buster-sid
- Python version: 3.7.9
- PyTorch version (GPU?): 1.5.0 (True)
- Tensorflow version (GPU?): 2.3.1 (True)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
```

@sshleifer


Only log total_flos at the end of training

# What does this PR do?

This PR removes the addition of `total_flos` at each (and every) log, since this kind of pollutes them, and only logs it once and for all at the end of training. Users can still define their own callbacks and do more with that value if they really want to, but from what I understood. @TevenLeScao, that value is mainly necessary at the end.

Also, now that it's not in the metrics anymore, I've reverted the default compute metrics to its previous behavior (sum of all metrics) since it's the documented behavior. (cc @madlag) If we want to really change it, we need to put more examples out there.


# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers
     -->

## Details
<!-- Description of your issue -->
Hey, I want to use EncoderDecoderModel for parallel trainging. When I save my model, I got the following questions. How can I fix this ?
'DistributedDataParallel' object has no attribute 'save_pretrained'
<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:


How to make some structural changes to the EncoderDecoderModel ?

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers
     -->

## Details
<!-- Description of your issue -->
Hey , I use EncoderDecoderModel for abstractive summarization. I load the bert2bert model like this
model=EncoderDecoderModel.from_encoder_decoder_pretrained('bert-base-uncased', 'bert-base-uncased')

And I want to make some structural changes to the output layer of  decoder model.

For example, in one decoder step, the output hidden state of bert-decoder is a vector (s).  I use another network and I get a vector (w) to make the summarization more accurate. I want to concatenate the two vectors in the output layer and use the final vector to generate a word in the vocabulary.

How can I do this ?
<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:


Disable inference API for t5-11b

Related to pretrained model cards


GPT2 - Remove else branch adding 0 to the hidden state if token_type_embeds is None.

Currently, when `token_type_embeds` is `None` we set its value to `0` and add it to the triplet `inputs_embeds + position_embeds + token_type_embeds`.

This can be simplified to:
- avoid summing 0 over many elements
- avoid using raw Python scalar value which cannot be traced by TorchScript / ONNX when exporting.

Leading to:

> [ONNXRuntimeError] : 1 : FAIL : TensorRT input: 200 has no shape specified. Please run shape inference on the onnx model first.

Signed-off-by: Morgan Funtowicz <funtowiczmo@gmail.com>