Error while trying to retrieve BERT embeddings for a custom task

# 🐛 Bug

## Information
I am using BERT [base-uncased].
Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

Here's the error:
```
I0615 15:19:52.531945 140468956133120 file_utils.py:41] PyTorch version 1.1.0 available.
I0615 15:19:53.723086 140468956133120 file_utils.py:57] TensorFlow version 2.0.0 available.
bert-base-cased
I0615 15:19:53.860575 140468956133120 configuration_utils.py:256] loading configuration file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-cased-config.json from cache at /users2/user1/.cache/torch/transformers/b945b69218e98b3e2c95acf911789741307dec43c698d35fad11c1ae28bda352.9da767be51e1327499df13488672789394e2ca38b877837e52618a67d7002391
I0615 15:19:53.861034 140468956133120 configuration_utils.py:292] Model config BertConfig {
  "architectures": [
    "BertForMaskedLM"
  ],
  "attention_probs_dropout_prob": 0.1,
  "bos_token_id": null,
  "do_sample": false,
  "eos_token_ids": null,
  "finetuning_task": null,
  "hidden_act": "gelu",
  "hidden_dropout_prob": 0.1,
  "hidden_size": 768,
  "id2label": {
    "0": "LABEL_0",
    "1": "LABEL_1"
  },
  "initializer_range": 0.02,
  "intermediate_size": 3072,
  "is_decoder": false,
  "label2id": {
    "LABEL_0": 0,
    "LABEL_1": 1
  },
  "layer_norm_eps": 1e-12,
  "length_penalty": 1.0,
  "max_length": 20,
  "max_position_embeddings": 512,
  "model_type": "bert",
  "num_attention_heads": 12,
  "num_beams": 1,
  "num_hidden_layers": 12,
  "num_labels": 2,
  "num_return_sequences": 1,
  "output_attentions": false,
  "output_hidden_states": true,
  "output_past": true,
  "pad_token_id": 0,
  "pruned_heads": {},
  "repetition_penalty": 1.0,
  "temperature": 1.0,
  "top_k": 50,
  "top_p": 1.0,
  "torchscript": false,
  "type_vocab_size": 2,
  "use_bfloat16": false,
  "vocab_size": 28996
}

I0615 15:19:53.934896 140468956133120 configuration_utils.py:256] loading configuration file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased-config.json from cache at /users2/user1/.cache/torch/transformers/4dad0251492946e18ac39290fcfe91b89d370fee250efe9521476438fe8ca185.7156163d5fdc189c3016baca0775ffce230789d7fa2a42ef516483e4ca884517
I0615 15:19:53.935213 140468956133120 configuration_utils.py:292] Model config BertConfig {
  "architectures": [
    "BertForMaskedLM"
  ],
  "attention_probs_dropout_prob": 0.1,
  "bos_token_id": null,
  "do_sample": false,
  "eos_token_ids": null,
  "finetuning_task": null,
  "hidden_act": "gelu",
  "hidden_dropout_prob": 0.1,
  "hidden_size": 768,
  "id2label": {
    "0": "LABEL_0",
    "1": "LABEL_1"
  },
  "initializer_range": 0.02,
  "intermediate_size": 3072,
  "is_decoder": false,
  "label2id": {
    "LABEL_0": 0,
    "LABEL_1": 1
  },
  "layer_norm_eps": 1e-12,
  "length_penalty": 1.0,
  "max_length": 20,
  "max_position_embeddings": 512,
  "model_type": "bert",
  "num_attention_heads": 12,
  "num_beams": 1,
  "num_hidden_layers": 12,
  "num_labels": 2,
  "num_return_sequences": 1,
  "output_attentions": false,
  "output_hidden_states": false,
  "output_past": true,
  "pad_token_id": 0,
  "pruned_heads": {},
  "repetition_penalty": 1.0,
  "temperature": 1.0,
  "top_k": 50,
  "top_p": 1.0,
  "torchscript": false,
  "type_vocab_size": 2,
  "use_bfloat16": false,
  "vocab_size": 30522
}

I0615 15:19:54.006190 140468956133120 tokenization_utils.py:501] loading file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased-vocab.txt from cache at /users2/user1/.cache/torch/transformers/26bc1ad6c0ac742e9b52263248f6d0f00068293b33709fae12320c0e35ccfbbb.542ce4285a40d23a559526243235df47c5f75c197f04f37d1a0c124c32c9a084
I0615 15:19:54.133543 140468956133120 modeling_utils.py:461] loading weights file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-cased-pytorch_model.bin from cache at /users2/user1/.cache/torch/transformers/35d8b9d36faaf46728a0192d82bf7d00137490cd6074e8500778afed552a67e5.3fadbea36527ae472139fe84cddaa65454d7429f12d543d80bfc3ad70de55ac2
BertConfig {
  "architectures": [
    "BertForMaskedLM"
  ],
  "attention_probs_dropout_prob": 0.1,
  "bos_token_id": null,
  "do_sample": false,
  "eos_token_ids": null,
  "finetuning_task": null,
  "hidden_act": "gelu",
  "hidden_dropout_prob": 0.1,
  "hidden_size": 768,
  "id2label": {
    "0": "LABEL_0",
    "1": "LABEL_1"
  },
  "initializer_range": 0.02,
  "intermediate_size": 3072,
  "is_decoder": false,
  "label2id": {
    "LABEL_0": 0,
    "LABEL_1": 1
  },
  "layer_norm_eps": 1e-12,
  "length_penalty": 1.0,
  "max_length": 20,
  "max_position_embeddings": 512,
  "model_type": "bert",
  "num_attention_heads": 12,
  "num_beams": 1,
  "num_hidden_layers": 12,
  "num_labels": 2,
  "num_return_sequences": 1,
  "output_attentions": false,
  "output_hidden_states": true,
  "output_past": true,
  "pad_token_id": 0,
  "pruned_heads": {},
  "repetition_penalty": 1.0,
  "temperature": 1.0,
  "top_k": 50,
  "top_p": 1.0,
  "torchscript": false,
  "type_vocab_size": 2,
  "use_bfloat16": false,
  "vocab_size": 28996
}

len(input_ids):  636
Num Batches: 40
---------------------------------------
---------------------------------------
---------------------------------------
Traceback (most recent call last):
  File "./dlatkInterface.py", line 2020, in <module>
    main()
  File "./dlatkInterface.py", line 1013, in main
    args.feattable = fe.addBERTTable_(modelName = args.bertmodel, aggregations=args.bertaggs, layersToKeep=args.bertlayers, noContext=args.bertnocontext, layerAggregations = args.bertlayeraggs, wordAggregations=args.transwordaggs, valueFunc = args.valuefunc)
  File "/users2/user1/NLP/dlatk/dlatk/featureExtractor.py", line 1327, in addBERTTable_
    encAllLayers = model(input_ids = input_ids_padded, attention_mask = attention_mask_padded,  token_type_ids = token_type_ids_padded)
  File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/users2/user1/.local/lib/python3.5/site-packages/transformers/modeling_bert.py", line 783, in forward
    input_ids=input_ids, position_ids=position_ids, token_type_ids=token_type_ids, inputs_embeds=inputs_embeds
  File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/users2/user1/.local/lib/python3.5/site-packages/transformers/modeling_bert.py", line 173, in forward
    inputs_embeds = self.word_embeddings(input_ids)
  File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/sparse.py", line 117, in forward
    self.norm_type, self.scale_grad_by_freq, self.sparse)
  File "/usr/local/lib/python3.5/dist-packages/torch/nn/functional.py", line 1506, in embedding
    return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: index out of range at /pytorch/aten/src/TH/generic/THTensorEvenMoreMath.cpp:193
```

I am just trying to retrieve the embeddings from the layers that I want and store it in a list. Here's the code block that hits the error:

```
config = AutoConfig.from_pretrained(modelName, output_hidden_states=True)
tokenizer = AutoTokenizer.from_pretrained(tokenizerName)
model = AutoModel.from_pretrained(modelName, config=config)
cuda = False
model.eval()
batch_size=16
.
.
.
num_batches = int(np.ceil(len(input_ids)/batch_size))
encSelectLayers = []
print ('len(input_ids): ',len(input_ids))
print ('Num Batches:', num_batches)

for i in range(num_batches):
    input_ids_padded = pad_sequence(input_ids[i*batch_size:(i+1)*batch_size], batch_first = True, padding_value=tokenizer.pad_token_id)
    token_type_ids_padded = pad_sequence(token_type_ids[i*batch_size:(i+1)*batch_size], batch_first = True, padding_value=0)
    attention_mask_padded = pad_sequence(attention_mask[i*batch_size:(i+1)*batch_size], batch_first = True, padding_value=0)

    if cuda:
        input_ids_padded = input_ids_padded.to('cuda')
        token_type_ids_padded = token_type_ids_padded.to('cuda')
        attention_mask_padded = attention_mask_padded.to('cuda')

    input_ids_padded = input_ids_padded.long()
    token_type_ids_padded = token_type_ids_padded.long()
    attention_mask_padded = attention_mask_padded.long()

    #print (input_ids_padded.shape, token_type_ids_padded.shape, attention_mask_padded.shape)
    #print (input_ids_padded)
    #print (token_type_ids_padded)
    #print (attention_mask_padded)
    print ('---------------------------------------')
    with torch.no_grad():

        encAllLayers = model(input_ids = input_ids_padded, attention_mask = attention_mask_padded,  token_type_ids = token_type_ids_padded)
        encAllLayers = encAllLayers[-1]
        for lyr in layersToKeep: #Shape: (num Layers, num_batches, batch_size, max Seq len, 768)
            encSelectLayers.append([encAllLayers[int(lyr)].detach().cpu().numpy()])

        del encAllLayers
attention_mask_padded.shape)
print (np.array(encSelectLayers).shape)
```

## Environment info

- `transformers` version: 2.5.1
- Platform:  Linux-4.4.0-171-generic-x86_64-with-debian-stretch-sid
- Python version: 3.5.2
- CUDA version: 10.1
- PyTorch version (GPU?): 1.1 (True)
- Tensorflow version (GPU?): 2.0 (True)
- Using GPU in script: No
- Using distributed or parallel set-up in script?: No


Convert hans to Trainer

This follows up from #4854 (@julien-c I took all your comments into account) and will close #4742.


[Bart] Question Answering Model is added to tests

As written in PR #4908, the Bart for QA was not added to the test suite. This PR fixes the output attentions test for encoder decoder QA models.

If we would have named tuples, such a test could be made much much cleaner.


Multi class classification using Reformer Model

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->

## Details
<!-- Description of your issue -->
Hi All:

I am trying to implement a **multi-class classification** using the **ReformerModel, ReformerModelWithLMHead** But I don't see any API implementation for the same.
I have 10+ class of text data and wanted to use the pre-trained ReformerModel, ReformerModelWithLMHead to classify the text.

I see classes like RobertaForSequenceClassification have support for text classification but could not find any for Reformer.

Please, let me if it is implemented in the Reformer model or is it work in progress. I tried to find any implementation for the same but could not find any.

ps: I am referring to this paper https://web.stanford.edu/class/cs224n/reports/custom/report21.pdf
where they have implemented text classification using Reformer.
Thank you
Amit
<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:


Latest merge [Benchmark] Memory benchmark utils #4198 fails at Windows

# 🐛 Bug

## Information

Model I am using: Bert

## To reproduce

Steps to reproduce the behavior:

1. Windows
2. Install transformers from source

## Environment info
python = 3.7.6
pytorch = 1.5
cuda = 10.2

I think the problem happens because in code:
https://github.com/huggingface/transformers/blob/master/src/transformers/__init__.py

`from .benchmark import PyTorchBenchmark, PyTorchBenchmarkArguments`

and in code:
https://github.com/huggingface/transformers/blob/master/src/transformers/benchmark/benchmark_utils.py

there is `from signal import SIGKILL`

> signal.SIGKILL
>
> Kill signal.
> It cannot be caught, blocked, or ignored.
> Availability: Unix.

I think it will fail for Windows if you try `from signal import SIGKILL`.

@patrickvonplaten
https://github.com/huggingface/transformers/pull/4198


Add position_ids in TFElectra models docstring

Just a small thing, but it was forgotten when adding those I guess.


Run error when run PPLM in example!

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->

## Details
<!-- Description of your issue -->
![image](https://user-images.githubusercontent.com/49581245/84684669-94247480-af6b-11ea-993b-a2eed85ac805.png)

And after I run other model also report error like this.

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:


Not able to reproduce XNLI results from Google's mBERT weights.

# 🐛 Bug

Hello,
I am using mBERT from https://github.com/google-research/bert/blob/master/multilingual.md and using it with the run_xnli.py file.

Language: Learning the model on English (en) and testing it on French (fr).

The problem arises when I download the mBERT's weights from: https://github.com/google-research/bert/blob/master/multilingual.md and then use https://github.com/huggingface/transformers/blob/master/src/transformers/convert_bert_original_tf_checkpoint_to_pytorch.py to get the PyTorch compatible weights.

I then used the PyTorch compatible weights with run_xnli provided here: https://github.com/huggingface/transformers/blob/master/examples/text-classification/run_xnli.py. Using the language as fr and train_language as en. The model doesn't learn properly and doesn't provide performance above 0.40 on fr.
Though if I use the weights directly provided by huggingface for bert-base-multilingual-cased (as mentioned here: https://huggingface.co/transformers/v2.3.0/examples.html). It works fine and the mBERT model is able to achieve SOTA performance. Would it be possible to understand as why this might be happening?

The tasks I am working on is XNLI. Learning on English (en) and zero-shot testing on French (fr).

Steps to reproduce the behavior:
1. Get the mBERT model from the link provided here: https://github.com/google-research/bert/blob/master/multilingual.md
2.Get the PyTorch compatible weights using: https://github.com/huggingface/transformers/blob/master/src/transformers/convert_bert_original_tf_checkpoint_to_pytorch.py
3. Copy the vocab.txt and config.json from TF weights. Add "model_type"="bert" to config.
4. run the example run_xnli.py provided here: https://github.com/huggingface/transformers/blob/master/examples/text-classification/run_xnli.py
5. Default parameters can be used with model_path set to the PyTorch weights of mBERT.

- `transformers` version: 2.11
- Python version: 3.7
- PyTorch version (GPU): 1.5.0
- Tensorflow version (GPU): 2.2.0
- Using GPU in script: Yes
- Using parallel setup across 3 1080ti GPUs


bart-large-xsum config task_specific_params['summarization_params'] wrong

They are bart-large-cnn-params.


Add CRF layer after Transformer model

I've read a paper titled "Named Entity Recognition in Chinese Electronic Medical Records Using Transformer-CRF".
It takes Transformer's output as CRF's input, as shown in the figure.
Which function could I use to implement it? model.add() doesn't work.
![Screenshot_20200615_173206_cn wps moffice_eng](https://user-images.githubusercontent.com/64955334/84675939-7d782080-af5f-11ea-8d9d-9dc812b2229a.png)


Cached TF files cannot be loaded with `from_pretrained` without internet connection

# 🐛 Bug

## Information
For TF models in latest version of `transformers v2.11.0`, when the internet is turned off, `from_pretrained` throws an error even when the model has previously been downloaded to cache:

```python
# does not work (wifi/internet turned off)
model = TFBertModel.from_pretrained('bert-base-uncased', local_files_only=True)
```
Error is:
```
OSError: Can't load weights for 'bert-base-uncased'. Make sure that:

- 'bert-base-uncased' is a correct model identifier listed on 'https://huggingface.co/models'

- or 'bert-base-uncased' is the correct path to a directory containing a file named one of tf_model.h5, pytorch_model.bin.

```
The above error occurs regardless of how `local_files_only` is set. This can be problematic in certain settings (e.g., air-gapped networks, firewalls).

By contrast, for PyTorch models with NO internet, `from_pretrained` works perfectly and correctly loads files from cache both when `local_files_only=False` and `local_files_only=True`, which is a  useful parameter recently added in PR #2930 by @BramVanroy and merged by @LysandreJik:

```python
# works and is fast (wifi/internet turned off)
model = BertModel.from_pretrained('bert-base-uncased', local_files_only=True)

# works but is slow due to outgoing connection attempts (wifi/internet turned off)
model = BertModel.from_pretrained('bert-base-uncased', local_files_only=False)
```

## Environment Info

- `transformers` version: 2.11.0
- Platform: Linux-4.15.0-88-generic-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.4.0 (True)
- Tensorflow version (GPU?): 2.1.0 (True)


Make DataCollator a callable

As discussed with @julien-c, change the `DataCollator` to be a callable (and the trainer now can take any function as data collator).
I removed the abstract class and made a type alias to keep the type annotations instead.


Add bart-base

Also adds two `@slow` integration test for fill mask tasks.
I made some guesses on the correct conversion, which will hopefully be confirmed by the authors in this [issue](https://github.com/pytorch/fairseq/issues/2242)


[Modelcard] xlm-roberta-squadv2

model card

Related to pretrained model cards


Raise errors that are not raised now

Raise errors that are not raised now.


[Modelcard] bart-squadv2

Related to pretrained model cards


How can I print output attention on test data?

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->

## Details
<!-- Description of your issue -->
I fine-tuned using BART model to carry out Summarization task.
I want to print output attention scores about test data, but I don't know which layer I should change "output_attention" be True.

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:


Create README.md for finetuned BERT model

Related to pretrained model cards


Add model card for StackOBERTflow-comments-small

Related to pretrained model cards


Create README.md

Related to pretrained model cards


[Ignore] Added data collator for XLNet and added related calls

Added modified data collator for XLNet and added related calls in `examples/language-modeling/run_language_modeling.py`


Increase pipeline support for ONNX export.

Supports all kind of models for all the pipelines we've except `summarization` because of `triu`not supported by ONNX.


[🚀 Feature request] upload bart-base checkpoint

# 🚀 Feature request

`bart-base`  checkpoint is now available in fairseq repo
https://github.com/pytorch/fairseq/tree/master/examples/bart#pre-trained-models
so now it can be made available here  `facebook/bart-base`

@sshleifer


Cannot import transformers due to issue with signal.py (SIGKILL)

# 🐛 Bug

Can't import transformers, get a problem with signal.py. I see that an issue was recently fixed... should I install a nightly build, or...?

>>> import transformers
2020-06-15 05:40:54.098560: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_101.dll
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\15194\anaconda3\envs\ai_env\lib\site-packages\transformers\__init__.py", line 371, in <module>
    from .benchmark import PyTorchBenchmark, PyTorchBenchmarkArguments
  File "C:\Users\15194\anaconda3\envs\ai_env\lib\site-packages\transformers\benchmark\__init__.py", line 10, in <module>
    from .benchmark import PyTorchBenchmark
  File "C:\Users\15194\anaconda3\envs\ai_env\lib\site-packages\transformers\benchmark\benchmark.py", line 32, in <module>
    from .benchmark_utils import Benchmark, Memory, measure_peak_memory_cpu, start_memory_tracing, stop_memory_tracing
  File "C:\Users\15194\anaconda3\envs\ai_env\lib\site-packages\transformers\benchmark\benchmark_utils.py", line 19, in <module>
    from signal import SIGKILL
**ImportError: cannot import name 'SIGKILL' from 'signal' (C:\Users\15194\anaconda3\envs\ai_env\lib\signal.py)**

## Expected behavior

The package to import.

## Environment info
- `transformers` version: 2.11
- Platform: Windows 10
- Python version: 3.8.3
- PyTorch version (GPU?): 1.5.0, GPU = yes ( torch.cuda.is_available == True).
- Tensorflow version (GPU?): 2.2.0; GPU = yes.
- Using GPU in script?: No.
- Using distributed or parallel set-up in script?: No.


Illegal memory access (cudaErrorIllegalAddress)

dependencies

Pull requests that update a dependency file

# 🐛 Bug

## Information
This bug/problem has been discussed on [Pytorch](https://github.com/pytorch/pytorch/issues/21819)/[Apex](https://github.com/NVIDIA/apex/issues/319) and [here](https://github.com/huggingface/transformers/issues/460) (bot marked it as stale) also.

I'm using Albert on GLUE (although this issue is model/dataset agnostic).
I've made a slight modifications in my train loop (as compared to `train()` in `Trainer()`.
The main one which throws this error is when I compute the gradients:
```
grad = torch.autograd.grad(loss, model.parameters(), allow_unused=True)
```
where loss is simply `model(**inputs)[0]`

I'm using Pytorch 1.5.0+cu101, transformers 2.11 on one GPU, no multiGPU, although the instance has 2 by (CUDA_VISIBLE_DEVICES=0). I tried with `torch.cuda.set_device()` also.

Can you suggest a workaround ?


Segmentation fault when loading pretrained file

When loading model weights file using <model.from_pretrained>, segmentation fault error occur..


Accessing scores for the entire vocabulary in GPT2

I want to access the prediciton scores from the GPT2 model.

Follwoing the example given in the docstring, I am using the code

```
    output = model(input_ids,labels=input_ids)
    logit = output[1][0][-1,:].detach().numpy()
```
Is it correct ? I am expecting logit variable to be the same size as the vocabulary with corresponding scores.


Improve ONNX logging


Remove deprecation warning TF2.2

`strategy.experimental_run_v2()` is deprecated in favor of `strategy.run()`
https://github.com/tensorflow/tensorflow/blob/2b96f3662bd776e277f86997659e61046b56c315/tensorflow/python/distribute/distribute_lib.py#L953-L957

Fix #4992


Fix importing transformers on Windows - SIGKILL not defined

On Windows platform there is no such `signal.SIGKILL`, we need to send `CTRL_C_EVENT` which is `Ctrl+C`.

This PR aims at:

1. Not importing the `signal.SIGKILL` on Windows platforms as it's undefined
2. Using the right `signal.CTRL_C_EVENT` on Windows platforms and `signal.SIGKILL` on Unix*.


❓ How to use TFTrainer on TPU ? Unable to destroy remote tensor handles

# ❓ Questions & Help

I'm trying to train my model on TPU using `TFTrainer`. Training starts fine, but after a few training steps, I'm having this error :

> Unable to destroy remote tensor handles. If you are running a tf.function, it usually indicates some op in the graph gets an error: 4 root error(s) found.

I don't know what I am doing wrong, any help is greatly appreciated.

---

Here is the full stacktrace :

```
2020/06/15 05:14:31 - INFO - transformers.trainer_tf -   Epoch 2 Step 1500 Train Loss 3.0905
2020/06/15 05:17:33 - INFO - transformers.trainer_tf -   Epoch 2 Step 2000 Train Loss 2.8351
2020/06/15 05:20:35 - INFO - transformers.trainer_tf -   Epoch 2 Step 2500 Train Loss 3.1043
2020-06-15 05:22:53.919745: W tensorflow/core/distributed_runtime/eager/remote_tensor_handle_data.cc:76] Unable to destroy remote tensor handles. If you are running a tf.function, it usually indicates some op in the graph gets an error: 4 root error(s) found.
  (0) Cancelled: {{function_node __inference__accumulate_next_440382}} Function was cancelled before it was started
  (1) Cancelled: {{function_node __inference__accumulate_next_440382}} Function was cancelled before it was started
  (2) Cancelled: {{function_node __inference__accumulate_next_440382}} Function was cancelled before it was started
  (3) Out of range: {{function_node __inference__accumulate_next_440382}} End of sequence
         [[{{node IteratorGetNext_6}}]]
0 successful operations.
5 derived errors ignored.
Traceback (most recent call last):
  File "train.py", line 136, in <module>
    main()
  File "train.py", line 112, in main
    trainer.train()
  File "/home/me/.venv/x/lib/python3.6/site-packages/transformers/trainer_tf.py", line 274, in train
    for training_loss in self._training_steps(train_ds, optimizer):
  File "/home/me/.venv/x/lib/python3.6/site-packages/transformers/trainer_tf.py", line 319, in _training_steps
    self._apply_gradients(optimizer)
  File "/home/me/.venv/x/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py", line 580, in __call__
    result = self._call(*args, **kwds)
  File "/home/me/.venv/x/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py", line 611, in _call
    return self._stateless_fn(*args, **kwds)  # pylint: disable=not-callable
  File "/home/me/.venv/x/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 2420, in __call__
    return graph_function._filtered_call(args, kwargs)  # pylint: disable=protected-access
  File "/home/me/.venv/x/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 1665, in _filtered_call
    self.captured_inputs)
  File "/home/me/.venv/x/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 1746, in _call_flat
    ctx, args, cancellation_manager=cancellation_manager))
  File "/home/me/.venv/x/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 598, in call
    ctx=ctx)
  File "/home/me/.venv/x/lib/python3.6/site-packages/tensorflow/python/eager/execute.py", line 60, in quick_execute
    inputs, attrs, num_outputs)
tensorflow.python.framework.errors_impl.OutOfRangeError: 4 root error(s) found.
  (0) Cancelled: {{function_node __inference__accumulate_next_440382}} Function was cancelled before it was started
  (1) Cancelled: {{function_node __inference__accumulate_next_440382}} Function was cancelled before it was started
  (2) Cancelled: {{function_node __inference__accumulate_next_440382}} Function was cancelled before it was started
  (3) Out of range: {{function_node __inference__accumulate_next_440382}} End of sequence
         [[{{node IteratorGetNext_6}}]]
0 successful operations.
5 derived errors ignored. [Op:__inference__apply_gradients_373775]
```


append keyword arguments to the output

prepare_inputs_for_generation function receives keyword arguments (model_specific_kwargs)
However, these arguments are not ignored.
I fix this codes to return arguments including these keyword arguments.

Actually, I don't have an idea about the purpose of prepare_logits_for_generation function, which only returns the logit values.


🐛 TFTrainer not working on TPU (TF2.2)

TensorFlow

Anything TensorFlow

# 🐛 Bug

## Information

The problem arises when using:
* [ ] the official example scripts
* [x] my own modified scripts

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: CNN/DM
* [ ] my own task or dataset

## To reproduce

Steps to reproduce the behavior:

1. Install `transformers` from `master`
2. Run TPU training using `TFTrainer`

I get the following error :

>TypeError: Failed to convert object of type <class 'transformers.optimization_tf.AdamWeightDecay'> to Tensor. Contents: <transformers.optimization_tf.AdamWeightDecay object at 0x7faddc7cfe80>. Consider casting elements to a supported type.

---

Here :
https://github.com/huggingface/transformers/blob/9931f817b75ecb2c8bb08b6e9d4cbec4b0933935/src/transformers/trainer_tf.py#L324

we pass `optimizer` as arguments.

But according to the documentation in TF :
https://github.com/tensorflow/tensorflow/blob/2b96f3662bd776e277f86997659e61046b56c315/tensorflow/python/distribute/distribute_lib.py#L890-L891

>All arguments in `args` or `kwargs` should either be nest of tensors or
    `tf.distribute.DistributedValues` containing tensors or composite tensors.

## Environment info

- `transformers` version: 2.11.0
- Platform: Linux-4.9.0-9-amd64-x86_64-with-debian-9.12
- Python version: 3.6.9
- PyTorch version (GPU?): 1.5.0 (False)
- Tensorflow version (GPU?): 2.2.0 (False)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: TPU training


Importing transformers causes segmentation fault when setting cuda device

# 🐛 Bug

## Information

The problem arises when using:
 my own modified scripts: (give details below)

## To reproduce
```
import torch
import transformers

def main(local_rank):
    torch.cuda.set_device(local_rank)
    device = torch.device('cuda', local_rank)

if __name__ == "__main__":
    print (torch.__version__)
    print (transformers.__version__)
    print (torch.cuda.is_available())
    main(0)

```

## Expected behavior
```
1.4.0
2.11.0
True
Segmentation fault (core dumped)
```
if commenting out `import transformers`,  everything will be fine.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:  2.11.0
- Platform: linux
- Python version:  3.6.8
- PyTorch version (GPU?): 1.4.0 GPU
- Tensorflow version (GPU?): None
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No


[TFTrainer] Tensorflow Warning : experimental_run_v2 is deprecated

# 🐛 Bug

When running my code with TFTrainer, I'm receiving :

>WARNING - tensorflow -   From /home/me/.venv/bart/lib/python3.6/site-packages/transformers/trainer_tf.py:355: StrategyBase.experimental_run_v2 (from tensorflow.python.distribute.distribute_lib) is deprecated and will be removed in a future version.
Instructions for updating:
renamed to `run`

Is it expected ? How can I remove this warning ?

## Information

The problem arises when using:
* [ ] the official example scripts
* [x] my own modified scripts

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: CNN/DM
* [ ] my own task or dataset

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.11.0
- Platform: Linux-4.9.0-9-amd64-x86_64-with-debian-9.12
- Python version: 3.6.9
- PyTorch version (GPU?): 1.5.0 (False)
- Tensorflow version (GPU?): 2.2.0 (False)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: Using TPU training


evaluating with trainer.py with TPU results in sudden RAM spike and crash

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): Roberta with the official how to train from scratch example

Language I am using the model on (English, Chinese ...): Esperanto

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. follow the official how to train with esperanto example
2. define a second dataset for testing, both datasets are TextDatasets, the train set is around 300MB, the test set around 3MB
3. run trainer.train() with evaluate_during_training

The evaluation loop runs a few examples before crashing the colab session due to an unknown reason. I have managed to increase the number of examples from 2 to 6 by reducing the per_device_eval_batch_size. My eval dataset only contains 31 examples. When checking the colab logs after crashing, it seems that trainer.py tried to allocate 4GB of memory before crashing, which seems unrealistic to me given the size of the datasets.

Specifically (I think this is the relevant part, please correct me if I'm wrong):

2020-06-14 21:45:53.187694: E tensorflow/compiler/xla/xla_client/xla_util.cc:76] (1) Resource exhausted: Attempting to reserve 2.78G at the bottom of memory. That was not possible. There are 2.75G free, 0B reserved, and 2.75G reservable.

When switching to GPU, I encounter a similar issue, but this time with a complete stack trace because this doesn't crash the runtime:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-14-10e96b9a36a3> in <module>()
----> 1 trainer.train()
      2 trainer.save_model("./drive/My Drive/models/roberta/output")

2 frames
<ipython-input-11-34c6e8e4c40d> in train(self, model_path)
    507
    508                         if self.args.evaluate_during_training:
--> 509                             self.evaluate()
    510
    511                     if self.args.save_steps > 0 and self.global_step % self.args.save_steps == 0:

<ipython-input-11-34c6e8e4c40d> in evaluate(self, eval_dataset, prediction_loss_only)
    706         eval_dataloader = self.get_eval_dataloader(eval_dataset)
    707         print(2)
--> 708         output = self._prediction_loop(eval_dataloader, description="Evaluation")
    709         print(3)
    710         self._log(output.metrics)

<ipython-input-11-34c6e8e4c40d> in _prediction_loop(self, dataloader, description, prediction_loss_only)
    775                     preds = logits.detach()
    776                 else:
--> 777                     preds = torch.cat((preds, logits.detach()), dim=0)
    778                 if inputs.get("labels") is not None:
    779                     if label_ids is None:

RuntimeError: CUDA out of memory. Tried to allocate 6.75 GiB (GPU 0; 15.90 GiB total capacity; 7.99 GiB already allocated; 6.40 GiB free; 8.81 GiB reserved in total by PyTorch)

## Expected behavior

I expected the eval loss or error to be computed.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.11.0
- Platform: colab
- Python version: 3.6
- Using GPU in script?: no, TPU with xla
- Using distributed or parallel set-up in script?: in theory, but only 1 TPU core available


Save the Dataset for training GPT2

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:

I am trying to train GPT2 on a custom dataset, the training text file is over 100GB, when creating the Dataset for training, it is taking too long and I was hoping if there is any way to save the dataset for later use.

I am using hugging face blog post on how to train transformers from scratch as a guide (https://huggingface.co/blog/how-to-train).

Thank You


attention


error while instantiating model

Hi,

For my experiments, I need to make some changes to the forward pass of Roberta sequence classification model. Thus, I copied **RobertaForSequenceClassification** from _modeling_roberta.py_ into a [separate file](https://github.com/kevinghst/mixmatch_from_scratch/blob/master/models_roberta.py) in my repo.

And I try to instantiate it in my _main.py_ like this:
```
    model = RobertaForSequenceClassification.from_pretrained(
        'roberta-base',
        num_labels = NUM_LABELS[cfg.task],
    )
```
However, it gives me the following stacktrace:
----------------------------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/scratch/wz1232/anaconda3/envs/mixmatch/lib/python3.6/site-packages/torch/serialization.py", line 186, in _check_seekable
    f.seek(f.tell())
AttributeError: 'NoneType' object has no attribute 'seek'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/scratch/wz1232/anaconda3/envs/mixmatch/lib/python3.6/site-packages/transformers/modeling_utils.py", line 516, in from_pretrained
    state_dict = torch.load(resolved_archive_file, map_location="cpu")
  File "/scratch/wz1232/anaconda3/envs/mixmatch/lib/python3.6/site-packages/torch/serialization.py", line 368, in load
    return _load(f, map_location, pickle_module)
  File "/scratch/wz1232/anaconda3/envs/mixmatch/lib/python3.6/site-packages/torch/serialization.py", line 517, in _load
    _check_seekable(f)
  File "/scratch/wz1232/anaconda3/envs/mixmatch/lib/python3.6/site-packages/torch/serialization.py", line 189, in _check_seekable
    raise_err_msg(["seek", "tell"], e)
  File "/scratch/wz1232/anaconda3/envs/mixmatch/lib/python3.6/site-packages/torch/serialization.py", line 182, in raise_err_msg
    raise type(e)(msg)
AttributeError: 'NoneType' object has no attribute 'seek'. You can only torch.load from a file that is seekable. Please pre-load the data into a buffer like io.BytesIO and try to load from it instead.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "main.py", line 208, in <module>
    num_labels = NUM_LABELS[cfg.task],
  File "/scratch/wz1232/anaconda3/envs/mixmatch/lib/python3.6/site-packages/transformers/modeling_utils.py", line 519, in from_pretrained
    "Unable to load weights from pytorch checkpoint file. "
OSError: Unable to load weights from pytorch checkpoint file. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.
-----------------------------------------------------------------------------------------------------------

It doesn't occur when I import **RobertaForSequenceClassification** straight from the library, as opposed to from my own file containing the copied code. Why is this?


Fix Inconsistent NER Grouping (Pipeline)

### This PR solves issue #4816 by:

1. Applying entity grouping to similar entity types with different prefixes (i.e. `B` and `I`)
2. Ensuring that separate entities at the last filtered index are no longer excluded from grouping.

Running the sample script below (based on reference issue #4816) returns the expected results. Do note that the `entity_group` is based on the `entity_type` of the first entity in the group.

```
from transformers import pipeline
NER_MODEL = "mrm8488/bert-spanish-cased-finetuned-ner"
nlp_ner = pipeline("ner", model=NER_MODEL,
                   grouped_entities=True,
                   tokenizer=(NER_MODEL, {"use_fast": False}))

t = """Consuelo Araújo Noguera, ministra de cultura del presidente Andrés Pastrana (1998.2002) fue asesinada por las Farc luego de haber permanecido secuestrada por algunos meses."""
nlp_ner(t)

[{'entity_group': 'B-PER', 'score': 0.9710702640669686, 'word': 'Consuelo Araújo Noguera'},
 {'entity_group': 'B-PER', 'score': 0.9997273534536362, 'word': 'Andrés Pastrana'},
 {'entity_group': 'B-ORG', 'score': 0.8589080572128296, 'word': 'Farc'}]
```

I also ran another test to ensure that number 2 (separate entity at the last index) is working properly. I confirmed that it is working properly now.

```
nlp = pipeline('ner', grouped_entities=False)
nlp("Enzo works at the the UN")
[{'entity': 'I-PER', 'index': 1, 'score': 0.9968166351318359, 'word': 'En'},
 {'entity': 'I-PER', 'index': 2, 'score': 0.9957635998725891, 'word': '##zo'},
 {'entity': 'I-ORG', 'index': 7, 'score': 0.9986497163772583, 'word': 'UN'}]

nlp2 = pipeline('ner', grouped_entities=True)
nlp2("Enzo works at the the UN")
[{'entity_group': 'I-PER', 'score': 0.9962901175022125, 'word': 'Enzo'},
 {'entity_group': 'I-ORG', 'score': 0.9986497163772583, 'word': 'UN'}]

```

You can test these out yourself in this colab [notebook](https://colab.research.google.com/drive/1D0xK7MSOQcxOCAe8hpnpFVqdoKrmejSS?usp=sharing).

cc @dav009 @mfuntowicz


BertTokenizer: ValueError: Input is not valid. Should be a string, a list/tuple of strings or a list/tuple of integers.

# 🐛 Bug

## Information

Tokenizer I am using is BertTokenizer and I've also tried using AlbertTokenizer, but it does not have any effect. So I'm thinking that the bug is in the base tokenizer

Language I am using the model on is English, but I don't believe that's the issue.

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Version: ```transformers==2.11.0```
2. Run this code
```python
from transformers import BertModel, BertTokenizer
text = 'A quick brown fox jumps over' # Just a dummy text
BertTokenizer.encode_plus(
    text.split(' '),
    None,
    add_special_tokens = True,
    max_length = 512)
```
3. This should be the error
```
Traceback (most recent call last):
  File "classification.py", line 23, in <module>
    max_length = 512)
  File "D:\Programmering\Python\lib\site-packages\transformers\tokenization_utils.py", line 1576, in encode_plus
    first_ids = get_input_ids(text)
  File "D:\Programmering\Python\lib\site-packages\transformers\tokenization_utils.py", line 1556, in get_input_ids
    "Input is not valid. Should be a string, a list/tuple of strings or a list/tuple of integers."
ValueError: Input is not valid. Should be a string, a list/tuple of strings or a list/tuple of integers.
```

And yes, I've tried just inputting a string, and I still got the same error.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

I want the encoder_plus function to return an encoded version of the input sequence.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.11.0
- Platform: Windows
- Python version: 3.7.4
- PyTorch version (GPU?): 1.5.0+cpu
- Tensorflow version (GPU?): (Not used)
- Using GPU in script?: Nope
- Using distributed or parallel set-up in script?: No


Word Embedding input to GPT-2

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->
Is there any way in which we can give custom word embeddings as input to GPT-2 instead of tokenized words?


FillMaskPipeline return word-piece

# 🚀 . Feature request
Not sure exactly if this is a bug/feature request/ or just me not understanding correctly :).
I am trying to use the [FillMaskPipeline](https://github.com/huggingface/transformers/blob/master/src/transformers/pipelines.py#L739) but as far as I understand the pipeline returns a single word-piece for a \<mask\>. But the mask could be of a word or even a text-spans (for the case of BART).

Sample code:
```python
from transformers import pipeline
nlp =pipeline("fill-mask",model='bart-large')
nlp(f'Their expression was divine; and as they glanced at me timidly but with parted'
    f' lips in great <mask>, I forgot all thoughts of their conversion in feelings '
    f'that were far more earthly.')
#missing word is bewilderment, this is from librispeech

..
{'sequence': '<s> Their expression was divine; and as they glanced at me timidly but with parted lips in great bewild, I forgot all thoughts of their conversion in feelings that were far more earthly.</s>',
  'score': 7.772801473038271e-05,
  'token': 33304},

print(tokenizer.decode(33304))
' bewild'
```
As can be seen above, one of the outputs is "bewild" which is the first word-piece in  bewilderment:
```python
[tokenizer.decode(i)+' ' for i in tokenizer.batch_encode_plus(['bewilderment'])['input_ids'][0]]
['<s> ', ' bewild ', 'er ', 'ment ', '</s> ']
```

## Your contribution

I assume we could go to the next output as see if it is a word-piece and if it is add it to the first token. not sure exactly if this is correct, specially when it seems that the size of the output is exactly as the number of input word-pieces.

Thanks


Getting very bad F1 Scores when training SQUAD v2.0 with robertadistil-base

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
https://stackoverflow.com/questions/62370365/getting-very-bad-f1-scores-when-training-squad-v2-0-with-robertadistil-base

Here is my notebook

https://github.com/manishiitg/ML_Experiments/blob/master/squad_huggingface_experiment_with_Trainer_TPU.ipynb

I am getting very bad f1 scores.

Any help on what i am doing wrong?


Why run_language_modelling.py does not use segment embeddings or language embeddings?

I am trying to train Bert and XLM on my own data. But I found that it seems run_language_modelling.py only tokenizes input text into ids, but it doesn't create token_type_ids or langs for Bert or XLM.

`batch_encoding = tokenizer.batch_encode_plus(lines, add_special_tokens=True, max_length=block_size)`
`self.examples = batch_encoding["input_ids"]`
As you can see from the codes, the examples only contain input_ids.

I set breakpoints in the modeling_bert.py and modeling_xlm.py, and there is no token_type_ids or langs as a part of input to Bert or XLM. For the downstream tasks, Bert and XLM always use segment embeddings or language embeddings as a part of their input. Why don't we use them in the pretraining step?
If we do not use segment embeddings or language embeddings during pretraining, but use them in fine-tuning, shouldn't that cause bias?
Sorry if misunderstood something, but I am quite confused now.


Create README.md


keras

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:


Add Code Coverage and Black badges to README

Add Black badge to README based on this:
https://github.com/huggingface/transformers/blob/403d3098572ac308416653648456a940860da39e/.circleci/config.yml#L101


Output hidden states

Core: Modeling

Internals of the library; Models.

Attempts to close issue #3879 by refactoring all models to take in an extra argument `output_hidden_states` in the `forward()` method.

@patrickvonplaten


[model card] model card for bart-large-finetuned-squadv1

model card

Related to pretrained model cards


Fix parameter 'output_attentions' docstring

This PR fixes parameter 'output_attentions' docstring as follow:

1. Remove duplicate parameter docstring in class OpenAIGPTModel.
2. Fix docstring due to web page display problems.


Create README.md

Related to pretrained model cards

Adding readme file for the model.


Patch 4

Related to pretrained model cards


How to make my own dataset to use BART summarization?

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->

## Details
<!-- Description of your issue -->

Hello I'm trying to use BART summarization model. I have a dataset that take the form of dataframe which has two columns 'document' and 'summary'.

Q1.
I read this Readme.md.
> line.this should make a directory called cnn_dm/ with files like test.source. To use your own data, copy that files format. Each article to be summarized is on its own

I don't understand this sentences well. So I just saw the cnn datasets(train.source, train.target, test.source, test.target). **How can i distinguish between each documents?**

Q2.
**How can i change my own datasets into the cnn files format?**

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:


Run run_tf_glue.py has bugs

do as https://github.com/huggingface/transformers/tree/master/examples/text-classification suggests, but has following bugs showing **/home/admin/tensorflow_datasets/glue/cola/1.0.0/glue-train.tfrecord-00000-of-00001; No such file or directory**

my dataset_info.json copy from https://github.com/tensorflow/datasets/blob/master/tensorflow_datasets/testing/metadata/glue/cola/1.0.0/dataset_info.json

`06/13/2020 14:21:11 - INFO - absl -   Field info.citation from disk and from code do not match. Keeping the one from code.
06/13/2020 14:21:11 - INFO - absl -   Field info.location from disk and from code do not match. Keeping the one from code.
06/13/2020 14:21:11 - INFO - absl -   Reusing dataset glue (/home/admin/tensorflow_datasets/glue/cola/1.0.0)
06/13/2020 14:21:11 - INFO - absl -   Constructing tf.data.Dataset for split train, from /home/admin/tensorflow_datasets/glue/cola/1.0.0
dddddddddddddd: <PrefetchDataset shapes: {idx: (), label: (), sentence: ()}, types: {idx: tf.int32, label: tf.int64, sentence: tf.string}>
2020-06-13 14:21:12.132176: W tensorflow/core/framework/op_kernel.cc:1622] OP_REQUIRES failed at iterator_ops.cc:929 : Not found: /home/admin/tensorflow_datasets/glue/cola/1.0.0/glue-train.tfrecord-00000-of-00001; No such file or directory
Traceback (most recent call last):
  File "./examples/text-classification/run_tf_glue.py", line 229, in <module>
    main()
  File "./examples/text-classification/run_tf_glue.py", line 175, in main
    if training_args.do_train
  File "./examples/text-classification/run_tf_glue.py", line 55, in get_tfds
    return glue_convert_examples_to_features(ds, tokenizer, max_seq_length, task_name)
  File "/export/sdb/test/tools/anaconda3/envs/tf2env/lib/python3.6/site-packages/transformers/data/processors/glue.py", line 62, in glue_convert_examples_to_features
    return _tf_glue_convert_examples_to_features(examples, tokenizer, max_length=max_length, task=task)
  File "/export/sdb/test/tools/anaconda3/envs/tf2env/lib/python3.6/site-packages/transformers/data/processors/glue.py", line 79, in _tf_glue_convert_examples_to_features
    examples = [processor.tfds_map(processor.get_example_from_tensor_dict(example)) for example in examples]
  File "/export/sdb/test/tools/anaconda3/envs/tf2env/lib/python3.6/site-packages/transformers/data/processors/glue.py", line 79, in <listcomp>
    examples = [processor.tfds_map(processor.get_example_from_tensor_dict(example)) for example in examples]
  File "/export/sdb/test/tools/anaconda3/envs/tf2env/lib/python3.6/site-packages/tensorflow_core/python/data/ops/iterator_ops.py", line 622, in __next__
    return self.next()
  File "/export/sdb/test/tools/anaconda3/envs/tf2env/lib/python3.6/site-packages/tensorflow_core/python/data/ops/iterator_ops.py", line 666, in next
    return self._next_internal()
  File "/export/sdb/test/tools/anaconda3/envs/tf2env/lib/python3.6/site-packages/tensorflow_core/python/data/ops/iterator_ops.py", line 651, in _next_internal
    output_shapes=self._flat_output_shapes)
  File "/export/sdb/test/tools/anaconda3/envs/tf2env/lib/python3.6/site-packages/tensorflow_core/python/ops/gen_dataset_ops.py", line 2673, in iterator_get_next_sync
    _six.raise_from(_core._status_to_exception(e.code, message), None)
  File "<string>", line 3, in raise_from
tensorflow.python.framework.errors_impl.NotFoundError: /home/admin/tensorflow_datasets/glue/cola/1.0.0/glue-train.tfrecord-00000-of-00001; No such file or directory [Op:IteratorGetNextSync]
run.sh: line 15: 15230 Segmentation fault      (core dumped) CUDA_VISIBLE_DEVICES=2 python ./examples/text-classification/run_tf_glue.py --model_name_or_path bert-base-uncased --task_name $TASK_NAME --do_train --do_eval --max_seq_length 12 --per_device_eval_batch_size=8 --per_device_train_batch_size=8 --learning_rate 2e-5 --num_train_epochs 3.0 --output_dir /home/admin/test/transformers/tmp/$TASK_NAME/
`


huggingface distillbert classification using multiprocessing

I am trying to use torch multiprocessing to parallelize the predictions from two separate huggingface distillbert classification models. It seems to be deadlocked at the prediction step. I am using python 3.6.5, torch 1.5.0 and huggingface transformers version 2.11.0. The output from running the code is
```
Tree enc done
Begin tree prediction<------(Comment: Both begin tree
End tree predictions<-------  and end tree predictions)
0.03125429153442383
Dn prediction
Dn enc done
Begin dn predictions<------(Comment: Both begin dn
End dn predictions<-------  and end dn predictions)
0.029727697372436523
----------Done sequential predictions-------------

--------Start Parallel predictions--------------
Tree prediction
Tree enc done
Begin tree prediction. <------(Comment: Process is deadlocked after this)
Dn prediction
Dn enc done
Begin dn predictions. <-------(Comment: Process is deadlocked after this)
```
and the code is
```
def predict(sentences =[], tokenizer=tokenizer,models=(tree_model,dn_model,None)):
  MAX_SENTENCE_LENGTH = 16
  start = time.time()
  input_ids = []
  attention_masks = []
  predictions = []

  tree_model = models[0]
  dn_model = models[1]

  if models[0]:
      print("Tree prediction")
  if models[1]:
    print("Dn prediction")
  for sent in sentences:
    encoded_dict = tokenizer.encode_plus(
                        sent,
                        add_special_tokens = True,
                        max_length = MAX_SENTENCE_LENGTH,
                        pad_to_max_length = True,
                        return_attention_mask = True,
                        return_tensors = 'pt',
                   )

    # Add the encoded sentence to the list.
    input_ids.append(encoded_dict['input_ids'])

    # And its attention mask (simply differentiates padding from non-padding).
    attention_masks.append(encoded_dict['attention_mask'])

  if tree_model:
      print("Tree enc done")
  if dn_model:
    print("Dn enc done")

  # Convert the lists into tensors.
  new_input_ids = torch.cat(input_ids, dim=0)
  new_attention_masks = torch.cat(attention_masks, dim=0)

  with torch.no_grad():
      # Forward pass, calculate logit predictions
    if tree_model:
      print("Begin tree prediction")
      outputs = tree_model(new_input_ids,
                      attention_mask=new_attention_masks)
      print("End tree predictions")
    else:
      print("Begin dn predictions")
      outputs = dn_model(new_input_ids,
                      attention_mask=new_attention_masks)
      print("End dn predictions")
  logits = outputs[0]
  logits = logits.detach().cpu()
  print(time.time()-start)
  predictions = logits
  return predictions

def get_tree_prediction(sentence, tokenizer=tokenizer,models=(tree_model,dn_model, None)):
    return predict(sentences =[sentence], tokenizer=tokenizer,models=models)

def get_dn_prediction(sentence, tokenizer=tokenizer,models=(tree_model,dn_model, None)):
  return predict(sentences =[sentence], tokenizer=tokenizer,models=models)

if __name__ == '__main__':
    sentence = "hello world"
    processes = []
    get_tree_prediction(sentence, tokenizer, (tree_model,None,None))
    get_dn_prediction(sentence, tokenizer, (None,dn_model,None))
    print("----------Done sequential predictions-------------")

    print('\n--------Start Parallel predictions--------------')
    tr_p = mp.Process(target=get_tree_prediction, args=(sentence, tokenizer,
                                                         (tree_model,None,None)))

    tr_p.start()
    processes.append(tr_p)

    dn_p = mp.Process(target=get_dn_prediction, args=(sentence, tokenizer,
                                                       (None,dn_model,None)))
    dn_p.start()
    processes.append(dn_p)

    for p in processes:
        p.join()

```


Handle unexpected weights in checkpoint loading (tf to pytorch)

If checkpoint has additional weights, then the current code will fail to load ignoring those weights.


Request: pretrained distilgpt2-medium, distilgpt2-large models

# Plans for distilgpt2-medium and distilgpt2-large

## Motivation
While distilgpt2 is useful, I was wondering if there are any plans to create a distilgpt2-medium and distilgpt2-large. I'm also wondering how the result of distilgpt2-medium compare to gpt2, and distilgpt2-large compare to gpt2-medium, in size and performance.

Maybe it's not even worth it to have those pretrained, if distilgpt2-medium is larger than gpt2 and perform worse.


Eli5 examples

This PR adds Explain Like I'm Five scripts and models to Transformers.

The `examples/eli5` folder contains training code for the dense retriever and to fine-tune a BART model, the jupyter notebook for the [blog post](https://yjernite.github.io/lfqa.html), and the code for the live demo.

The RetriBert model implements the dense passage retriever. It's basically a wrapper for two Bert models and projection matrices, but it does gradient checkpointing in a way that is very different from [a concurrent PR](https://github.com/huggingface/transformers/pull/4659) and I thought it would be easier to write its own class for now and see if we can merge later.

The Bart files are only modified to add a reference to the ELI5 fine-tuned model on the model repo.


Add Linformer model

# 🌟 New model addition

## Model description

### Linformer: Self-Attention with Linear Complexity

Paper published June 9th on ArXiv: https://arxiv.org/abs/2006.04768

Large transformer models have shown extraordinary success in achieving state-of-the-art results in many natural language processing applications. However, training and deploying these models can be prohibitively costly for long sequences, as the standard self-attention mechanism of the Transformer uses O(n²) time and space with respect to sequence length. In this paper, we demonstrate that the self-attention mechanism can be approximated by a low-rank matrix. We further exploit this finding to propose a new self-attention mechanism, which reduces the overall self-attention complexity from O(n²) to O(n) in both time and space. The resulting linear transformer, the **Linformer**, performs on par with standard Transformer models, while being much more memory- and time-efficient.

## Open source status

* [ ] the model implementation is available: (give details)
* [ ] the model weights are available: (give details)
* [x] who are the authors: Sinong Wang, Belinda Z. Li, Madian Khabsa, Han Fang, Hao Ma


Spanbert TACRED model not found, despite model card

I am trying to load this model: https://huggingface.co/mrm8488/spanbert-large-finetuned-tacred in Transformers 2.11.0. My exact code is:

` tokenizer = AutoTokenizer.from_pretrained("mrm8488/spanbert-large-finetuned-tacred")  model = AutoModel.from_pretrained("mrm8488/spanbert-large-finetuned-tacred")`.

However, I get a 'not found' error: `OSError: Model name 'mrm8488/spanbert-base-finetuned-tacred' was not found in tokenizers model name list (bert-base-uncased, bert-large-uncased, bert-base-cased, bert-large-cased, bert-base-multilingual-uncased, bert-base-multilingual-cased, bert-base-chinese, bert-base-german-cased, bert-large-uncased-whole-word-masking, bert-large-cased-whole-word-masking, bert-large-uncased-whole-word-masking-finetuned-squad, bert-large-cased-whole-word-masking-finetuned-squad, bert-base-cased-finetuned-mrpc, bert-base-german-dbmdz-cased, bert-base-german-dbmdz-uncased, TurkuNLP/bert-base-finnish-cased-v1, TurkuNLP/bert-base-finnish-uncased-v1, wietsedv/bert-base-dutch-cased). We assumed 'mrm8488/spanbert-base-finetuned-tacred' was a path, a model identifier, or url to a directory containing vocabulary files named ['vocab.txt'] but couldn't find such vocabulary files at this path or url.
`.

Guidance would be appreciated. Thanks!


How to Paraphrase with GPT2?

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->


GPT: Weights not being initialized

Hello covid-19 survivors,

I have been trying to use GPT for token classification, however currently there is none from hugging face, hence I copied your code from berttokenclassification and stitched the below code. But it says all the weights not initialized. Did I make a mistake, please help me!!!

`
class GPTClassifier(OpenAIGPTPreTrainedModel):
    def __init__(self, config):
        super().__init__(config)
        self.num_labels = 2

        self.gpt = OpenAIGPTModel(config)
        self.dropout = nn.Dropout(0.1)
        self.classifier = nn.Linear(768, 2)

        self.init_weights()

    @add_start_docstrings(
    """GPT Model with a token classification head on top (a linear layer on top of
    the hidden-states output) e.g. for Named-Entity-Recognition (NER) tasks. """,
    )
    def forward(
        self,
        input_ids=None,
        attention_mask=None,
        token_type_ids=None,
        position_ids=None,
        head_mask=None,
        inputs_embeds=None,
        labels=None,
    ):
        outputs = self.gpt(
            input_ids,
            attention_mask=attention_mask,
            token_type_ids=token_type_ids,
            position_ids=position_ids,
            head_mask=head_mask,
            inputs_embeds=inputs_embeds,
        )

        sequence_output = outputs[0]

        sequence_output = self.dropout(sequence_output)
        logits = self.classifier(sequence_output)

        outputs = (logits,) + outputs[2:]  # add hidden states and attention if they are here
        if labels is not None:
            loss_fct = CrossEntropyLoss()
            # Only keep active parts of the loss
            if attention_mask is not None:
                active_loss = attention_mask.view(-1) == 1
                active_logits = logits.view(-1, self.num_labels)
                active_labels = torch.where(
                    active_loss, labels.view(-1), torch.tensor(loss_fct.ignore_index).type_as(labels)
                )
                loss = loss_fct(active_logits, active_labels)
            else:
                loss = loss_fct(logits.view(-1, self.num_labels), labels.view(-1))
            outputs = (loss,) + outputs

        return outputs  # (loss), scores, (hidden_states), (attentions)
`

weights not initalized:
`Weights of GPTClassifier not initialized from pretrained model: ['gpt.tokens_embed.weight', 'gpt.positions_embed.weight', 'gpt.h.0.attn.bias', 'gpt.h.0.attn.c_attn.weight', 'gpt.h.0.attn.c_attn.bias', 'gpt.h.0.attn.c_proj.weight', 'gpt.h.0.attn.c_proj.bias', 'gpt.h.0.ln_1.weight', 'gpt.h.0.ln_1.bias', 'gpt.h.0.mlp.c_fc.weight', 'gpt.h.0.mlp.c_fc.bias', 'gpt.h.0.mlp.c_proj.weight', 'gpt.h.0.mlp.c_proj.bias', 'gpt.h.0.ln_2.weight', 'gpt.h.0.ln_2.bias', 'gpt.h.1.attn.bias', 'gpt.h.1.attn.c_attn.weight', 'gpt.h.1.attn.c_attn.bias', 'gpt.h.1.attn.c_proj.weight', 'gpt.h.1.attn.c_proj.bias', 'gpt.h.1.ln_1.weight', 'gpt.h.1.ln_1.bias', 'gpt.h.1.mlp.c_fc.weight', 'gpt.h.1.mlp.c_fc.bias', 'gpt.h.1.mlp.c_proj.weight', 'gpt.h.1.mlp.c_proj.bias', 'gpt.h.1.ln_2.weight', 'gpt.h.1.ln_2.bias', 'gpt.h.2.attn.bias', 'gpt.h.2.attn.c_attn.weight', 'gpt.h.2.attn.c_attn.bias', 'gpt.h.2.attn.c_proj.weight', 'gpt.h.2.attn.c_proj.bias', 'gpt.h.2.ln_1.weight', 'gpt.h.2.ln_1.bias', 'gpt.h.2.mlp.c_fc.weight', 'gpt.h.2.mlp.c_fc.bias', 'gpt.h.2.mlp.c_proj.weight', 'gpt.h.2.mlp.c_proj.bias', 'gpt.h.2.ln_2.weight', 'gpt.h.2.ln_2.bias', 'gpt.h.3.attn.bias', 'gpt.h.3.attn.c_attn.weight', 'gpt.h.3.attn.c_attn.bias', 'gpt.h.3.attn.c_proj.weight', 'gpt.h.3.attn.c_proj.bias', 'gpt.h.3.ln_1.weight', 'gpt.h.3.ln_1.bias', 'gpt.h.3.mlp.c_fc.weight', 'gpt.h.3.mlp.c_fc.bias', 'gpt.h.3.mlp.c_proj.weight', 'gpt.h.3.mlp.c_proj.bias', 'gpt.h.3.ln_2.weight', 'gpt.h.3.ln_2.bias', 'gpt.h.4.attn.bias', 'gpt.h.4.attn.c_attn.weight', 'gpt.h.4.attn.c_attn.bias', 'gpt.h.4.attn.c_proj.weight', 'gpt.h.4.attn.c_proj.bias', 'gpt.h.4.ln_1.weight', 'gpt.h.4.ln_1.bias', 'gpt.h.4.mlp.c_fc.weight', 'gpt.h.4.mlp.c_fc.bias', 'gpt.h.4.mlp.c_proj.weight', 'gpt.h.4.mlp.c_proj.bias', 'gpt.h.4.ln_2.weight', 'gpt.h.4.ln_2.bias', 'gpt.h.5.attn.bias', 'gpt.h.5.attn.c_attn.weight', 'gpt.h.5.attn.c_attn.bias', 'gpt.h.5.attn.c_proj.weight', 'gpt.h.5.attn.c_proj.bias', 'gpt.h.5.ln_1.weight', 'gpt.h.5.ln_1.bias', 'gpt.h.5.mlp.c_fc.weight', 'gpt.h.5.mlp.c_fc.bias', 'gpt.h.5.mlp.c_proj.weight', 'gpt.h.5.mlp.c_proj.bias', 'gpt.h.5.ln_2.weight', 'gpt.h.5.ln_2.bias', 'gpt.h.6.attn.bias', 'gpt.h.6.attn.c_attn.weight', 'gpt.h.6.attn.c_attn.bias', 'gpt.h.6.attn.c_proj.weight', 'gpt.h.6.attn.c_proj.bias', 'gpt.h.6.ln_1.weight', 'gpt.h.6.ln_1.bias', 'gpt.h.6.mlp.c_fc.weight', 'gpt.h.6.mlp.c_fc.bias', 'gpt.h.6.mlp.c_proj.weight', 'gpt.h.6.mlp.c_proj.bias', 'gpt.h.6.ln_2.weight', 'gpt.h.6.ln_2.bias', 'gpt.h.7.attn.bias', 'gpt.h.7.attn.c_attn.weight', 'gpt.h.7.attn.c_attn.bias', 'gpt.h.7.attn.c_proj.weight', 'gpt.h.7.attn.c_proj.bias', 'gpt.h.7.ln_1.weight', 'gpt.h.7.ln_1.bias', 'gpt.h.7.mlp.c_fc.weight', 'gpt.h.7.mlp.c_fc.bias', 'gpt.h.7.mlp.c_proj.weight', 'gpt.h.7.mlp.c_proj.bias', 'gpt.h.7.ln_2.weight', 'gpt.h.7.ln_2.bias', 'gpt.h.8.attn.bias', 'gpt.h.8.attn.c_attn.weight', 'gpt.h.8.attn.c_attn.bias', 'gpt.h.8.attn.c_proj.weight', 'gpt.h.8.attn.c_proj.bias', 'gpt.h.8.ln_1.weight', 'gpt.h.8.ln_1.bias', 'gpt.h.8.mlp.c_fc.weight', 'gpt.h.8.mlp.c_fc.bias', 'gpt.h.8.mlp.c_proj.weight', 'gpt.h.8.mlp.c_proj.bias', 'gpt.h.8.ln_2.weight', 'gpt.h.8.ln_2.bias', 'gpt.h.9.attn.bias', 'gpt.h.9.attn.c_attn.weight', 'gpt.h.9.attn.c_attn.bias', 'gpt.h.9.attn.c_proj.weight', 'gpt.h.9.attn.c_proj.bias', 'gpt.h.9.ln_1.weight', 'gpt.h.9.ln_1.bias', 'gpt.h.9.mlp.c_fc.weight', 'gpt.h.9.mlp.c_fc.bias', 'gpt.h.9.mlp.c_proj.weight', 'gpt.h.9.mlp.c_proj.bias', 'gpt.h.9.ln_2.weight', 'gpt.h.9.ln_2.bias', 'gpt.h.10.attn.bias', 'gpt.h.10.attn.c_attn.weight', 'gpt.h.10.attn.c_attn.bias', 'gpt.h.10.attn.c_proj.weight', 'gpt.h.10.attn.c_proj.bias', 'gpt.h.10.ln_1.weight', 'gpt.h.10.ln_1.bias', 'gpt.h.10.mlp.c_fc.weight', 'gpt.h.10.mlp.c_fc.bias', 'gpt.h.10.mlp.c_proj.weight', 'gpt.h.10.mlp.c_proj.bias', 'gpt.h.10.ln_2.weight', 'gpt.h.10.ln_2.bias', 'gpt.h.11.attn.bias', 'gpt.h.11.attn.c_attn.weight', 'gpt.h.11.attn.c_attn.bias', 'gpt.h.11.attn.c_proj.weight', 'gpt.h.11.attn.c_proj.bias', 'gpt.h.11.ln_1.weight', 'gpt.h.11.ln_1.bias', 'gpt.h.11.mlp.c_fc.weight', 'gpt.h.11.mlp.c_fc.bias', 'gpt.h.11.mlp.c_proj.weight', 'gpt.h.11.mlp.c_proj.bias', 'gpt.h.11.ln_2.weight', 'gpt.h.11.ln_2.bias', 'classifier.weight', 'classifier.bias']
I0612 13:58:30.055731 4472821184 modeling_utils.py:460] Weights from pretrained model not used in GPTClassifier: ['tokens_embed.weight', 'positions_embed.weight', 'h.0.attn.bias', 'h.0.attn.c_attn.weight', 'h.0.attn.c_attn.bias', 'h.0.attn.c_proj.weight', 'h.0.attn.c_proj.bias', 'h.0.ln_1.weight', 'h.0.ln_1.bias', 'h.0.mlp.c_fc.weight', 'h.0.mlp.c_fc.bias', 'h.0.mlp.c_proj.weight', 'h.0.mlp.c_proj.bias', 'h.0.ln_2.weight', 'h.0.ln_2.bias', 'h.1.attn.bias', 'h.1.attn.c_attn.weight', 'h.1.attn.c_attn.bias', 'h.1.attn.c_proj.weight', 'h.1.attn.c_proj.bias', 'h.1.ln_1.weight', 'h.1.ln_1.bias', 'h.1.mlp.c_fc.weight', 'h.1.mlp.c_fc.bias', 'h.1.mlp.c_proj.weight', 'h.1.mlp.c_proj.bias', 'h.1.ln_2.weight', 'h.1.ln_2.bias', 'h.2.attn.bias', 'h.2.attn.c_attn.weight', 'h.2.attn.c_attn.bias', 'h.2.attn.c_proj.weight', 'h.2.attn.c_proj.bias', 'h.2.ln_1.weight', 'h.2.ln_1.bias', 'h.2.mlp.c_fc.weight', 'h.2.mlp.c_fc.bias', 'h.2.mlp.c_proj.weight', 'h.2.mlp.c_proj.bias', 'h.2.ln_2.weight', 'h.2.ln_2.bias', 'h.3.attn.bias', 'h.3.attn.c_attn.weight', 'h.3.attn.c_attn.bias', 'h.3.attn.c_proj.weight', 'h.3.attn.c_proj.bias', 'h.3.ln_1.weight', 'h.3.ln_1.bias', 'h.3.mlp.c_fc.weight', 'h.3.mlp.c_fc.bias', 'h.3.mlp.c_proj.weight', 'h.3.mlp.c_proj.bias', 'h.3.ln_2.weight', 'h.3.ln_2.bias', 'h.4.attn.bias', 'h.4.attn.c_attn.weight', 'h.4.attn.c_attn.bias', 'h.4.attn.c_proj.weight', 'h.4.attn.c_proj.bias', 'h.4.ln_1.weight', 'h.4.ln_1.bias', 'h.4.mlp.c_fc.weight', 'h.4.mlp.c_fc.bias', 'h.4.mlp.c_proj.weight', 'h.4.mlp.c_proj.bias', 'h.4.ln_2.weight', 'h.4.ln_2.bias', 'h.5.attn.bias', 'h.5.attn.c_attn.weight', 'h.5.attn.c_attn.bias', 'h.5.attn.c_proj.weight', 'h.5.attn.c_proj.bias', 'h.5.ln_1.weight', 'h.5.ln_1.bias', 'h.5.mlp.c_fc.weight', 'h.5.mlp.c_fc.bias', 'h.5.mlp.c_proj.weight', 'h.5.mlp.c_proj.bias', 'h.5.ln_2.weight', 'h.5.ln_2.bias', 'h.6.attn.bias', 'h.6.attn.c_attn.weight', 'h.6.attn.c_attn.bias', 'h.6.attn.c_proj.weight', 'h.6.attn.c_proj.bias', 'h.6.ln_1.weight', 'h.6.ln_1.bias', 'h.6.mlp.c_fc.weight', 'h.6.mlp.c_fc.bias', 'h.6.mlp.c_proj.weight', 'h.6.mlp.c_proj.bias', 'h.6.ln_2.weight', 'h.6.ln_2.bias', 'h.7.attn.bias', 'h.7.attn.c_attn.weight', 'h.7.attn.c_attn.bias', 'h.7.attn.c_proj.weight', 'h.7.attn.c_proj.bias', 'h.7.ln_1.weight', 'h.7.ln_1.bias', 'h.7.mlp.c_fc.weight', 'h.7.mlp.c_fc.bias', 'h.7.mlp.c_proj.weight', 'h.7.mlp.c_proj.bias', 'h.7.ln_2.weight', 'h.7.ln_2.bias', 'h.8.attn.bias', 'h.8.attn.c_attn.weight', 'h.8.attn.c_attn.bias', 'h.8.attn.c_proj.weight', 'h.8.attn.c_proj.bias', 'h.8.ln_1.weight', 'h.8.ln_1.bias', 'h.8.mlp.c_fc.weight', 'h.8.mlp.c_fc.bias', 'h.8.mlp.c_proj.weight', 'h.8.mlp.c_proj.bias', 'h.8.ln_2.weight', 'h.8.ln_2.bias', 'h.9.attn.bias', 'h.9.attn.c_attn.weight', 'h.9.attn.c_attn.bias', 'h.9.attn.c_proj.weight', 'h.9.attn.c_proj.bias', 'h.9.ln_1.weight', 'h.9.ln_1.bias', 'h.9.mlp.c_fc.weight', 'h.9.mlp.c_fc.bias', 'h.9.mlp.c_proj.weight', 'h.9.mlp.c_proj.bias', 'h.9.ln_2.weight', 'h.9.ln_2.bias', 'h.10.attn.bias', 'h.10.attn.c_attn.weight', 'h.10.attn.c_attn.bias', 'h.10.attn.c_proj.weight', 'h.10.attn.c_proj.bias', 'h.10.ln_1.weight', 'h.10.ln_1.bias', 'h.10.mlp.c_fc.weight', 'h.10.mlp.c_fc.bias', 'h.10.mlp.c_proj.weight', 'h.10.mlp.c_proj.bias', 'h.10.ln_2.weight', 'h.10.ln_2.bias', 'h.11.attn.bias', 'h.11.attn.c_attn.weight', 'h.11.attn.c_attn.bias', 'h.11.attn.c_proj.weight', 'h.11.attn.c_proj.bias', 'h.11.ln_1.weight', 'h.11.ln_1.bias', 'h.11.mlp.c_fc.weight', 'h.11.mlp.c_fc.bias', 'h.11.mlp.c_proj.weight', 'h.11.mlp.c_proj.bias', 'h.11.ln_2.weight', 'h.11.ln_2.bias']`


Cannot load optimizer and lr_scheduler states with TPU training

# 🐛 Bug
When restarting training and loading the optimizer.pt and scheduler.pt, the training crashes as the existing code does not know how to load it with TPU.
## Information

The stacktrace -
```
Exception in device=TPU:5: don't know how to restore data location of torch.FloatStorage (tagged with xla:0)
Traceback (most recent call last):
  File "/home/saurabh/venv/lib/python3.6/site-packages/torch_xla/distributed/xla_multiprocessing.py", line 231, in _start_fn
    fn(gindex, *args)
  File "/home/saurabh/<retracted>", line 334, in _mp_fn
    main()
  File "/home/saurabh/<retracted>", line 303, in main
    trainer.train(model_path=model_path)
  File "/home/saurabh/venv/lib/python3.6/site-packages/transformers/trainer.py", line 386, in train
    torch.load(os.path.join(model_path, "optimizer.pt"), map_location=self.args.device)
  File "/home/saurabh/venv/lib/python3.6/site-packages/torch/serialization.py", line 584, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File "/home/saurabh/venv/lib/python3.6/site-packages/torch/serialization.py", line 764, in _legacy_load
    result = unpickler.load()
  File "/home/saurabh/venv/lib/python3.6/site-packages/torch/serialization.py", line 720, in persistent_load
    deserialized_objects[root_key] = restore_location(obj, location)
  File "/home/saurabh/venv/lib/python3.6/site-packages/torch/serialization.py", line 802, in restore_location
    return default_restore_location(storage, str(map_location))
  File "/home/saurabh/venv/lib/python3.6/site-packages/torch/serialization.py", line 179, in default_restore_location
    + location + ")")
RuntimeError: don't know how to restore data location of torch.FloatStorage (tagged with xla:0)
```
This happens when loading a partially trained model.

A reference implementation is this
https://github.com/pytorch-tpu/fairseq/blob/tpu/fairseq/trainer.py#L195
With a discussion here https://github.com/pytorch/xla/issues/1343
Model I am using (Bert, XLNet ...): any model

Language I am using the model on (English, Chinese ...):

The problem arises when using:
* [x] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Train any model on TPU, wait for a checkpoint to happen
2. move the tokenizer files to the checkpoint dir (another bug, where the trainer expects the tokenizer configs to be present at the same directory as checkpoint dir, that only happens at the very end of training, not at one of the earlier checkpoints)
3. Restart training again from the checkpoint on TPU

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior
Trainer loads the optimizer and scheduler  to TPU and starts training.
<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.11.0 (master)
- Platform: Linux-5.3.0-1026-gcp-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.6.0a0+6bdfd6a (False)
- Tensorflow version (GPU?): 2.2.0 (False)
- Using GPU in script?: False
- Using distributed or parallel set-up in script?: yes, 8 way with xla_spawn.py


How to implement differential learning rates and still ensure "weight_decay" = 0 for the the parameters it should?

I notice that most models are trained with the following parameter groups:
```
no_decay = ["bias", "LayerNorm.weight"]

optimizer_grouped_parameters = [
    {
        "params": [p for n, p in model.named_parameters() if not any(nd in n for nd in no_decay)],
        "weight_decay": 1e-5,
    },
    {
        "params": [p for n, p in model.named_parameters() if any(nd in n for nd in no_decay)],
        "weight_decay": 0.0,
    },
]
optimizer = AdamW(optimizer_grouped_parameters, lr=5e-3, eps=1e-9)
```

**Questions:**
1. Why do we set `weight_decay` = 0.0 for the `no_decay` named parameters?

2. Assuming we need to set `weight_decay` = 0.0 as such, how would we extend the above to allow for differential learning rates (which may be especially helpful for models that include both an encoder and decoder stack)?

3. Have there been any tests to demonstrate the relative effectiveness of differential learning rates for the various transformer models?


Extending Encoder Decoder to GPT-2

Adding GPT2 initialization for EncoderDecoder model as pointed out in the issue below.

> Currently, only Bert works as a decoder. We might add GPT2 in a couple of weeks. Note that no model has `cross-attention` layers if it is not already an encoder-decoder model (like Bart or T5) and in this case it does not make sense to use the encoder-decoder wrapper. The model is initialized with random weights for the cross attention layers which will have to be fine-tuned. I agree, that this should be made clearer in the documentation!

_Originally posted by @patrickvonplaten in https://github.com/huggingface/transformers/issues/4517#issuecomment-638058577_


unexpected keyword argument 'lm_labels' when using BertModel as Decoder with EncoderDecoderModel

The `BertModel.forward()` method does not expect a `lm_labels` and `masked_lm_labels` arguments. Yet, it looks like the `EncoderDecoderModel.forward()` method calls it's decoder's `forward()` method with those arguments which throws a TypeError when a BertModel is used as a decoder.

Am I using the BertModel incorrectly? I can get rid of the error by modifying the EncoderDecoderModel to not use those arguments for the decoder.

Exact Error:

```
File "/Users/utkarsh/anaconda3/envs/py37/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/Users/utkarsh/Projects/ai4code/transformers/bert2bert/models.py", line 12, in forward
    dec_out, dec_cls, enc_out, enc_cls = self.bertmodel(input_ids=inputs, attention_mask=input_masks, decoder_input_ids=targets, decoder_attention_mask=target_masks)
  File "/Users/utkarsh/anaconda3/envs/py37/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/Users/utkarsh/anaconda3/envs/py37/lib/python3.7/site-packages/transformers/modeling_encoder_decoder.py", line 283, in forward
    **kwargs_decoder,
  File "/Users/utkarsh/anaconda3/envs/py37/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
TypeError: forward() got an unexpected keyword argument 'lm_labels'
```

Relevant part of the code:
```
encoder = BertModel(enc_config)
dec_config = BertConfig(...,is_decoder=True)
decoder = BertModel(dec_config)
model = EncoderDecoderModel(encoder=encoder, decoder=decoder)
```
...
`dec_out, dec_cls, enc_out, enc_cls = model(input_ids=inputs, attention_mask=input_masks, decoder_input_ids=targets, decoder_attention_mask=target_masks)`


Add AlbertForMultipleChoice

Cleaning up the rebase and opening a fresh PR.


Issue with an inline code comment

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...):

Language I am using the model on (English, Chinese ...):

The problem arises when using:
* [x ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ x] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

https://github.com/huggingface/transformers/blob/d4c2cb402d6674211726fd5f4803d1090664e438/examples/question-answering/run_squad.py#L322
This specific comment written is wrong.

I wasted lot of time due to this comment.
i and feature_index are not the same no

:)

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- Platform:
- Python version:
- PyTorch version (GPU?):
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:


Memory leakage with bert-large-uncased-whole-word-masking-finetuned-squad

# 🐛 Bug

## Information
I am using 'bert-large-uncased-whole-word-masking-finetuned-squad'  and observed an memory leak  during the inference. Below the code snippet to reproduce it

tokenizer = AutoTokenizer.from_pretrained("bert-large-uncased-whole-word-masking-finetuned-squad")
model = AutoModelForQuestionAnswering.from_pretrained("bert-large-uncased-whole-word-masking-finetuned-squad")

device = 'cuda'
question = 'what is your question'
answer = 'This is the  answer'

 def tokenize_qa(tokenizer,question, answer_context):
    input_ids =  tokenizer.encode(question, answer_context, max_length=512)
    tokens =  tokenizer.convert_ids_to_tokens(input_ids)
    sep_index = input_ids.index( tokenizer.sep_token_id)
    num_seg_a = sep_index + 1
    num_seg_b = len(input_ids) - num_seg_a
    segment_ids = [0] * num_seg_a + [1] * num_seg_b
    assert len(segment_ids) == len(input_ids)
    return input_ids, segment_ids, tokens

###################### Inference Code  ##########################
input_ids, segment_ids, tokens= tokenize_qa(tokenizer,question,answer_context)
start_scores, end_scores = model(torch.tensor([input_ids], device= device),token_type_ids=torch.tensor([segment_ids], device= device))

When i run the inference code multiple times it accumulates large  gpu memory and finally runs OOM. Please check let me know if anything wrong here.

Thank you.

Language I am using the model : English,

- `transformers` version: '2.11.0'
- Platform: Ubuntu 18.04
- Python version:3.6.9
- PyTorch version (GPU): 1.4.0


RobertaForMaskedLM Failing for example code given on Hugging Face' documentation page

# 🐛 Bug

from transformers import RobertaTokenizer, RobertaForMaskedLM
import torch

tokenizer = RobertaTokenizer.from_pretrained('roberta-base')
model = RobertaForMaskedLM.from_pretrained('roberta-base')
input_ids = torch.tensor(tokenizer.encode("Hello, my dog is cute", add_special_tokens=True)).unsqueeze(0)  # Batch size 1
outputs = model(input_ids, labels=input_ids)
loss, prediction_scores = outputs[:2]

I am using this example code given on hugging face's documentation page , but this also gives an error
TypeError: forward() got an unexpected keyword argument 'labels'

The doc clearly states that forward accepts labels ,what's wrong here?


How to use fine-tuned BERT to fill <mask>

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
I have fine-tuned a BERT model by classification task, use transformers.BertForSequenceClassification. Now,  i  want use this model to fill mask when i give a input like: 'my dog \<mask\> beautiful'.

Can this be implemented?
I would really appreciate if someone can teach me how to implement.


ElectraForMultipleChoice

This PR add `ElectraForMultipleChoice`. One of the missing models in this [project](https://github.com/huggingface/transformers/projects/17).

Since, for multiple choice pooled outputs are needed, added `ElectraPooler` class.

@sgugger , @LysandreJik


Added feature to move added tokens in vocabulary for Transformer-XL

As discussed in #3554 the tokens in the tokenizer have to be shifted if adding a new token into aka resizing an embedding layer other than the last one. Of course this applies only for an `AdaptiveEmbedding` with more than one layer.

This implementation adds a function to move an added token in the tokenizer to a specific position.

This is closely related to the PR #4759


Data used for training MarianMT models

# ❓ Questions & Help
Are there specifications available on the amount and type of training data used for the released MarianMT [models](https://huggingface.co/transformers/model_doc/marian.html)?

## Details
<!-- Description of your issue -->
I want to use these models for back translation in order to augment my current training data (the [VQA dataset](http://visualqa.org/) which is in English). I understand that for this, I could use any number of models in conjunction with source and target languages as `en`.

However, I am concerned about the domain mismatch of the VQA dataset vs training data of the MT model. For example, using `en-ROMANCE` and `ROMANCE-en` models, I find that back-translation of the input sentence:

`['>>lmo<< What do you think are the children playing with?']` is

`['What do you think of the youth of the jubilee?']`

Also, do you have any suggestions/guidelines on which/how many models to use particularly for this use case? I did not find this question suitable for SO hence it posted here directly

Thank you for releasing the models!


[examples] SummarizationModule improvements

Examples

Which is related to examples in general

This PR makes the SummarizationTrainer much more usable, and when improvements are not unique to summarization, they are implemented in `lightning_base.py` instead.

- **Checkpointing** Before this PR, the code saves 5GB of PL checkpoints per epoch, now SummarizationTrainer saves the best checkpoint based on ROUGE 2 score, and also saves it in huggingface `save_pretrained` format using the `on_save_checkpoint`. This will help resolve lots of confusion in various issues about how to load the pl checkpoints.

The current summarization code can only accept bs=1 and takes 24h to run 1 epoch on CNN DM. With the following changes, you can train much faster, if you wish. The docs suggested that larger batch sizes were possible with default params, which is fixed.

### Changes to Allow Faster Summarization Training
*these are all optional and turned off by default*

1) freezing: before this PR, it was basically only possible to finetune with batchsize 2-4 on a 16GB system. With `--freeze_embeds` and `--freeze_encoder`, you can get batch size MUCH higher, towards 32. I've seen strong results with these options.

2) On CNNDM and XSUM the datasets are 200K examples, and epochs are very long. For this reason it is preferable to run validation (and get a rouge score) more frequently, but with previous params each `validation_step` took 1hr. By passing `--n_val=1000 --val_check_interval=0.25`, you can run validation 4x per epoch and it only takes 3 minutes. I also allows the config's beam search parameters to be used, rather than hardcoding faster but lower scoring ones.

3) `{train|val|test}_max_target_length`: I have found it preferable to truncate train summaries to 56 for XSUM and CNNDM respectively, but doing this for val/test artificially inflates rouge scores. So these clargs are separated.

Changes to `lightning_base`
- Number of trainable parameters and total parameters are logged by default.
- All possible `pl.Trainer` clargs are passed through `add_generic_args` (Inspired by @nateraw)

### WandbLogger
- `--logger wandb` will instantiate a default wandb logger.
- `--logger wandb_shared` will post results to [here](https://app.wandb.ai/sshleifer/hf_summarization/table?workspace=user-), so that the community can compare hyperparameter settings empirically.
- the default logger is still tensorboard logger because it doesn't require making an account.

### Distillation
- `SummarizationDistiller` and `T5SummarizationDistiller` are checked in. This code was sent to me by a researcher who wishes to remain anonymous. DM to discuss.


GPTDoubleHeadsModel Unexpected node type: onnx:: Sub

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): GPT2DoubleHeadsModel

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:
I've been following the ipython notebook provided [here](https://github.com/huggingface/transformers/blob/master/notebooks/04-onnx-export.ipynb)

1. Take an off-the-shelf pretrained `gpt` model and export to onnx format using the following script:
```
import torch
from transformers import (GPT2Config, GPT2Model, GPT2Tokenizer)

# use_cache is True by default in GPT2Model. Here we wrap a class to disable past state output.
class GPT2DoubleHeadsModelNoPastState(GPT2DoubleHeadsModel):
    def __init__(self, config):
        super().__init__(config)

    def forward(self, input_ids, token_type_ids):
        return super().forward(input_ids, past=None, attention_mask=None, token_type_ids=token_type_ids, use_cache=False)

model_name="gpt2"
config = GPT2Config.from_pretrained(model_name)
tokenizer = GPT2Tokenizer.from_pretrained(model_name)
model = GPT2ModelNoPastState.from_pretrained(model_name)

example_inputs = tokenizer.encode_plus("This is a sample input", return_tensors="pt")
del example_inputs["attention_mask"]
example_outputs = model(**example_inputs)

input_names = ['input_ids',  'token_type_ids']
output_names=['output_1', 'output_2']
dynamic_axes={'input_ids': {0: 'batch_size', 1: 'num_choices', 2: 'seq_len'},
     'token_type_ids': {0: 'batch_size', 1: 'num_choices', 2: 'seq_len'},
     'output_1': {0: 'batch_size', 1: 'num_choices': 2: 'seq_len', 3: 'vocab_size'},
     'output_2': {0: 'batch_size', 1: 'num_choices'}
}
output_path='gpt2.onnx'
torch.onnx.export(model=model,
                  args=(example_inputs[input_names[0]].unsqueeze(0), example_inputs[input_names[1]].unsqueeze(0)),
                  f=output_path,
                  input_names=input_names,
                  output_names=output_names,
                  example_outputs=example_outputs,
                  dynamic_axes=dynamic_axes,
                  do_constant_folding=True,
                  opset_version=11,
                  use_external_data_format=False)
```

This script is based off of #4805

2.

After invoking the above, I get the error:
```
.../torch/onnx/symbolic_helper.py", line 87...
RuntimeError: Unexpected node type: onnx::Sub
```

## Expected behavior

I would expect this to work successfully, and unfortunately I'm not exactly sure how to interpret this error. There's not a lot of documentation online.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: Commit 0e1869cc286d607f1598506be7bd1312b76ca82c
- Onnxruntime: 1.3.0
- Python version: 3.6.10
- PyTorch version (GPU?): 1.5.0+cu101
- Using GPU in script?: Yes

Thanks for your help! @mfuntowicz @tianleiwu


[mbart] Fix fp16 testing logic

the expected logits must be in the same dtype as the resulting logits.


[wip] Send slack message if self-scheduled runner fails


Trainer.evaluate does not support seq2seq models

# 🐛 Bug

## Information

Hi! I can't thank you enough for Transformers. I know that the Trainer is still under development, but would like to report this just to know the current status.

Currently `Trainer._prediction_loop` assumes that different batches of data have the same shape.
Specifically, [this line](https://github.com/huggingface/transformers/blob/473808da0d476792070f0e7dfebcf1121a12a34f/src/transformers/trainer.py#L786)

```python
preds = torch.cat((preds, logits.detach()), dim=0)
```

This does not allow to use Trainer.evaluate for models with a variable output (e.g. seq2seq models). One of the possible solutions is to pad all batches to the same length, but it is pretty inefficient.

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. create seq2seq model
2. pad batches in such a way that each batch is padded to the maximum length within batch
3. create Trainer for the model, call .evaluate()

```
Traceback (most recent call last):
  File "/home/vlialin/miniconda3/lib/python3.7/site-packages/transformers/trainer.py", line 509, in train
    self.evaluate()
  File "/home/vlialin/miniconda3/lib/python3.7/site-packages/transformers/trainer.py", line 696, in evaluate
    output = self._prediction_loop(eval_dataloader, description="Evaluation")
  File "/home/vlialin/miniconda3/lib/python3.7/site-packages/transformers/trainer.py", line 767, in _prediction_loop
    preds = torch.cat((preds, logits.detach()), dim=0)
RuntimeError: Sizes of tensors must match except in dimension 0. Got 29 and 22 in dimension 1
```

## Expected behavior

Trainer is able to evaluate Seq2seq

## Environment info

- `transformers` version: 2.11
- Platform: Linux
- Python version: 3.7.6
- PyTorch version (GPU?): 1.5.0
- Tensorflow version (GPU?): 2.2.0
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


feat(TFTrainer): improve logging

This PR supersedes PR #4756
I wanted initially to refactor logging between `Trainer` and `TFTrainer` but there seems to be too many differences between them for it to make sense (distributed training, logging methods, `wandb.watch`…).

Logging is now handled directly within each Trainer and all comments from previous PR have been applied here.

Notes:

* `TFTrainer`: @jplu There should be the same modifications we discussed. I just moved the check of `global_step` within logging for better readability.
* `TFTrainer._log` does not have a tqdm iterator argument (unlike in `Trainer`) since you're not using it at the moment but it could be useful in the future
* `Trainer`: main update is to handle the case where we do only evaluations (no training)