Added data collator for permutation (XLNet) language modeling and related calls

Added `DataCollatorForPermutationLanguageModeling` in `data/data_collator.py` to return necessary inputs (applies masking and generates revelant tensors input_ids, perm_mask, target_mask and labels as per https://github.com/zihangdai/xlnet/blob/master/data_utils.py) for language modeling training with XLNetLMHeadModel. Also added related arguments, logic and calls in `examples/language-modeling/run_language_modeling.py`. Defined a separate `--plm_probability` flag for its use.

Also looked into CTRL - it uses a CLM loss just like GPT and GPT-2, so should work out of the box with this script (provided `past` is taken care of similar to `mems` for XLNet). Added a few words in the comments to reflect this.

Changed calls and imports appropriately.

Resolves: #4739, #2008 (partially)


What should i do if I want a model class similar to BertForSequenceClassification?

# ❓ Questions & Help

Hi!
For my task I want to implement a class similar to BertForSequenceClassification, but a little different, for example, i may want multiple linear head on top of the bert encoding. Call it BertForMySequenceClassification,

But I still want to use the pretrained bert parameters, and hopefully do easy load/save.
For example, i hope i can BertForSequenceClassification.from_pretrianed(some_path)

Any suggestion or help are much appreciated!
Thanks!


AdamW step device error

I have a model:
```
class M(nn.Module):
  def __init__(self, ...):
    self.A = nn.Parameter(...)
    self.B = nn.Parameter(...)
    self.C = torch.einsum(..., self.A, self.B)

  def forward(self, D):
    return func(self.C.to(D.device), D)
```

However, I'm getting the following error when training this model on GPU:
```
File "file.py", line 71, in optimizer_step
optimizer.step()
File ".../lib/python3.7/site-packages/torch/optim/lr_scheduler.py", line 67, in wrapper
return wrapped(*args, **kwargs)
File ".../lib/python3.7/site-packages/transformers/optimization.py", line 155, in step
exp_avg.mul_(beta1).add_(grad, alpha=1.0 - beta1)
RuntimeError: expected device cuda:0 but got device cpu
```

This is happening in 2.11.0. Am I doing something incorrectly here?


Get prediction_scores from BART forward method

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->

## Details
<!-- Description of your issue -->
I'm trynig to implement a model using finetuining BART for dialouge task. This is my sample code:
```
tokenizer = BartTokenizer.from_pretrained('facebook/bart-large')
model = BartForConditionalGeneration.from_pretrained('facebook/bart-large')
inputs = tokenizer.encode(" Hello, my dog is cute", return_tensors="pt")
decoder_input = tokenizer.encode(" Oh! I don't know that you have dog! How is it?", return_tensors="pt")
output = model(input_ids=inputs,decoder_input_ids=decoder_input)[0]
```
I want to get prediction scores for tokens, so I expected that output would have shape [1, 17, 50265] (The length of the decoder_input string), but it has shape [1, 1, 50265].

How can I get the prediction_scores?


Make T5 compatible with ONNX

This is a small PR to make T5 exportable to ONNX with any op>9. It addresses an issue outlined in #5075 where T5 would not export to ONNX. In order to make it exportable, 2 changes are made:
- A torch.einsum is replaced with a tensor multiplication in 96d0ec7 since onnx does not currently support this notation
- Decoder inputs / embeddings are defaulted to the encoder's inputs / embeddings if they are not declared. I believe this is clearer as most of the examples right now include something along the lines of `model(input_ids=input_ids, decoder_input_ids=input_ids)`. It also allows t5 to be executed with the more common paradigm of calls like model(inputs)


getting different model result from tokenizer vs tokenizer.encode function

Core: Tokenization

Internals of the library; Tokenization.

# 🐛 Bug

I am using gpt2 model to encode sentences. I am confused between tokenizer.encode vs tokenizer :

If I am using tokenizer.encode:

```

sentence = 'checking single sentences'

from transformers import GPT2Tokenizer, GPT2Model
import torch

tokenizer = GPT2Tokenizer.from_pretrained('gpt2-medium')
model     = GPT2Model.from_pretrained('gpt2-medium')
input_ids = torch.tensor(tokenizer.encode(sentence)).unsqueeze(0)  # Batch size 1
outputs   = model(input_ids)
outputs[0]
```

output is  :

```
tensor([[[ 0.6804,  0.4182,  0.3483,  ..., -0.3102,  0.0341,  0.4901],
         [-0.4060,  0.7790,  0.2695,  ..., -0.4763,  0.1817,  0.0600],
         [ 0.7916,  0.6078,  0.4642,  ..., -0.5557, -0.1571, -0.1220]]],
       grad_fn=<ViewBackward>)
```

While using tokenizer alone :

```
from transformers import GPT2Tokenizer, GPT2Model
sentence = 'checking single sentences'

tokenizer = GPT2Tokenizer.from_pretrained('gpt2-medium')
model     = GPT2Model.from_pretrained('gpt2-medium')
inputs    = tokenizer(sentence, return_tensors="pt")
outputs   = model(**inputs)
outputs[0]
```

output is :

```
# tensor([[[ 0.5511,  0.4063,  0.2453,  ..., -0.4217,  0.1519,  0.0898],
#          [-0.1138, -0.0232,  0.1736,  ..., -0.5408,  0.0145,  0.2985],
#          [ 0.1856,  0.3127,  0.2430,  ..., -0.7532, -0.2332,  0.1506]]],
#        grad_fn=<ViewBackward>)
```

I tried to examine the output of both tokenizer :

```
from transformers import GPT2Tokenizer, GPT2Model
import torch

sentence = 'checking single sentences'

tokenizer = GPT2Tokenizer.from_pretrained('gpt2-medium')
input_ids = tokenizer.encode(sentence)
inputs    = tokenizer(sentence, return_tensors="pt")

print(input_ids)

print(inputs)
```

output :

```
[41004, 2060, 13439]
{'input_ids': tensor([[41004,  2060, 13439]]), 'token_type_ids': tensor([[0, 0, 0]]), 'attention_mask': tensor([[1, 1, 1]])}
```

Which is recommended tokenizer function?


Addition of a DialoguePipeline

- Addition of a Conversation object to keep track of multi-turn conversation
- Creation of a DialoguePipeline to process Conversations using the history context
- Integration tests for DialoguePipeline using `microsoft/DialoGPT-medium`


Create model card

model card

Related to pretrained model cards


added model card for ukr-roberta-base

Related to pretrained model cards


Fail in some tests (with detailed description)

# ❓ Questions & Help
I have followed the README to install transformers.
However, there are some failures in the test. (mainly about multigpu_data_paralle_forward)
```
======================================================================================================================== short test summary info =========================================================================================================================
FAILED tests/test_modeling_electra.py::ElectraModelTest::test_multigpu_data_parallel_forward - RuntimeError: tensor.ndimension() == static_cast<int64_t>(expected_size.size()) INTERNAL ASSERT FAILED at /pytorch/torch/csrc/cuda/comm.cpp:225, please report a bug to ...
FAILED tests/test_modeling_gpt2.py::GPT2ModelTest::test_multigpu_data_parallel_forward - RuntimeError: Gather got an input of invalid size: got [2, 2, 4, 7, 8], but expected [2, 4, 4, 7, 8] (gather at /pytorch/torch/csrc/cuda/comm.cpp:231)
FAILED tests/test_modeling_bart.py::BartHeadTests::test_lm_forward - timeout_decorator.timeout_decorator.TimeoutError: 'Timed Out'
FAILED tests/test_modeling_ctrl.py::CTRLModelTest::test_multigpu_data_parallel_forward - RuntimeError: CUDA error: out of memory
FAILED tests/test_modeling_xlnet.py::XLNetModelTest::test_multigpu_data_parallel_forward - RuntimeError: CUDA error: out of memory
FAILED tests/test_modeling_openai.py::OpenAIGPTModelTest::test_multigpu_data_parallel_forward - RuntimeError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 15.78 GiB total capacity; 5.33 MiB already allocated; 17.19 MiB free; 6.00 MiB reserved in total ...
FAILED tests/test_modeling_mobilebert.py::MobileBertModelTest::test_multigpu_data_parallel_forward - RuntimeError: CUDA error: out of memory
================================================================================================= 7 failed, 1066 passed, 562 skipped, 1396 warnings in 333.27s (0:05:33) =================================================================================================
```

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->

## Details
<!-- Description of your issue -->
I have tried many environments:
- conda: python3.7 + torch1.5.1 + cuda10.2
- venv: python3.7 + torch1.5.1 + cuda10.2
- conda: python3.6 + torch1.4.0 + cuda10.1
- venv: python3.6 + torch1.4.0 + cuda10.1
- conda: python3.7 + torch1.5.1 + cuda10.2 + apex
- venv: python3.6 + torch1.4.0 + cuda10.1 + apex

And I installed `transformers` **from source**.

All of them failed in 5-7 tests.
The `short test summary info` outputs of them are not all the same.
But **all** of them are related to `test_multigpu_data_parallel_forward`.

One of the outputs of `transformers-cli env` is as the following
(with the setting: venv: python3.6 + torch1.4.0 + cuda10.1 + apex):
```
The current process just got forked. Disabling parallelism to avoid deadlocks...
To disable this warning, please explicitly set TOKENIZERS_PARALLELISM=(true | false)

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- `transformers` version: 3.0.1
- Platform: Linux-4.15.0-1067-azure-x86_64-with-Ubuntu-16.04-xenial
- Python version: 3.6.11
- PyTorch version (GPU?): 1.4.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
```
For my case (just fill out the two last points),
```
- `transformers` version: 3.0.1
- Platform: Linux-4.15.0-1067-azure-x86_64-with-Ubuntu-16.04-xenial
- Python version: 3.6.11
- PyTorch version (GPU?): 1.4.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: yes
```
For this case, the `short test summary info` shows:
```
======================================================================================================================== short test summary info =========================================================================================================================
FAILED tests/test_modeling_electra.py::ElectraModelTest::test_multigpu_data_parallel_forward - RuntimeError: tensor.ndimension() == static_cast<int64_t>(expected_size.size()) INTERNAL ASSERT FAILED at /pytorch/torch/csrc/cuda/comm.cpp:225, please report a bug to ...
FAILED tests/test_modeling_gpt2.py::GPT2ModelTest::test_multigpu_data_parallel_forward - RuntimeError: Gather got an input of invalid size: got [2, 2, 4, 7, 8], but expected [2, 4, 4, 7, 8] (gather at /pytorch/torch/csrc/cuda/comm.cpp:231)
FAILED tests/test_modeling_bart.py::BartHeadTests::test_lm_forward - timeout_decorator.timeout_decorator.TimeoutError: 'Timed Out'
FAILED tests/test_modeling_ctrl.py::CTRLModelTest::test_multigpu_data_parallel_forward - RuntimeError: CUDA error: out of memory
FAILED tests/test_modeling_xlnet.py::XLNetModelTest::test_multigpu_data_parallel_forward - RuntimeError: CUDA error: out of memory
FAILED tests/test_modeling_openai.py::OpenAIGPTModelTest::test_multigpu_data_parallel_forward - RuntimeError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 15.78 GiB total capacity; 5.33 MiB already allocated; 17.19 MiB free; 6.00 MiB reserved in total ...
FAILED tests/test_modeling_mobilebert.py::MobileBertModelTest::test_multigpu_data_parallel_forward - RuntimeError: CUDA error: out of memory
================================================================================================= 7 failed, 1066 passed, 562 skipped, 1396 warnings in 333.27s (0:05:33) =================================================================================================
```
Some failures are about CUDA memory.
Seems weird, especially the following one.
```
FAILED tests/test_modeling_openai.py::OpenAIGPTModelTest::test_multigpu_data_parallel_forward - RuntimeError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 15.78 GiB total capacity; 5.33 MiB already allocated; 17.19 MiB free; 6.00 MiB reserved in total ...
```
So I checked the output of `nvidia-smi` **before and after** the test to make sure that **no other** processes hold memory.
```
Sat Jul  4 08:49:02 2020
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 440.33.01    Driver Version: 440.33.01    CUDA Version: 10.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Tesla V100-PCIE...  Off  | 00000217:00:00.0 Off |                    0 |
| N/A   30C    P0    37W / 250W |      0MiB / 16160MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   1  Tesla V100-PCIE...  Off  | 00001C95:00:00.0 Off |                    0 |
| N/A   30C    P0    36W / 250W |      0MiB / 16160MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   2  Tesla V100-PCIE...  Off  | 0000735E:00:00.0 Off |                    0 |
| N/A   31C    P0    39W / 250W |      0MiB / 16160MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   3  Tesla V100-PCIE...  Off  | 0000AC50:00:00.0 Off |                    0 |
| N/A   31C    P0    36W / 250W |      0MiB / 16160MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

```

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->

The **details of the failures** are too long to post here.
Please refer to [this page](https://github.com/xx-zhou16/tmp_issue/blob/master/README.md) for details.

## Related
I find a similar issue. #5070
I think the only difference between his environment and mine is the version of transformers.
The failures of his and mine are both about `test_multigpu_data_parallel_forward`.
But #5070 has not been solved.

----
If you need me to provide other details, please reply to this issue. I will reply ASAP.
I am focusing on this issue recently.
I would be very glad if anyone can help solve this problem.


Allow tests in examples to use cuda or fp16,if they are available

The tests in examples didn't use the Cuda or fp16 even if they were available.

- The text classification example (`run_glue.py`) didn't use the fp16 even if it was available but
  the device was take based on the availability(Cuda/CPU).
- The language-modeling example (`run_language_modeling.py`) was having `--no_cuda` argument which
  made the test work without Cuda. This example is having an issue when running with fp16
  thus it not enabled (got an assertion error for perplexity due to its higher value).
- The Cuda and fp16 is not enabled for question-answering example (`run_squad.py`) as it is having a
  the difference in the f1 score.
- The text-generation example (`run_generation.py`) will take the Cuda or fp16 whenever it is available.

Resolves some of #5057


example code missing `encode`

following example, and realized by running
```
In [27]:     encoding = tokenizer(text_batch, return_tensors='pt', padding=True, truncation=True)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-27-c8731cd264f2> in <module>
----> 1 encoding = tokenizer(text_batch, return_tensors='pt', padding=True, truncation=True)

TypeError: 'BertTokenizer' object is not callable
```


Fix typo in training

Fixes a small typo in the code example.

Ref code: https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_bert.py#L1211


TPU Trainer memory leak and memory requirements

# 🐛 Bug
The TPU Trainer for Pytorch is extremely memory inefficient as it eats up lot of CPU memory while training. Moreover the training pipeline has a memory leak somewhere that leads to increased memory utilization as the training proceeds.

## Information
I am using a huge machine (n1-highmem-32) with 208 GB of RAM. I am trying to train seq-seq models like T5-base and BART-base with 880,000 examples. The problem is that with xla_spawn.py method of training models, multiprocessing starts 8 processes that all load up the features separately, resulting in 8x the memory usage than if they all shared the memory. Also this scaling of memory restricts what TPU one can use, I cannot imagine having to use a 1 TB RAM machine just to use v2-32 instance with 32 processes! I am trying to productionize a training pipeline with TPUs and the memory scaling problem makes it really hard and costly. TPUs are designed for training really large models and with large datasets but this memory issue is a large bottleneck against it. I could train with larger datasets but the memory usage prevents me.

In the images below one can see that for BART training, the memory utilization at the beginning is 80% but it rises to 100% in 14 hours, killing the training. I have to train the model 3 times to make progress.
![image](https://user-images.githubusercontent.com/1271289/86500284-e1169200-bd44-11ea-877f-337d5854c393.png)
![image](https://user-images.githubusercontent.com/1271289/86500267-c17f6980-bd44-11ea-8083-1949883883d9.png)
This is the memory usage with T5. It starts with 45% and ends with 75%.
![image](https://user-images.githubusercontent.com/1271289/86500290-f1c70800-bd44-11ea-8aac-8cc182f750a0.png)

Could you consider rearchitecting the TPU Trainer so that it does not takes humongous amounts of RAM? Maybe a streaming or lazy loading architecture that only loads the features as and when they are required? Also multiprocessing is inherently not scalable, does one launch 1024 processes to train on a 1024 Core TPU Pod? I have made a TPU Training pipeline with TensorFlow 1 previously and it worked spectacularly effectively with its architecture of TFRecords and Tensorflow handling all data streaming through Google storage. The pipeline performance scaled linearly with the number of TPU Cores without consuming large memory. (Image attached below).

![image](https://user-images.githubusercontent.com/1271289/86500867-12915c80-bd49-11ea-83ef-7cb9271c5022.png)

A scalable pipeline would make Huggingface a real contender for production level TPU PyTorch training pipelines.

Model I am using (Bert, XLNet ...): T5, BART

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

Use the official example at https://github.com/huggingface/transformers/tree/master/examples#running-on-tpus

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

Memory usage contained and does not increase with time.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.11.0
- Platform: Linux-5.3.0-1026-gcp-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.7.0a0+4121d34 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: False
- Using distributed or parallel set-up in script?: Yes


T5 Masking:

While this works, the outputs are not according to what is specified in "model.generate."

```
# tokenizer = T5Tokenizer.from_pretrained(model_name)
# model = T5ForConditionalGeneration.from_pretrained(model_name)
# # Input text

# text = 'This <extra_id_0> sentence. </s>'

# encoded = tokenizer.encode_plus(text, add_special_tokens=True, return_tensors='pt')
# input_ids = encoded['input_ids']

# # Generating 20 sequences with maximum length set to 5
# outputs = model.generate(input_ids=input_ids,
#                           num_beams=200, num_return_sequences=20,
#                           min_length = 5, max_length=10)

# _0_index = text.index('<extra_id_0>')
# _result_prefix = text[:_0_index]
# _result_suffix = text[_0_index+12:]  # 12 is the length of <extra_id_0>

# def _filter(output, end_token='<extra_id_1>'):
#     # The first token is <unk> (inidex at 0) and the second token is <extra_id_0> (indexed at 32099)
#     _txt = tokenizer.decode(output[2:], skip_special_tokens=False, clean_up_tokenization_spaces=False)
#     if end_token in _txt:
#         _end_token_index = _txt.index(end_token)
#         return _result_prefix + _txt[:_end_token_index] + _result_suffix
#     else:
#         return _result_prefix + _txt + _result_suffix

# results = list(map(_filter, outputs))
```

Example Output:

`This perception holds no validity.`

As can be seen, "holds no" is only two words despite that I put the "min_length" as five.


What's the correct way to use add_prefix_space for the fast RoBERTa tokenizer in 3.0.0/3.0.1?

Core: Tokenization

Internals of the library; Tokenization.

The documentation says we should pass this flag to **the encoding methods**.

https://github.com/huggingface/transformers/blob/58cca47c16149e43d1b516623d59e3c5d97f695e/src/transformers/tokenization_roberta.py#L259-L260

However, the encoding methods don't take such an argument, and passing it in actually causes an error:

```
File "test.py", line 127, in func
  self.tokenizer.encode(token, add_special_tokens=False, add_prefix_space=True)
File ".../transformers/tokenization_utils_base.py", line 1425, in encode
  **kwargs,
File ".../transformers/tokenization_utils_base.py", line 1737, in encode_plus
  **kwargs,
File ".../transformers/tokenization_gpt2.py", line 377, in _encode_plus
  return super()._encode_plus(*args, **kwargs)
File ".../transformers/tokenization_utils_fast.py", line 420, in _encode_plus
  **kwargs,
File ".../transformers/tokenization_gpt2.py", line 367, in _batch_encode_plus
  return super()._batch_encode_plus(*args, **kwargs)
File ".../transformers/tokenization_utils_fast.py", line 313, in _batch_encode_plus
  raise ValueError(f"Keyword arguments {kwargs} not recognized.")
ValueError: Keyword arguments {'add_prefix_space': True} not recognized.
```

Then, I assumed that this documentation was a typo and we should pass this argument to `__init__`. But why is it default to `False` in `__init__`? Don't we pretty much always need to add a prefix space? Does `from_pretrained` set it to `True` automatically? If not, should we always do `AutoTokenizer.from_pretrained(..., use_prefix_space=True)`?

https://github.com/huggingface/transformers/blob/58cca47c16149e43d1b516623d59e3c5d97f695e/src/transformers/tokenization_roberta.py#L299-L314


Why is `encoder_extended_attention_mask = None` when `config.is_decoder == False`

Potential Bug(?)

Reading the codebase I see that attention masks are ignored for many of the pretrained model configs such as `'bert-base-uncased'`. We can see [here](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_bert.py#L743) that the attention mask is simply cleared out. Is this intentional?

```
from transformers import BertModel
config_path = 'bert-base-uncased'
config = BertModel.config_class.from_pretrained(config_path)
print(f'is_decoder: {config.is_decoder}')
```

outputs False


3.0.1 BertTokenizer batch_encode_plus() shows warnings "Truncation was not explicitely activated but `max_length` is provided a specific value"

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): I'm using Transformers 3.0.1 and the BERT model to do two-sentence NLI-style classification.

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

I have a question and two issues:

1. What happened to the `BertTokenizer.encode_plus()` and `BertTokenizer.batch_encode_plus()` methods? I see there must have been a change somewhere to remove them in Transformers 3.0.0, but I cannot find any online change log or other description of what the replacement methods are.

2. This issue I am getting is that there are a lot of repeated messages stating
> Truncation was not explicitely activated but `max_length` is provided a specific value, please use `truncation=True` to explicitely truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.

This issue is apparently the same as the closed issue #5377. However, that one was with respect to `encode_plus ()`, **but my issue is with `batch_encode_plus()`**.

3. Please fix the typo: `Truncation was not explicitely ...` to be `Truncation was not explicitly ...`.

## To reproduce

Steps to reproduce the behavior for `batch_encode_plus()`:

I adapted the code from #5377:

```python
import transformers
print(transformers.__version__)
# 3.0.1

from transformers import BertTokenizer
max_len = 50
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')

# This example works fine with encode_plus().
text = 'Transformers are more than meets the eye'
encoded_dict = tokenizer.encode_plus(text,
                                     text_pair=None,
                                     add_special_tokens=True,
                                     max_length=max_len,
                                     pad_to_max_length=False,
                                     truncation='longest_first')

# This call to batch_encode_plus() shows warnings.
list_of_pair_of_string = []
list_of_pair_of_string.append( ('Transformers are', 'more than meets the eye') )
encoded_dict = tokenizer.batch_encode_plus(batch_text_or_text_pairs=list_of_pair_of_string,
                                           add_special_tokens=True,
                                           max_length=max_len,
                                           pad_to_max_length=False,
                                           truncation='longest_first'
                                           #truncation=True
                                           )
```

Note that for my call to `batch_encode_plus()`, I tried both `truncation='longest_first'` and also `truncation=True`.

However, the call always shows:

> Truncation was not explicitely activated but `max_length` is provided a specific value, please use `truncation=True` to explicitely truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior
<!-- A clear and concise description of what you would expect to happen. -->

The call to `batch_encode_plus()` should not show any warning because I specifically provided the `truncation=` parameter.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.0.1
- Platform: Google Colab
- Python version: 3.6.9 (default, Apr 18 2020, 01:56:04)
- PyTorch version (GPU?): 1.5.1+cu101
- Tensorflow version (GPU?):
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No


Write With Transformers

# 🐛 Bug

## Information

Model: XLNet

Outputs are extremely unusual, and I suspect there is an issue.


T5 Training on TPU doesnt use TPU

I am trying to train T5 on TPU .Training started without errors but its very slow . I think its not using TPU backend.
https://colab.research.google.com/drive/10TN0zgPWCIAzbA0PKYIo3fF8wOvfa1P7?usp=sharing


licens

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...):

Language I am using the model on (English, Chinese ...):

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1.
2.
3.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- Platform:
- Python version:
- PyTorch version (GPU?):
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:


Merge pull request #1 from huggingface/master

Updated


batch_encode_plus model output is different from tokenizer.encode model's output

Internals of the library; Tokenization.

I am trying to encode multiple sentences with BertTokenizer. I tried batch_encode_plus but I am getting different output when I am feeding BertTokenizer's output vs batch_encode_plus's output to model.

```
single_sentence = 'checking single sentences'
sentences = ['checking single sentences', 'many sentences encoding together', 'hello world how and', 'All model checkpoint weights were used when initializing BertModel.']

# loading models
from transformers import BertModel, BertConfig, BertTokenizer
import torch
tokenizer = BertTokenizer.from_pretrained('bert-large-uncased')
model     = BertModel.from_pretrained('bert-large-uncased')

# single encoding and hidden layer weights

def single_query(sentence):
    single_input_id = torch.tensor(tokenizer.encode(sentence)).unsqueeze(0)  # Batch size 1
    outputs         = model(single_input_id)
    features        = outputs[0][:,0,:].detach().numpy()

    return features

def batch_encoding(sentences):

    input_ids   = torch.tensor(tokenizer.batch_encode_plus(sentences, pad_to_max_length=True)['input_ids'])
    outputs     = model(all_f)
    features = outputs[0][:,0,:].detach().numpy()

    return features
```

single_query(single_sentence) return :
```
array([[-0.39814326, -0.4902882 ,  0.02681825, ..., -0.28256905,
        -1.0546892 ,  0.1055279 ]], dtype=float32)

```
while batch_enc = batch_encoding(sentences)[0] return :

```
array([ 0.1909762 ,  0.05580305,  0.221862  , ...,  0.16220105,
       -0.88524836,  0.12994497], dtype=float32)
```

Why there is difference in the model's output?

Is it because of padding?


[ERROR] add_special_tokens = True not working in version 3.0.0

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): BERT

Language I am using the model on (English, Chinese ...): Multi-Lingual

The problem arises when using:
* [x ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:
In version 2.11 Everything is OK but I updated to 3.0.0 and I can not create fixed_length with padding for the encoding of the text

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- Platform:
- Python version:
- PyTorch version (GPU?):
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:


What happened to https://huggingface.co/zero-shot/ ?

Hi, this page was very interesting.
https://huggingface.co/zero-shot/

It is down since yesterday.
What is up with it? Do you plan to bring it up again?


[Generation] better error message

If `cur_len` of input context is as long or longer than `max_length` a nice error message should be shown.


QA pipeline BART compatible


Typo fix in `training` doc

`provides` -> `provided`


The inference speed of gpt2-xl has a gap between pytorch and tensorflow.

**Environment:**
- OS: Ubuntu 18.04
- Python: 3.7.6
- Transformers: 3.0.0
- PyTorch: 1.4.0
- Tensorflow: 2.2.0
- CUDA: 10.1
- CUDNN: 7.6
- GPU: V100

**My code:**
```import time
import torch
import tensorflow as tf
from transformers import AutoTokenizer, AutoModelWithLMHead, TFAutoModelWithLMHead

TIMES = 100

tokenizer = AutoTokenizer.from_pretrained("./gpt2-xl")

# pytorch
model = AutoModelWithLMHead.from_pretrained("./gpt2-xl")
model = model.to("cuda")
input = tokenizer.encode("This is the benchmark of gpt2-xl.", return_tensors="pt").to("cuda")
total = 0
cnt = 0
start = torch.cuda.Event(enable_timing=True)
end = torch.cuda.Event(enable_timing=True)
for i in range(TIMES):
    start.record()
    o = model(input)
    end.record()
    torch.cuda.synchronize()
    if not i:
        continue
    total += start.elapsed_time(end)/1000
    cnt += 1
print("Pytorch version --- cnt: {}, avg_time_cost: {}s".format(cnt, total/cnt))

# tensorflow
gpus = tf.config.experimental.list_logical_devices('GPU')
gpu = gpus[0].name
with tf.device(gpu):
    model = TFAutoModelWithLMHead.from_pretrained("./gpt2-xl")
input = tokenizer.encode("This is the benchmark of gpt2-xl.", return_tensors="tf")
total = 0
cnt = 0
with tf.device(gpu):
    for i in range(TIMES):
        start = time.time()
        o = model(input)
        end = time.time()
        if not i:
            continue
        total += (end-start)
        cnt += 1
print("Tensorflow version --- cnt: {}, avg_time_cost: {}s".format(cnt, total/cnt))
```

**Output:**
```
Pytorch version --- cnt: 99, avg_time_cost: 0.05521844493981564s
Tensorflow version --- cnt: 99, avg_time_cost: 0.2912752628326416s
```

**The utilization of gpu:**
- PyTorch 33%
- Tensorflow 8%

I'm new to Transformers. Did i use it in a wrong way, or any mistakes in my code?
Thanks!


Create README.md

model card

Related to pretrained model cards


Update README.md

Related to pretrained model cards

I set the language


Update README.md

Related to pretrained model cards


[ERROR] Tokenizer and TokenizerFast ???

Core: Tokenization

Internals of the library; Tokenization.

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): BERT

Language I am using the model on (English, Chinese ...): 'bert-base-multilingual-cased'

The problem arises when using:
* [ x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ x] my own task or dataset: (give details below)
## To reproduce

Steps to reproduce the behavior:

1. `from transformers import *`
2. `tokenizer = BertTokenizerFast.from_pretrained('bert-base-multilingual-cased')`
3. `tokenizer.decode(tokenizer.encode('mở bài lạc trôi'))` --> wrong

but:

1. `from transformers import *`
2. `tokenizer = BertTokenizer.from_pretrained('bert-base-multilingual-cased')`
3. `tokenizer.decode(tokenizer.encode('mở bài lạc trôi'))` --> true

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior
the decode sentence after encoding and decoding using TokenizerFast should be true
<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- Platform: Pytorch and TF
- Python version: 3.6
- PyTorch version (GPU?): GPU
- Tensorflow version (GPU?): 2.2
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No


encoder_outputs are always the same when generating with different inputs

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->

## Details
<!-- Description of your issue -->

HI,

I've trained a bert2bert model to generate answers with different questions.
But after training, the bert2bert model always produces the same encoder_outputs with different inputs.
Does anyone know how to fix or avoid the problem?
If I dont resize the bert's embedding size, will this solve the problem?

Thanks in advance.

## The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

## The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## Environment info
- `transformers` version: 2.11.0
- Platform: linux
- Python version: 3.7 64bit
- PyTorch version (GPU?): 1.5.0
- Tensorflow version (GPU?): No
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Using parallel setting only

Below is my training code.
The inputs are turned to indices by tokenizer.encode_plus

```
import logging
import os
import sys
import inspect
import json
import argparse
from dataclasses import dataclass, fields

from tqdm.auto import tqdm, trange
import torch
from torch.utils.data import DataLoader
from transformers import (
    EncoderDecoderModel,
    AdamW,
    get_linear_schedule_with_warmup,
    BertTokenizer,
    PreTrainedModel
)

# import utils
logger = logging.getLogger(__name__)

@dataclass
class training_args:
    weight_decay: float = 0.0
    learning_rate: float = 5e-5
    adam_epsilon: float = 1e-8
    warmup_steps: int = 0
    gradient_accumulation_steps: int = 1
    # num_train_epochs: 10
    max_grad_norm: float = 1.0
    early_stop:  float = 1e-5
    stop_barrier: float = 1e-5

def set_args():
    parser = argparse.ArgumentParser()
    parser.add_argument("--vocab_file", default='vocab_trad_clean.txt')
    # parser.add_argument("--encoder_config", default='Configs/encoder.json')
    # parser.add_argument("--decoder_config", default='Configs/decoder.json')
    parser.add_argument("--data_folder", required=True)
    # parser.add_argument("--output_folder", required=True)
    # parser.add_argument("--from_pretrained", action='store_true')
    parser.add_argument("--logging_steps", default=1000, type=int)
    parser.add_argument("--save_total_limit", default=5, type=int)
    parser.add_argument("--save_steps", default=10000, type=int)
    parser.add_argument("--batch_size", default=20, type=int)
    parser.add_argument("--num_train_epochs", default=30, type=int)
    args = parser.parse_args()
    return args

class Generator_Data(Dataset):
    def __init__(self, data):
        super(Generator_Data, self).__init__()
        self.inputs = []
        self.outputs = []
        for example in data:
            self.inputs.append(example['source'])
            self.outputs.append(example['target'])

    def __len__(self):
        return len(self.inputs)

    def __getitem__(self, index):
        return self.inputs[index], self.outputs[index]

def collate_fn(batch):
    input_dict = {
        "input_ids": [],
        "decoder_input_ids": [],
        "labels": [],
    }
    for data in batch:
        input_data = data[0]
        output_data = data[1]
        input_dict["input_ids"].append(input_data["input_ids"])
        input_dict["decoder_input_ids"].append(output_data["input_ids"])
        input_dict["labels"].append(output_data["input_ids"])
    input_dict = {k: torch.LongTensor(v) for k, v in input_dict.items()}
    return input_dict

def Get_DataLoader(data_file, batch_size, training=False):
    if not os.path.isfile(data_file):
        raise Exception(f"data file [{data_file}] doesn\'t exist in util, LoadDataset")
    logger.info(f"start loading data from {data_file}")
    data = torch.load(data_file)
    dataset = Generator_Data(data)
    logger.info("turn dataset into dataloader")
    if training:
        loader = DataLoader(dataset, batch_size, shuffle=True, collate_fn=collate_fn)
    else:
        loader = DataLoader(dataset, batch_size, shuffle=False, collate_fn=collate_fn)
    return loader

if __name__ == "__main__":
    args = set_args()
    # Setup logging
    logging.basicConfig(
        format="%(asctime)s - %(levelname)s - %(name)s -   %(message)s",
        datefmt="%m/%d/%Y %H:%M:%S",
        level=logging.INFO,
    )

    tokenizer = BertTokenizer.from_pretrained('bert-base-chinese', vocab_file=args.vocab_file)
    tokenizer.add_tokens('[NewLine]')
    tokenizer.add_tokens('[space]')
    args.output_folder = 'Seq2Seq_Transformers/Model/test'
    os.makedirs(args.output_folder, exist_ok=True)
    tokenizer.save_pretrained(args.output_folder)

    model = EncoderDecoderModel.from_encoder_decoder_pretrained("bert-base-chinese", "bert-base-chinese")
    model.encoder.resize_token_embeddings(len(tokenizer))
    model.decoder.resize_token_embeddings(len(tokenizer))
    model.config.encoder.vocab_size = len(tokenizer)
    model.config.decoder.vocab_size = len(tokenizer)
    if torch.cuda.is_available():
        args.device = torch.device("cuda")
        args.n_gpu = torch.cuda.device_count()
    else:
        args.device = torch.device("cpu")
        args.n_gpu = 0
    model.to(args.device)
    if args.n_gpu > 1:
        model = torch.nn.DataParallel(model)
    # loading the data
    train_pt_file = os.path.join(args.data_folder, 'train.pt')
    valid_pt_file = os.path.join(args.data_folder, 'valid.pt')
    train_dataloader = Get_DataLoader(train_pt_file, batch_size=args.batch_size, training=True)
    valid_dataloader = Get_DataLoader(valid_pt_file, batch_size=args.batch_size)
    # Prepare optimizer and schedule (linear warmup and decay)
    t_total = int(len(train_dataloader) // training_args.gradient_accumulation_steps * args.num_train_epochs)
    no_decay = ["bias", "LayerNorm.weight"]
    optimizer_grouped_parameters = [
        {
            "params": [p for n, p in model.named_parameters() if not any(nd in n for nd in no_decay)],
            "weight_decay": training_args.weight_decay
        },
        {
            "params": [p for n, p in model.named_parameters() if any(nd in n for nd in no_decay)],
            "weight_decay": 0.0,
        },
    ]
    optimizer = AdamW(optimizer_grouped_parameters, lr=training_args.learning_rate, eps=training_args.adam_epsilon)
    scheduler = get_linear_schedule_with_warmup(
        optimizer, num_warmup_steps=training_args.warmup_steps, num_training_steps=t_total
    )
    # start training
    logger.info("***************************")
    for field in fields(training_args):
        logger.info(f"{field.name}: {getattr(training_args, field.name)}")
    logger.info("***************************")
    global_step = 0
    tr_loss = 0.0
    logging_loss = 0.0
    loss_scalar = 1000000
    previous_loss_scaler = -1
    model.train()
    model.zero_grad()
    for epoch in tqdm(range(args.num_train_epochs), desc="Epoch", ascii=True):
        epoch_iterator = tqdm(train_dataloader, desc="Iteration", ascii=True)
        for step, inputs in enumerate(epoch_iterator):
            model.train()
            for k, v in inputs.items():
                inputs[k] = v.to(args.device)

            outputs = model(**inputs)
            # loss, outputs = model(input_ids=inputs["input_ids"], decoder_input_ids=inputs["input_ids"], lm_labels=inputs["input_ids"])[:2]
            loss = outputs[0]  # model outputs are always tuple in transformers (see doc)

            if args.n_gpu > 1:
                loss = loss.mean()  # mean() to average on multi-gpu parallel training
            if training_args.gradient_accumulation_steps > 1:
                loss = loss / training_args.gradient_accumulation_steps

            loss.backward()
            tr_loss += loss.item()

            if (step + 1) % training_args.gradient_accumulation_steps == 0 or (
                # last step in epoch but step is always smaller than gradient_accumulation_steps
                len(epoch_iterator) <= training_args.gradient_accumulation_steps
                and (step + 1) == len(epoch_iterator)
            ):
                torch.nn.utils.clip_grad_norm_(model.parameters(), training_args.max_grad_norm)

                optimizer.step()
                scheduler.step()
                model.zero_grad()
                global_step += 1

                if args.logging_steps > 0 and global_step % args.logging_steps == 0:
                    logs = {}
                    loss_scalar = (tr_loss - logging_loss) / args.logging_steps
                    learning_rate_scalar = scheduler.get_last_lr()[0]
                    logs["learning_rate"] = learning_rate_scalar
                    logs["loss"] = loss_scalar
                    logs["loss_difference"] = abs(loss_scalar-previous_loss_scaler)
                    previous_loss_scaler = loss_scalar
                    logging_loss = tr_loss

                    epoch_iterator.write(json.dumps({**logs, **{"step": global_step}}))
                    if loss_scalar < training_args.early_stop:# or logs["loss_difference"] < training_args.stop_barrier:
                        break

                if args.save_steps > 0 and global_step % args.save_steps == 0:
                    # Save model checkpoint
                    output_dir = os.path.join(args.output_folder, f"checkpoint-{global_step}")
                    os.makedirs(output_dir, exist_ok=True)
                    logger.info("Saving model checkpoint to %s", output_dir)
                    # Save a trained model and configuration using `save_pretrained()`.
                    # They can then be reloaded using `from_pretrained()`
                    if isinstance(model, torch.nn.DataParallel):
                        model = model.module
                    if not isinstance(model, PreTrainedModel):
                        raise ValueError("Trainer.model appears to not be a PreTrainedModel")
                    model.save_pretrained(output_dir)

                    torch.save(optimizer.state_dict(), os.path.join(output_dir, "optimizer.pt"))
                    torch.save(scheduler.state_dict(), os.path.join(output_dir, "scheduler.pt"))
                    logger.info("Saving optimizer and scheduler states to %s", output_dir)
        if loss_scalar < training_args.early_stop:
            break

    output_dir = args.output_folder
    os.makedirs(output_dir, exist_ok=True)
    logger.info("Saving model checkpoint to %s", output_dir)
    # Save a trained model and configuration using `save_pretrained()`.
    # They can then be reloaded using `from_pretrained()`
    if isinstance(model, torch.nn.DataParallel):
        model = model.module
    if not isinstance(model, PreTrainedModel):
        raise ValueError("Trainer.model appears to not be a PreTrainedModel")
    model.save_pretrained(output_dir)
```

Besides, for each time step, encoder_outputs are the same, like the picture below. I think it's very strange.
I am not sure if they are the same problems.

![image](https://user-images.githubusercontent.com/15016623/86547733-8c556000-bf6c-11ea-8a41-76524a98f0b3.png)


Cannot train RoBERTa from scratch with multiple nodes and multiple GPUs

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...):

Language I am using the model on (English, Chinese ...):

The problem arises when using:
- [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
- [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Use the `transformers/examples/language-modeling/run_language_modeling.py` to train RoBERT model from scratch.
2. I am using the Slurm tool to submit a job that requires multiple nodes and multiple GPUs.  In this case, it requests 2 nodes and 2 GPUs for each. The shell scripts:
```
#!/bin/bash
#SBATCH --time=3:00:00
#SBATCH --ntasks=2
#SBATCH --nodes=2
#SBATCH --cpus-per-task=2
#SBATCH --gres=gpu:2
#SBATCH --mem=64G
#SBATCH --job-name=pre-train
#SBATCH --output=pre-train.out
#SBATCH --account=XXXXXX

module load gcc
module load cuda cudnn
module load openmpi nccl

source ~/roberta/bin/activate
export NCCL_DEBUG=INFO
export NPROC_PER_NODE=2
export HDF5_USE_FILE_LOCKING='FALSE'
export PARENT=`/bin/hostname -s`
export MPORT=13000
export CHILDREN=`scontrol show hostnames $SLURM_JOB_NODELIST | grep -v $PARENT`
export HOSTLIST="$PARENT $CHILDREN"
echo $HOSTLIST
export WORLD_SIZE=$SLURM_NTASKS

srun distributed_runner.sh
```
3. `distributed_runner.sh` script is:
```
#!/bin/bash
/bin/hostname -s
source ~/roberta/bin/activate
python3 -m torch.distributed.launch \
		--nproc_per_node=$NPROC_PER_NODE \
		--nnodes=$SLURM_JOB_NUM_NODES \
		--node_rank=$SLURM_PROCID \
		--master_addr="$PARENT" --master_port="$MPORT" \
		run_language_modeling.py \
		--gradient_accumulation_steps=16 \
		--train_data_file="./data/sample.txt" \
		--output_dir="./sample_model/" \
		--model_type=roberta \
		--mlm \
		--local_rank=$SLURM_LOCALID \
		--config_name="./sample_config" \
		--tokenizer_name="./sample_config" \
		--do_train \
		--line_by_line \
		--learning_rate=1e-4 \
		--num_train_epochs=40 \
		--save_total_limit=5 \
		--save_steps=20 \
		--per_gpu_train_batch_size=16 \
		--seed=42
```
4. `sample.txt` is a dummy training set with 20K lines where each line is a text string. Data is already pre-processed following the official instruction. `sample_config/` includes all the pre-processed vocabulary and config files: config.json, merges.txt, tokenizer_config.json, and vocab.json.

5. The job can be successfully launched on two nodes, with no error. But **each node only uses one GPU** based on the GPU usage with `nvidia-smi` command.

**Node 1**
```
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 440.33.01    Driver Version: 440.33.01    CUDA Version: 10.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Tesla V100-SXM2...  On   | 00000000:3B:00.0 Off |                    0 |
| N/A   40C    P0    70W / 300W |  21175MiB / 32510MiB |    100%      Default |
+-------------------------------+----------------------+----------------------+
|   1  Tesla V100-SXM2...  On   | 00000000:86:00.0 Off |                    0 |
| N/A   35C    P0    42W / 300W |     11MiB / 32510MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0    190822      C   /home/py3.6/bin/python3                    13003MiB |
|    0    190823      C   /home/py3.6/bin/python3                     8161MiB |
+-----------------------------------------------------------------------------+
```

**Node 2**
```
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 440.33.01    Driver Version: 440.33.01    CUDA Version: 10.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Tesla V100-SXM2...  On   | 00000000:18:00.0 Off |                    0 |
| N/A   43C    P0    77W / 300W |  17127MiB / 32510MiB |     97%      Default |
+-------------------------------+----------------------+----------------------+
|   1  Tesla V100-SXM2...  On   | 00000000:3B:00.0 Off |                    0 |
| N/A   33C    P0    40W / 300W |     11MiB / 32510MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0    209232      C   /home/py3.6/bin/python3                     6899MiB |
|    0    209233      C   /home/py3.6/bin/python3                    10217MiB |
+-----------------------------------------------------------------------------+
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
Expect each node can use two GPUs. Namely, the Processes should show the usage of GPU 0 and 1 for both nodes.
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.0.1
- Platform: CentOS Linux 7 (Core)
- Python version: Python 3.6.3
- PyTorch version (GPU?): torch==1.5.1
- Tensorflow version (GPU?): tensorflow-gpu==2.1.0
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Yes


Better TPU Support in examples

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

I tried to train BERT on TPU recently, and found your [examples](https://github.com/huggingface/transformers/blob/master/examples) has done work on this topic. However, some code looks experimental and not perfectly ready to be used. The combination of [xla and pytorch-lightening](https://github.com/huggingface/transformers/blob/master/examples/lightning_base.py) looks great, but seems not be used neither in any training scripts nor in documents. I'd like to know when will those codes be ready, and I'd be very glad to contribute some code.


Tokenizers throwing warning "The current process just got forked, Disabling parallelism to avoid deadlocks.. To disable this warning, please explicitly set TOKENIZERS_PARALLELISM=(true | false)"

Internals of the library; Tokenization.

I know this warning is because the transformer library is updated to 3.x.
I know the warning saying to set TOKENIZERS_PARALLELISM = true / false

My question is where should i set TOKENIZERS_PARALLELISM = true  / false
is this when defining tokenizers like
```
tok = Tokenizer.from_pretrained('xyz', TOKENIZERS_PARALLELISM=True) // this doesn't work
```
or is this when encoding text like
```
tok.encode_plus(text_string, some=some, some=some, TOKENIZERS_PARALLELISM = True) // this also didn't work
```

Suggestions anyone?


Bert-extractive-summaizer importing issue

Hi ,

I am facing issues while import summarizer.

NameError                                 Traceback (most recent call last)
<ipython-input-6-3b4384c20fe2> in <module>()
----> 1 from summarizer import Summarizer
      2
      3 body = 'Text body that you want to summarize with BERT'
      4 body2 = 'Something else you want to summarize with BERT'
      5 model = Summarizer()

~/anaconda3/envs/amazonei_tensorflow_p36/lib/python3.6/site-packages/summarizer/__init__.py in <module>()
----> 1 from summarizer.model_processors import Summarizer, SingleModel, TransformerSummarizer

~/anaconda3/envs/amazonei_tensorflow_p36/lib/python3.6/site-packages/summarizer/model_processors.py in <module>()
----> 1 from summarizer.bert_parent import BertParent
      2 from summarizer.cluster_features import ClusterFeatures
      3 from summarizer.sentence_handler import SentenceHandler
      4 from typing import List
      5 from abc import abstractmethod

~/anaconda3/envs/amazonei_tensorflow_p36/lib/python3.6/site-packages/summarizer/bert_parent.py in <module>()
      9
     10
---> 11 class BertParent(object):
     12
     13     """

~/anaconda3/envs/amazonei_tensorflow_p36/lib/python3.6/site-packages/summarizer/bert_parent.py in BertParent()
     16
     17     MODELS = {
---> 18         'bert-base-uncased': (BertModel, BertTokenizer),
     19         'bert-large-uncased': (BertModel, BertTokenizer),
     20         'xlnet-base-cased': (XLNetModel, XLNetTokenizer),

**NameError: name 'BertModel' is not defined**

Please help me on this issue


Error using t5-base-cnn

I'm trying to fine-tune all the t5 models over CNN/DailyMail to see how they perform compared to the BART ones. I came across your t5-base-cnn model day. I tried using it in the way mentioned but got interrupted by an error that says :

OSError: Model name 'sshleifer/t5-base-cnn' was not found in tokenizers model name list (t5-small, t5-base, t5-large, t5-3b, t5-11b). We assumed 'sshleifer/t5-base-cnn' was a path, a model identifier, or url to a directory containing vocabulary files named ['spiece.model'] but couldn't find such vocabulary files at this path or url.

Earlier when I fine-tuned t5-small over CNN, in the output file there was this spiece.model but it's not present in your listed files.

Any suggestions how to get over this so that I can use your t5-base-cnn instead of fine-tuning all-over myself?

Thanks.

@sshleifer


can't get models directory after running python run_squad.py

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...):
Bert

Language I am using the model on (English, Chinese ...):
English

The problem arises when using:
after running "python run_squad.py ", I didn't get models directory.
running time of my code in colab is only 20 minutes, I think the training process is not done, what's the problem? How to solve that?

The tasks I am working on is:
SQUaD

## To reproduce

Steps to reproduce the behavior:

1.https://qa.fastforwardlabs.com/pytorch/hugging%20face/wikipedia/bert/transformers/2020/05/19/Getting_Started_with_QA.html
2.click the "Open in Colab" button above

!python run_squad.py  \
    --model_type bert   \
    --model_name_or_path bert-base-uncased  \
    --output_dir models/bert/ \
    --data_dir data/squad   \
    --overwrite_output_dir \
    --overwrite_cache \
    --do_train  \
    --train_file train-v2.0.json   \
    --version_2_with_negative \
    --do_lower_case  \
    --do_eval   \
    --predict_file dev-v2.0.json   \
    --per_gpu_train_batch_size 2   \
    --learning_rate 3e-5   \
    --num_train_epochs 2.0   \
    --max_seq_length 384   \
    --doc_stride 128   \
    --threads 10   \
    --save_steps 5000

## Expected behavior

## Environment info
colab GPU

- `transformers` version:
- Platform:colab
- Python version:3.6.9
- PyTorch version (GPU?):1.5.1+cu101
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:


Can't pickle tokenizers ...

Tokenizers are losing all their special tokens when un-pickled.  Not sure if there are other attributes that aren't being rehydrated as well ...

```
hf_tokenizer
# <transformers.tokenization_roberta.RobertaTokenizer at 0x7fc4ce625f10>

hf_tokenizer.special_tokens_map
# {'bos_token': '<s>',
# 'eos_token': '</s>',
# 'unk_token': '<unk>',
# 'sep_token': '</s>',
# 'pad_token': '<pad>',
# 'cls_token': '<s>',
# 'mask_token': '<mask>'}

pickle.dump( hf_tokenizer, open( "save.p", "wb" ) )
hf_tokenizer = pickle.load( open( "save.p", "rb" ) )

hf_tokenizer.special_tokens_map
# {'bos_token': '',
# 'eos_token': '',
# 'unk_token': '',
# 'sep_token': '',
# 'pad_token': '',
# 'cls_token': '',
# 'mask_token': ''}

```


Version track


'Size' Error while loading t5-large model

# 🐛 Bug

## Information

Model I am using t5-large:

Language I am using the model on English

The problem arises when using:
```
from transformers import T5Tokenizer, T5Model

tokenizer   = T5Tokenizer.from_pretrained('t5-large')
model       = T5Model.from_pretrained('t5-large')

input_ids   = tokenizer('sentence embeddings from t5 model', return_tensors="pt")
outputs     = model(input_ids=input_ids, decoder_input_ids=input_ids)
```

Error message :

```
KeyError: 'size'

During handling of the above exception, another exception occurred:

AttributeError                            Traceback (most recent call last)
<ipython-input-2-0ba21adf6547> in <module>
      1 input_ids   = tokenizer('chyuu wow this is working', return_tensors="pt")
----> 2 outputs     = model(input_ids=input_ids, decoder_input_ids=input_ids)

~/tfproject/tfenv/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    548             result = self._slow_forward(*input, **kwargs)
    549         else:
--> 550             result = self.forward(*input, **kwargs)
    551         for hook in self._forward_hooks.values():
    552             hook_result = hook(self, input, result)

~/tfproject/tfenv/lib/python3.7/site-packages/transformers/modeling_t5.py in forward(self, input_ids, attention_mask, encoder_outputs, decoder_input_ids, decoder_attention_mask, decoder_past_key_value_states, use_cache, inputs_embeds, decoder_inputs_embeds, head_mask, output_attentions, output_hidden_states)
    949                 head_mask=head_mask,
    950                 output_attentions=output_attentions,
--> 951                 output_hidden_states=output_hidden_states,
    952             )
    953

~/tfproject/tfenv/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    548             result = self._slow_forward(*input, **kwargs)
    549         else:
--> 550             result = self.forward(*input, **kwargs)
    551         for hook in self._forward_hooks.values():
    552             hook_result = hook(self, input, result)

~/tfproject/tfenv/lib/python3.7/site-packages/transformers/modeling_t5.py in forward(self, input_ids, attention_mask, encoder_hidden_states, encoder_attention_mask, inputs_embeds, head_mask, past_key_value_states, use_cache, output_attentions, output_hidden_states)
    676             raise ValueError("You cannot specify both input_ids and inputs_embeds at the same time")
    677         elif input_ids is not None:
--> 678             input_shape = input_ids.size()
    679             input_ids = input_ids.view(-1, input_shape[-1])
    680         elif inputs_embeds is not None:

~/tfproject/tfenv/lib/python3.7/site-packages/transformers/tokenization_utils_base.py in __getattr__(self, item)
    185             return self.data[item]
    186         except KeyError:
--> 187             raise AttributeError
    188
    189     def __getstate__(self):

AttributeError:
```

The tasks I am working on is:

Getting sentence representation

## To reproduce

## Environment info

- `transformers` version:
- Platform: Ubuntu 18.04
- Python version: python3.7
- PyTorch version (GPU?): 1.5.1
- Tensorflow version (GPU?): 1.14
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No


Exposing prepare_for_model for both slow & fast tokenizers

Internals of the library; Tokenization.

With version v3.0.0, two breaking changes that could have been avoided have been introduced. After discussion with @n1to and @thomwolf, this PR aims to revert these changes, by implementing two changes:

- The `prepare_for_model` method for both slow and tokenizers is now publicly exposed (it was only the case for the slow tokenizers before v3.0.0)
- The truncation methods now default to `longest_first` instead of `first_only` .

This PR adds two tests, for both Python and Rust tokenizers:

- Assert that `tokenizer.prepare_for_model(tokenizer.encode(x)) == tokenizer.encode_plus(x)`
- Assert that the output of `prepare_for_model` for rust and python tokenizers is equal.

closes https://github.com/huggingface/transformers/issues/5377
closes https://github.com/huggingface/transformers/issues/5447
closes https://github.com/huggingface/transformers/issues/5460


Possible breaking undetected change to "data/processors/squad.py"

# ❓ Questions & Help

It looks like the squad data utils hasn't been updated to the new version.

https://github.com/huggingface/transformers/blob/fcf0652460753f8a81f7576e8abdaa6b3742f00e/src/transformers/data/processors/squad.py#L136

```
        encoded_dict = tokenizer.encode_plus(  # TODO(thom) update this logic
            truncated_query if tokenizer.padding_side == "right" else span_doc_tokens,
            span_doc_tokens if tokenizer.padding_side == "right" else truncated_query,
            truncation="only_second" if tokenizer.padding_side == "right" else "only_first",
            padding="max_length",
            max_length=max_seq_length,
            return_overflowing_tokens=True,
            stride=max_seq_length - doc_stride - len(truncated_query) - sequence_pair_added_tokens,
            return_token_type_ids=True,
        )
```

In squad utils, `encoded_dict` used to have a 'overflowing_tokens' key, but in the new version, all the overflowing tokens are returned as list of lists. But it looks like the data processor doesn't take this into account

```
        if "overflowing_tokens" not in encoded_dict or (
            "overflowing_tokens" in encoded_dict and len(encoded_dict["overflowing_tokens"]) == 0
        ):
            break
        span_doc_tokens = encoded_dict["overflowing_tokens"]
```

So this logic would interpret that there are no overflowing tokens since that key doesn't exist.

The code I am working on is using version 2.11.0, doesn't have 'overflowing_tokens' either, but it doesn't have the list of lists like in version 3.0.0. It has something called 'overflow_to_sample_mapping', which returns only zeros, which I am not sure how to use.

The 2.11.0 documentation doesn't mention this return value. (https://huggingface.co/transformers/v2.11.0/main_classes/tokenizer.html) .

But it looks the data processor wouldn't give overflowing tokens or notice an issue for 2.11.0 as well. And it doesn't seem to have a method to get strided overflowing tokens.


Add DeeBERT (entropy-based early exiting for *BERT).
Paper: https://www.aclweb.org/anthology/2020.acl-main.204/
Based on its original repository: https://github.com/castorini/DeeBERT


Seq2Seq: Option to not store whole dataset in memory

Help wanted

Extra attention is needed, help appreciated


35 Model Hub entries fail AutoConfig

Here is what I ran:

```python
from transformers.hf_api import HfApi
from tqdm import tqdm
import pandas as pd

model_list = HfApi().model_list()
model_ids = [x.modelId for x in model_list]

from transformers import AutoConfig

def check_hub(cls, model_ids):
    results = {}
    failure_data = {}
    for m in tqdm(model_ids):
        try:
            cls.from_pretrained(m)
            results[m] = True
        except Exception as e:
            failure_data[m] = e.args
            results[m] = False
    return results, failure_data
results, failure_data = check_hub(AutoConfig, model_ids)
print(failure_data)
```

Results:
```python

{'DeBERTa/base': ('Unrecognized model in DeBERTa/base. Should have a `model_type` key in its config.json, or contain one of the following strings in its name: retribert, t5, mobilebert, distilbert, albert, camembert, xlm-roberta, marian, mbart, bart, reformer, longformer, roberta, flaubert, bert, openai-gpt, gpt2, transfo-xl, xlnet, xlm, ctrl, electra, encoder-decoder',),
 'DeBERTa/large': ('Unrecognized model in DeBERTa/large. Should have a `model_type` key in its config.json, or contain one of the following strings in its name: retribert, t5, mobilebert, distilbert, albert, camembert, xlm-roberta, marian, mbart, bart, reformer, longformer, roberta, flaubert, bert, openai-gpt, gpt2, transfo-xl, xlnet, xlm, ctrl, electra, encoder-decoder',),
 'Itcast/cnc_output': ('Unrecognized model in Itcast/cnc_output. Should have a `model_type` key in its config.json, or contain one of the following strings in its name: retribert, t5, mobilebert, distilbert, albert, camembert, xlm-roberta, marian, mbart, bart, reformer, longformer, roberta, flaubert, bert, openai-gpt, gpt2, transfo-xl, xlnet, xlm, ctrl, electra, encoder-decoder',),
 'Narsil/fr_pretrained': ('Unrecognized model in Narsil/fr_pretrained. Should have a `model_type` key in its config.json, or contain one of the following strings in its name: retribert, t5, mobilebert, distilbert, albert, camembert, xlm-roberta, marian, mbart, bart, reformer, longformer, roberta, flaubert, bert, openai-gpt, gpt2, transfo-xl, xlnet, xlm, ctrl, electra, encoder-decoder',),
 'Narsil/pretrained': ('Unrecognized model in Narsil/pretrained. Should have a `model_type` key in its config.json, or contain one of the following strings in its name: retribert, t5, mobilebert, distilbert, albert, camembert, xlm-roberta, marian, mbart, bart, reformer, longformer, roberta, flaubert, bert, openai-gpt, gpt2, transfo-xl, xlnet, xlm, ctrl, electra, encoder-decoder',),
 'Narsil/pretrained2': ('Unrecognized model in Narsil/pretrained2. Should have a `model_type` key in its config.json, or contain one of the following strings in its name: retribert, t5, mobilebert, distilbert, albert, camembert, xlm-roberta, marian, mbart, bart, reformer, longformer, roberta, flaubert, bert, openai-gpt, gpt2, transfo-xl, xlnet, xlm, ctrl, electra, encoder-decoder',),
 'abryee/TigXLNet': ('`d_head` (64) should be equal to `d_model // n_head` (48)',),
 'adamlin/ClinicalBert_all_notes': ('Unrecognized model in adamlin/ClinicalBert_all_notes. Should have a `model_type` key in its config.json, or contain one of the following strings in its name: retribert, t5, mobilebert, distilbert, albert, camembert, xlm-roberta, marian, mbart, bart, reformer, longformer, roberta, flaubert, bert, openai-gpt, gpt2, transfo-xl, xlnet, xlm, ctrl, electra, encoder-decoder',),
 'adamlin/ClinicalBert_disch': ('Unrecognized model in adamlin/ClinicalBert_disch. Should have a `model_type` key in its config.json, or contain one of the following strings in its name: retribert, t5, mobilebert, distilbert, albert, camembert, xlm-roberta, marian, mbart, bart, reformer, longformer, roberta, flaubert, bert, openai-gpt, gpt2, transfo-xl, xlnet, xlm, ctrl, electra, encoder-decoder',),
 'adamlin/NCBI_BERT_pubmed_mimic_uncased_base_transformers': ('Unrecognized model in adamlin/NCBI_BERT_pubmed_mimic_uncased_base_transformers. Should have a `model_type` key in its config.json, or contain one of the following strings in its name: retribert, t5, mobilebert, distilbert, albert, camembert, xlm-roberta, marian, mbart, bart, reformer, longformer, roberta, flaubert, bert, openai-gpt, gpt2, transfo-xl, xlnet, xlm, ctrl, electra, encoder-decoder',),
 'adamlin/NCBI_BERT_pubmed_mimic_uncased_large_transformers': ('Unrecognized model in adamlin/NCBI_BERT_pubmed_mimic_uncased_large_transformers. Should have a `model_type` key in its config.json, or contain one of the following strings in its name: retribert, t5, mobilebert, distilbert, albert, camembert, xlm-roberta, marian, mbart, bart, reformer, longformer, roberta, flaubert, bert, openai-gpt, gpt2, transfo-xl, xlnet, xlm, ctrl, electra, encoder-decoder',),
 'dccuchile/cased': ('Unrecognized model in dccuchile/cased. Should have a `model_type` key in its config.json, or contain one of the following strings in its name: retribert, t5, mobilebert, distilbert, albert, camembert, xlm-roberta, marian, mbart, bart, reformer, longformer, roberta, flaubert, bert, openai-gpt, gpt2, transfo-xl, xlnet, xlm, ctrl, electra, encoder-decoder',),
 'dccuchile/uncased': ('Unrecognized model in dccuchile/uncased. Should have a `model_type` key in its config.json, or contain one of the following strings in its name: retribert, t5, mobilebert, distilbert, albert, camembert, xlm-roberta, marian, mbart, bart, reformer, longformer, roberta, flaubert, bert, openai-gpt, gpt2, transfo-xl, xlnet, xlm, ctrl, electra, encoder-decoder',),
 'djstrong/bg_cs_pl_ru_cased_L-12_H-768_A-12': ('Unrecognized model in djstrong/bg_cs_pl_ru_cased_L-12_H-768_A-12. Should have a `model_type` key in its config.json, or contain one of the following strings in its name: retribert, t5, mobilebert, distilbert, albert, camembert, xlm-roberta, marian, mbart, bart, reformer, longformer, roberta, flaubert, bert, openai-gpt, gpt2, transfo-xl, xlnet, xlm, ctrl, electra, encoder-decoder',),
 'facebook/dpr-ctx_encoder-single-nq-base': ('dpr',),
 'facebook/dpr-question_encoder-single-nq-base': ('dpr',),
 'facebook/dpr-reader-single-nq-base': ('dpr',),
 'healx/gpt-2-pubmed-large': ('Unrecognized model in healx/gpt-2-pubmed-large. Should have a `model_type` key in its config.json, or contain one of the following strings in its name: retribert, t5, mobilebert, distilbert, albert, camembert, xlm-roberta, marian, mbart, bart, reformer, longformer, roberta, flaubert, bert, openai-gpt, gpt2, transfo-xl, xlnet, xlm, ctrl, electra, encoder-decoder',),
 'healx/gpt-2-pubmed-medium': ('Unrecognized model in healx/gpt-2-pubmed-medium. Should have a `model_type` key in its config.json, or contain one of the following strings in its name: retribert, t5, mobilebert, distilbert, albert, camembert, xlm-roberta, marian, mbart, bart, reformer, longformer, roberta, flaubert, bert, openai-gpt, gpt2, transfo-xl, xlnet, xlm, ctrl, electra, encoder-decoder',),
 'hfl/rbt3': ('Unrecognized model in hfl/rbt3. Should have a `model_type` key in its config.json, or contain one of the following strings in its name: retribert, t5, mobilebert, distilbert, albert, camembert, xlm-roberta, marian, mbart, bart, reformer, longformer, roberta, flaubert, bert, openai-gpt, gpt2, transfo-xl, xlnet, xlm, ctrl, electra, encoder-decoder',),
 'hfl/rbtl3': ('Unrecognized model in hfl/rbtl3. Should have a `model_type` key in its config.json, or contain one of the following strings in its name: retribert, t5, mobilebert, distilbert, albert, camembert, xlm-roberta, marian, mbart, bart, reformer, longformer, roberta, flaubert, bert, openai-gpt, gpt2, transfo-xl, xlnet, xlm, ctrl, electra, encoder-decoder',),
 'microsoft/unilm-base-cased': ('Unrecognized model in microsoft/unilm-base-cased. Should have a `model_type` key in its config.json, or contain one of the following strings in its name: retribert, t5, mobilebert, distilbert, albert, camembert, xlm-roberta, marian, mbart, bart, reformer, longformer, roberta, flaubert, bert, openai-gpt, gpt2, transfo-xl, xlnet, xlm, ctrl, electra, encoder-decoder',),
 'microsoft/unilm-large-cased': ('Unrecognized model in microsoft/unilm-large-cased. Should have a `model_type` key in its config.json, or contain one of the following strings in its name: retribert, t5, mobilebert, distilbert, albert, camembert, xlm-roberta, marian, mbart, bart, reformer, longformer, roberta, flaubert, bert, openai-gpt, gpt2, transfo-xl, xlnet, xlm, ctrl, electra, encoder-decoder',),
 'mrm8488/prunebert-base-uncased-finepruned-topK-squadv2': ('masked_bert',),
 'mrm8488/prunebert-multi-uncased-finepruned-l0-reg-tydiqa-for-xqa': ('masked_bert',),
 'mrm8488/prunebert-multi-uncased-finepruned-magnitude-tydiqa-for-xqa': ('masked_bert',),
 'mrm8488/prunebert-multi-uncased-finepruned-soft-movement-tydiqa-for-xqa': ('masked_bert',),
 'mrm8488/prunebert-multi-uncased-finepruned-topK-tydiqa-for-xqa': ('masked_bert',),
 'mrm8488/prunebert-multi-uncased-finepruned-tydiqa-for-xqa': ('masked_bert',),
 'oda/music5': ('Unrecognized model in oda/music5. Should have a `model_type` key in its config.json, or contain one of the following strings in its name: retribert, t5, mobilebert, distilbert, albert, camembert, xlm-roberta, marian, mbart, bart, reformer, longformer, roberta, flaubert, bert, openai-gpt, gpt2, transfo-xl, xlnet, xlm, ctrl, electra, encoder-decoder',),
 'pertschuk/0_RoBERTa': ('Unrecognized model in pertschuk/0_RoBERTa. Should have a `model_type` key in its config.json, or contain one of the following strings in its name: retribert, t5, mobilebert, distilbert, albert, camembert, xlm-roberta, marian, mbart, bart, reformer, longformer, roberta, flaubert, bert, openai-gpt, gpt2, transfo-xl, xlnet, xlm, ctrl, electra, encoder-decoder',),
 'radha1258/save': ('Unrecognized model in radha1258/save. Should have a `model_type` key in its config.json, or contain one of the following strings in its name: retribert, t5, mobilebert, distilbert, albert, camembert, xlm-roberta, marian, mbart, bart, reformer, longformer, roberta, flaubert, bert, openai-gpt, gpt2, transfo-xl, xlnet, xlm, ctrl, electra, encoder-decoder',),
 'sshleifer/blenderbot-3B': ('blenderbot',),
 'subbareddyiiit/iiit': ('Unrecognized model in subbareddyiiit/iiit. Should have a `model_type` key in its config.json, or contain one of the following strings in its name: retribert, t5, mobilebert, distilbert, albert, camembert, xlm-roberta, marian, mbart, bart, reformer, longformer, roberta, flaubert, bert, openai-gpt, gpt2, transfo-xl, xlnet, xlm, ctrl, electra, encoder-decoder',),
 'subbareddyiiit/tftelugu': ('Unrecognized model in subbareddyiiit/tftelugu. Should have a `model_type` key in its config.json, or contain one of the following strings in its name: retribert, t5, mobilebert, distilbert, albert, camembert, xlm-roberta, marian, mbart, bart, reformer, longformer, roberta, flaubert, bert, openai-gpt, gpt2, transfo-xl, xlnet, xlm, ctrl, electra, encoder-decoder',)}
```


Can't use AutoModelForCausalLM with bert

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): bert-base-uncased

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] my own modified scripts: (give details below)
Here is a simple 3 lines of code you can try to replicate the bug:
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained('bert-base-uncased')
model = AutoModelForCausalLM.from_pretrained('bert-base-uncased', is_decoder=True)

The tasks I am working on is:
XSUM / CNNDM summarization

## To reproduce

Steps to reproduce the behavior:

1. run the first 2 lines of code I put in the script section
2. run the first and third line of code I put in the script section

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

If you run the second line of code, you get:
AssertionError: If you want to use `BertLMHeadModel` as a standalone, add `is_decoder=True`.

If you run the third line of code (add is_decoder=True), you get:
TypeError: __init__() got an unexpected keyword argument 'is_decoder'

The first error occurs because it creates a default bert-base-uncased config, which does not set is_decoder to True. This is reasonable behavior.

The second error occurs because when you pass in is_decoder=True, it correctly gets added to the config, but is incorrectly passed to the model __init__. In this case, BertLMHeadModel's init ONLY takes a config - it does not accept ANY kwargs. Thus we crash. I don't think this is intended behavior - I feel like its reasonable to think you can pass in is_decoder to the config you want to create in AutoModelForCausalLM without crashing.

## Expected behavior

I expect if I run the code AutoModelForCausalLM('bert-base-uncased'), I will get back a BertLMHeadModel back with the is_decoder flag set to true in the config. Alternatively, I expect if I run the code AutoModelForCausalLM('bert-base-uncased', is_decoder=True) to get the same result.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.0.0
- Platform: Linux-3.10.0-862.14.4.el7.x86_64-x86_64-with-centos-7.5.1804-Core
- Python version: 3.7.3
- PyTorch version (GPU?): 1.4.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: tried with both
- Using distributed or parallel set-up in script?: no


TFAutoModelForSequenceClassification: ValueError: Layer #1 (named "classifier") expects 2 weight(s), but the saved weights have 4 element(s).

# 🐛 Bug

## Information

`TFAutoModelForSequenceClassification` does not work on v3.0.0 / can't load a model that was working on v2.11.0

## To reproduce

Steps to reproduce the behavior:

- This work:

```python
!pip install transformers==2.11.0
from transformers import TFAutoModelForSequenceClassification
model = TFAutoModelForSequenceClassification.from_pretrained("tblard/tf-allocine")
```

but not this:
```python
!pip install transformers>=3.0.0
from transformers import TFAutoModelForSequenceClassification
model = TFAutoModelForSequenceClassification.from_pretrained("tblard/tf-allocine")
```

as it outputs:
```shell
ValueError: Layer #1 (named "classifier") expects 2 weight(s), but the saved weights have 4 element(s).
```

- Using `TFCamembertForSequenceClassification` instead of `TFAutoModelForSequenceClassification` also works.

- I couldn't find any other model using `TFAutoModelForSequenceClassification` on the model zoo to verify the issue does not come from the model itself.

## Expected behavior

No errors.

## Environment info

Standard Google colab environment.

- `transformers` version: 3.0.0
- Platform: Linux-4.19.104+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.5.1+cu101 (True)
- Tensorflow version (GPU?): 2.2.0 (True)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No


Truncation in GLUE should be longest first

The GLUE example currently crashes with the QQP task because of the truncation.

It outputs the following warnings:

```
ERROR:transformers.tokenization_utils:We need to remove 186 to truncate the inputbut the first sequence has a length 25. Please select another truncation strategy than TruncationStrategy.ONLY_FIRST, for instance 'longest_first' or 'only_second'.
ERROR:transformers.tokenization_utils:We need to remove 49 to truncate the inputbut the first sequence has a length 34. Please select another truncation strategy than TruncationStrategy.ONLY_FIRST, for instance 'longest_first' or 'only_second'.
ERROR:transformers.tokenization_utils:We need to remove 203 to truncate the inputbut the first sequence has a length 42. Please select another truncation strategy than TruncationStrategy.ONLY_FIRST, for instance 'longest_first' or 'only_second'.
ERROR:transformers.tokenization_utils:We need to remove 39 to truncate the inputbut the first sequence has a length 28. Please select another truncation strategy than TruncationStrategy.ONLY_FIRST, for instance 'longest_first' or 'only_second'.
ERROR:transformers.tokenization_utils:We need to remove 23 to truncate the inputbut the first sequence has a length 20. Please select another truncation strategy than TruncationStrategy.ONLY_FIRST, for instance 'longest_first' or 'only_second'.
ERROR:transformers.tokenization_utils:We need to remove 91 to truncate the inputbut the first sequence has a length 63. Please select another truncation strategy than TruncationStrategy.ONLY_FIRST, for instance 'longest_first' or 'only_second'.
```

before crashing with the following:

```
ValueError: expected sequence of length 128 at dim 1 (got 202)
```

closes https://github.com/huggingface/transformers/issues/5460


Update: ElectraDiscriminatorPredictions forward.

`ElectraDiscriminatorPredictions.forward` should not need `attention_mask`.


Unable to use run_squad with xla_spawn.py on TPU

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...):
Electra

Language I am using the model on (English, Chinese ...):
ENG

The problem arises when using:
* [ ] the official example scripts: RUN_squad.py + xla_spawn.py

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name): official SQUaD task

## To reproduce

Steps to reproduce the behavior:

1. Install pytorch-xla on colab using:
```
VERSION = "20200325"  #@param ["1.5" , "20200325", "nightly"]
!curl https://raw.githubusercontent.com/pytorch/xla/master/contrib/scripts/env-setup.py -o pytorch-xla-env-setup.py
!python pytorch-xla-env-setup.py --version $VERSION
```
2. Trying to run_squad on colab TPUs using xla_spawn.py
```
python examples/xla_spawn.py --num_cores 8 \
   examples/question-answering/run_squad.py \
  --model_type electra \
  --model_name_or_path google/electra-base-discriminator \
  --do_train \
  --do_eval \
  --do_lower_case \
  --train_file "/content/drive/My Drive/bert/train.json" \
  --predict_file "/content/drive/My Drive/bert/val.json" \
  --per_gpu_train_batch_size 12 \
  --learning_rate 3e-5 \
  --num_train_epochs 2.0 \
  --max_seq_length 384 \
  --doc_stride 128 \
  --output_dir "/content/drive/My Drive/bert/newdir6"

```

2. Error is thrown up
```
Traceback (most recent call last):
  File "examples/xla_spawn.py", line 72, in <module>
    main()
  File "examples/xla_spawn.py", line 68, in main
    xmp.spawn(mod._mp_fn, args=(), nprocs=args.num_cores)
AttributeError: module 'run_squad' has no attribute '_mp_fn'

```
## Expected behavior
Training should run properly using xla_spawn.py, which it does for GLUE tasks using:
```
python examples/xla_spawn.py --num_cores 8 \
	examples/text-classification/run_glue.py
```

## Environment info

- `transformers` version: 2nd July 2020 clone.
- Platform: Colab
- Python version: 3.8
- PyTorch version (GPU?): 20200325 (pytorch-xla)
- Tensorflow version (GPU?):N
- Using GPU in script?:N
- Using distributed or parallel set-up in script?:N


[Discussion] fix zero divison error (Reformer batch size bug)

This PR is for discussion
During the training of the reformer model, I noticed that when you increase the batch size, a zero divison error often occurs
With these changes the error no longer occurs


Fix saved model creation

Fix a bug when a saved model is created the parameters `output_hidden_states` and `output_attentions` was ignored.

Reproducibility with TF 2.2:

```python
import tensorflow as tf
from transformers import TFBertModel, BertTokenizer, BertConfig
config = BertConfig.from_pretrained("bert-base-multilingual-uncased", output_hidden_states=True)
model = TFBertModel.from_pretrained('bert-base-multilingual-uncased', config=config)
tokenizer = BertTokenizer.from_pretrained("bert-base-multilingual-uncased", use_fast=False)
features = tokenizer.encode_plus("Hello world.", add_special_tokens=True, max_length=48, pad_to_max_length=True, return_tensors="tf", truncation=True)
model._saved_model_inputs_spec = None
model._set_save_spec(dict(features))
tf.saved_model.save(model, "save/model")
```

Then run the serving CLI with:

```
saved_model_cli show --dir save/model/ --tag_set serve --signature_def serving_default
```

There will be only 2 outputs instead of 3.


Tokenizer summary

This PR introduces a mid/high-level summary of the different tokenizer types used in the library (a bit like the model summary).

Preview is [here](https://56179-155220641-gh.circle-artifacts.com/0/docs/_build/html/tokenizer_summary.html).


Fix typo in glossary


Fixing missing arguments for TransfoXL tokenizer when using TextGenerationPipeline

As discussed with @LysandreJik and @mfuntowicz , `TextGenerationPipeline` gives imperfect results when using TransfoXL as the tokenizer lacks the `add_space_before_punct_symbol` argument. In order to fix this, this PR overrides `_parse_and_tokenize` for this pipeline in order to pass tokenizer arguments.


Create model card

model card

Related to pretrained model cards

Create model card for electra-small-discriminator fine-tuned on SQUAD v2.0


Pre-Trained Model (ipuneetrathore/bert-base-cased-finetuned-finBERT) loads in PyTorch but not Tensorflow

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): TFBertModel

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [X] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [X] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

This Works:
```
import torch
PRE_TRAINED_MODEL_NAME = 'ipuneetrathore/bert-base-cased-finetuned-finBERT'
model = BertForSequenceClassification.from_pretrained(PRE_TRAINED_MODEL_NAME)

# loads just fine
```

This Does NOT Work:
```
import tensorflow as tf
PRE_TRAINED_MODEL_NAME = 'ipuneetrathore/bert-base-cased-finetuned-finBERT'
model = TFBertForSequenceClassification.from_pretrained(PRE_TRAINED_MODEL_NAME)

# ERROR:

OSError: Can't load weights for 'ipuneetrathore/bert-base-cased-finetuned-finBERT'. Make sure that:

- 'ipuneetrathore/bert-base-cased-finetuned-finBERT' is a correct model identifier listed on 'https://huggingface.co/models'

- or 'ipuneetrathore/bert-base-cased-finetuned-finBERT' is the correct path to a directory containing a file named one of tf_model.h5, pytorch_model.bin.
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

It should load the model.

<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.0.0
- Platform: Ubuntu 18.04
- Python version: 3.7
- PyTorch version (GPU?): 1.3.1 with GPU
- Tensorflow version (GPU?): 2.1.0 with GPU
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No


Changed expected_output_ids in TransfoXL generation test

#4826 fixed TransfoXL's `prepare_inputs_for_generation` function. This PR changes the expected outputs in the TransfoXL generation test to match the new correct outputs.