Different outputs in using convert_roberta_original_pytorch_checkpoint_to_pytorch.py

# 🐛 Bug
We have an issue in converting roberta model from fairseq format to huggingface format. The conversion function provided in the transformers library gives you different outputs when you pass sample data through the fairseq and huggingface-transformers versions seperately.

⬇Problem with our pretrained model
There's a difference between two output tensors. The actual number is⬇

huggingface transformers output:

>  tensor([[[ 2.1787e+01, -4.7770e+00,  6.1631e+00,  ..., -4.6316e+00,
>           -4.7297e+00, -3.9510e-01],
>          [ 2.0051e+00, -2.7158e+00,  5.2598e+00,  ..., -2.3681e+00,
>           -2.0179e+00, -1.5263e-02],
>          [-2.7891e+00, -4.7558e+00,  5.3717e+00,  ..., -4.5290e+00,
>           -3.8888e+00, -5.7892e-02],
>          [ 1.3125e+00, -3.9378e+00,  6.7551e+00,  ..., -3.6842e+00,
>           -3.4968e+00,  5.4736e-01],
>          [-3.4706e+00, -7.7992e+00,  1.6678e+01,  ..., -6.1806e+00,
>           -7.4419e+00, -8.5062e-02]]], grad_fn=<AddBackward0>)

fairseq output:

>  tensor([[[21.2672, -4.8905,  6.2439,  ..., -4.8653, -4.9650, -1.6207],
>          [ 1.4856, -2.8294,  5.3406,  ..., -2.6018, -2.2533, -1.2408],
>          [-3.3087, -4.8693,  5.4525,  ..., -4.7626, -4.1241, -1.2835],
>          [ 0.7930, -4.0513,  6.8359,  ..., -3.9179, -3.7322, -0.6782],
>          [-3.9902, -7.9127, 16.7589,  ..., -6.4142, -7.6773, -1.3106]]],
>        grad_fn=<AddBackward0>)

abs difference:

>  tensor([[[0.5195, 0.1135, 0.0808,  ..., 0.2336, 0.2354, 1.2256],
>          [0.5195, 0.1135, 0.0808,  ..., 0.2336, 0.2354, 1.2256],
>          [0.5195, 0.1135, 0.0808,  ..., 0.2336, 0.2354, 1.2256],
>          [0.5195, 0.1135, 0.0808,  ..., 0.2336, 0.2354, 1.2256],
>          [0.5195, 0.1135, 0.0808,  ..., 0.2336, 0.2354, 1.2256]]],
>        grad_fn=<AbsBackward>)

The same issue happens when we try to convert the default roberta-base model from fairseq format into transformers format.
We have change some source code from fairseq to register our model name and architecture(Just changes in some hyperparameters).
Our initial guess is that there are some parameters isn't or is wrongly loaded.

The error looks like this:

> max_absolute_diff = 1.2255859375
> Do both models output the same tensors? 💩
> Traceback (most recent call last):
>   File "convert_roberta_original_pytorch_checkpoint_to_pytorch.py", line 181, in <module>
>     args.roberta_checkpoint_path, args.pytorch_dump_folder_path, args.classification_head
>   File "convert_roberta_original_pytorch_checkpoint_to_pytorch.py", line 160, in convert_roberta_checkpoint_to_pytorch
>     raise Exception("Something went wRoNg")
> Exception: Something went wRoNg

Thanks a lot for helping.

## Information

Model I am using (Bert, XLNet ...): RoBERTa

Language I am using the model on (English, Chinese ...): English

## Expected behavior
Explanation about the function (or further contact in refining the function?)

## Environment info

- `transformers` version: the default version
- Platform: NYU Prince Cluster
- Python version: python 3.7
- PyTorch version (GPU?): No GPU
- Tensorflow version (GPU?): No GPU
- Using GPU in script?: No GPU
- Using distributed or parallel set-up in script?: No GPU


wrong parameters order in TFTransfoXLMainLayer _update_mems call

# 🐛 Bug

## Information

In file src/transformers/modeling_tf_transfo_xl.py present small typo in method parameters call.
In line 491 present method:
    def _update_mems(self, hids, mems, qlen, mlen)
And in line 610 on it's call:
    new_mems = self._update_mems(hids, mems, mlen, qlen)

As you can see qlen and mlen placed in wrong order. In result memory size can raise outside of specified value.


Summarization pipeline max_length parameter seems to just cut the summary rather than generating a complete sentence within the max length

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): default model from pipeline("summarization")

Language I am using the model on (English, Chinese ...): English

I am using the pipeline for summarization in most up to date version of Transformers. I am inputing a long piece of tax and setting the summarizer to be: summarizer(PIECE_OF_TEXT, max_length = 50).

I was expecting the summarizer to generate a summary within 50 words but it seems to only generate a summary that seems cut off (the ending of the summary ends with a comma and doesn't end in a grammatical sensible way. See example below.

**The piece of text to be summarized:**
Renal-cell carcinoma is characterized by susceptibility to both immunotherapeutic and antiangiogenic treatment approaches and resistance to cytotoxic chemotherapy.1 Agents such as sunitinib that target the vascular endothelial growth factor (VEGF) pathway are standard first-line therapy for advanced disease.2-7 Despite the approval of several targeted therapies by entities such as the Food and Drug Administration, the European Medicines Agency, and the Pharmaceuticals and Medical Devices Agency, the survival rate among patients with metastatic renal-cell carcinoma has plateaued.

Both the VEGF receptor tyrosine kinase inhibitor axitinib and the anti–programmed death 1 (PD-1) monoclonal antibody pembrolizumab have shown antitumor activity in patients with previously untreated advanced clear-cell renal-cell carcinoma.6,10 In a phase 1b trial involving patients with previously untreated metastatic renal-cell carcinoma, 73% (95% confidence interval [CI], 59 to 84) of the patients who received pembrolizumab plus axitinib had a response; 65% of patients had at least one treatment-related adverse event.11 We conducted the KEYNOTE-426 trial to determine whether pembrolizumab plus axitinib would result in better outcomes than sunitinib in patients with previously untreated advanced renal-cell carcinoma.

**And the summary:**
Renal-cell carcinoma is characterized by susceptibility to both immunotherapeutic and antiangiogenic treatment approaches. Agents such as sunitinib that target the vascular endothelial growth factor (VEGF) pathway are standard first,  axitinib and the anti–programmed death 1 (PD-1) monoclonal antibody pembrolizumab have shown antitumor activity in patients with previously untreated advanced clear-cell renal-cell carcin,


[WIP] Adding model parallelism for T5 (should work for other models as well)

This PR adds:
- a `get_block_list()` utility method which returns a list of the blocks in a Transformers model (currently only added on T5). Block can be Modules or list/tuple of Modules (if a single transformer block is spread in several ModuleList like in XLM).
- a `spread_on_devices(devices: Optional[List] = None)` method to spread a model on several devices by spreading the transformers blocks (roughly) evenly on the provided device list or all visible CUDA devices if no device list is given. The first device will host the remaining non-block modules in addition (the embeddings usually).

Currently, the code is in the T5 model but should be generic enough to be applied to other models if needed.

To use:
``` python
model = T5ForConditionalGeneration.from_pretrained('...')
model.spread_on_devices()  # Will spread on all visible CUDA devices by default
input = torch.tensor([...]).to('cuda:0')  # Inputs and outputs are on the first device
model(input)  # you should probably use only positional arguments for the forward pass (see spread_on_devices's docstring)
```

TODO:
 - [ ] try it
 - [ ] add tests if possible (on a dummy device list like ['cpu', 'cpu']?)

cc @patrickvonplaten @craffel


DistilBert not giving hidden states

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): DistilBert

Language I am using the model on (English, Chinese ...): Multilingual

The problem arises when using:
* [X] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: (give the name) just running inference
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Load modules
2. Create model with output_hidden_layers=True
3. run inference on one sentece

`from transformers import BertModel, BertTokenizerFast, BertConfig`
`from transformers import DistilBertModel, DistilBertTokenizerFast, DistilBertConfig`
`import torch`

`bert_config = BertConfig(output_hidden_states=True)`
`bert = BertModel(bert_config)`
`bert = bert.from_pretrained("bert-base-multilingual-cased")`
`bert_tokenizer = BertTokenizerFast.from_pretrained("bert-base-multilingual-cased")`

`distil_bert_config = DistilBertConfig(output_hidden_states=True)`
`distil_bert = DistilBertModel(distil_bert_config)`
`distil_bert = distil_bert.from_pretrained("distilbert-base-multilingual-cased")`
`distil_bert_tokenizer = DistilBertTokenizerFast.from_pretrained("distilbert-base-multilingual-cased")`

`sentence = "One stupid dummy sentence to test !"`

`input_bert = torch.tensor(bert_tokenizer.encode(sentence)).unsqueeze(0)`
`input_distil_bert = torch.tensor(distil_bert_tokenizer.encode(sentence)).unsqueeze(0)
`

`output_bert = bert(input_bert)`
`outupt_distil_bert = distil_bert(input_distil_bert)`
## Expected behavior
Return a tuple with 2 elements (like Bert)

Exemple of bert and desired behavior of distilbert:
`print(len(output_bert)) # => 2`
`print(output_bert[0].size()) # => 1, 18, 768`
`print(output_bert[1].size()) # => 1, 768`

## Real behavior
Return tuple of only 1 element
`print(len(outupt_distil_bert)) # => 1`
`print(outupt_distil_bert[0].size()) # => 1, 18, 768`

## Environment info
- `transformers` version: 2.7.0
- Platform: Linux 5.5.8-arch1-1
- Python version: Python 3.8.1
- PyTorch version (GPU?): 1.4.0 (GPU)
- Tensorflow version (GPU?): None
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


T5 fine tune for seq2seq generation

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->

## Details
<!-- Description of your issue -->

Hi
Is a script available for fine-tuning T5 base or large to to do seq2seq generative tasks like translation or dialog generation?

https://github.com/huggingface/transformers/blob/master/examples/run_generation.py

Doesn't seem to have T5


Create README.md

model card

Related to pretrained model cards


[Benchmark] QUAERO French Medical Corpus for Named Entity Recognition

# 🖥 Benchmarking `transformers`

## Benchmark

I am trying the Transformers library out on [QUAERO French Medical Corpus](https://quaerofrenchmed.limsi.fr/) NER-dataset (consisting of Medline titles and EMEA documents). Using Camembert-base 'out of the box' with default hyperparameters, I get an F1 measure of 85% on the EMEA dataset and 64% on Medline, while [one of the few papers](http://ceur-ws.org/Vol-1391/158-CR.pdf) I found that did an experiment on the same dataset reported F1 of 70% and 52%, respectively, using a classic CRF.

Since extensive grid search for hyperparameter optimization is computationally expensive even with a GPU, and given that I am relatively new to the field, I was wondering how to actually go about further optimizing the current model that I have. Is it even worth doing a lot of hyperparameter optimization or is it common that out of the box transformer models already do a decent job?

In particular, I'm not so sure which of the following things are worthwhile trying out;
- Making the input for the BERT model longer than 1 sentence
e.g. 3 input sentences [CLS] sentence1 sentence2 sentence3 [SEP]
- Extending the vocab of the tokenizer by adding tokens related to the medical domain? Not sure
if this even makes sense doing..
- Which hyperparameters do I focus on the most?
I assume epochs and batch size are the most important, but which others are worth trying out as well?
- Other suggestions to improve this model?

If you need more information regarding the setup I will be happy to provide it, thanks!


How can I use masked_lm_labels correctly?

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->

## Details
<!-- Description of your issue -->
Hi, I have a dataset like :

From Monday to Friday most people are busy working or studying, but in the evenings and weekends they are free and _ themselves.

And there are four candidates for the missing blank area:

["love", "work", "enjoy", "play"], here "enjoy" is the correct answer, it is a cloze-style task, and it looks like the maskLM in the BERT.

I want to train the model so that it can work better. I notice that there is a parameter called masked_lm_labels, and it can computing the masked language modeling loss. What should I do to train the BertForMaskedLM model with it.

Do you have any Example? Or can you teach me how to do that?
Thanks!
<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:


BART run run_train.sh RuntimeError: expected device cuda:0 but got device cpu

# 🐛 Bug

## Information

Model I am using (BART):

Language I am using the model on CNN dailymail (English, Chinese ...):

The problem arises when using:
* [* ] the official example scripts: (give details below)
text summarization of bart, run the script run_train.sh, following the guideline
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [*] an official GLUE/SQUaD task: (give the name)
the text summarization task, using cnn dailymail dataset.
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. run the script run_train.sh in examples/summarization/bart/run_train.sh
2. got the error information RuntimeError: expected device cuda:0 but got device cpu

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: I pip install transformers two days ago, I am not sure the version.
- Platform: ubuntu 1604
- Python version: 3.6.7
- PyTorch version (GPU?):  1.4.0
- Tensorflow version (GPU?):
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

Traceback:
Traceback (most recent call last):
  File "run_bart_sum.py", line 166, in <module>
    trainer = generic_train(model, args)
  File "******/transformer_base.py", line 304, in generic_train
    trainer.fit(model)
  File "******/anaconda3/lib/python3.6/site-packages/pytorch_lightning/trainer/trainer.py", line 676, in fit
    mp.spawn(self.ddp_train, nprocs=self.num_gpus, args=(model,))
  File "******/anaconda3/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 171, in spawn
    while not spawn_context.join():
  File "******/anaconda3/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 118, in join
    raise Exception(msg)
Exception:

-- Process 0 terminated with the following error:
Traceback (most recent call last):
  File "******/anaconda3/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 19, in _wrap
    fn(i, *args)
  File "******/anaconda3/lib/python3.6/site-packages/pytorch_lightning/trainer/distrib_data_parallel.py", line 341, in ddp_train
    self.run_pretrain_routine(model)
  File "******/anaconda3/lib/python3.6/site-packages/pytorch_lightning/trainer/trainer.py", line 924, in run_pretrain_routine
    False)
  File "******/anaconda3/lib/python3.6/site-packages/pytorch_lightning/trainer/evaluation_loop.py", line 263, in _evaluate
    output = self.evaluation_forward(model, batch, batch_idx, dataloader_idx, test_mode)
  File "******/anaconda3/lib/python3.6/site-packages/pytorch_lightning/trainer/evaluation_loop.py", line 418, in evaluation_forward
    output = model(*args)
  File "******/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "******/anaconda3/lib/python3.6/site-packages/pytorch_lightning/overrides/data_parallel.py", line 96, in forward
    output = self.module.validation_step(*inputs[0], **kwargs[0])
  File "******/run_bart_sum.py", line 58, in validation_step
    loss = self._step(batch)
  File "******/run_bart_sum.py", line 44, in _step
    lm_labels=lm_labels.cuda(),
  File "******/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "******/run_bart_sum.py", line 32, in forward
    lm_labels=lm_labels,
  File "******/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "******/anaconda3/lib/python3.6/site-packages/transformers/modeling_bart.py", line 925, in forward
    decoder_cached_states=decoder_cached_states,
  File "******/anaconda3/lib/python3.6/site-packages/transformers/modeling_bart.py", line 844, in forward
    decoder_cached_states=decoder_cached_states,
  File "******/anaconda3/lib/python3.6/site-packages/transformers/modeling_bart.py", line 499, in forward
    need_attn_weights=self.output_attentions,
  File "******/anaconda3/lib/python3.6/site-packages/transformers/modeling_bart.py", line 372, in forward
    attn_mask=attention_mask,
  File "******/anaconda3/lib/python3.6/site-packages/transformers/modeling_bart.py", line 629, in forward
    attn_weights = attn_weights.view(bsz, self.num_heads, tgt_len, src_len) + attn_mask
RuntimeError: expected device cuda:0 but got device cpu


Help wanted

Extra attention is needed, help appreciated

# ❓ Transformers pipeline in GCP

I am trying to use transformers pipeline in GCP cloud functions. While calling the function the downloading of model is happening everytime. How can we sort this issue.


tokenizer cannot load form model on disk

I wanted to load model saved on disk but it keeps on throwing this error

    File "train.py", line 1, in <module>
     import config
    File "/media/saurabh/D/code/bert_imdb_sentiment/src/config.py", line 14, in <module>
     do_lower_case=True
     File "/home/saurabh/anaconda3/lib/python3.7/site-packages/transformers/tokenization_utils.py", line 393, in from_pretrained
     return cls._from_pretrained(*inputs, **kwargs)
     File "/home/saurabh/anaconda3/lib/python3.7/site-packages/transformers/tokenization_utils.py", line 496, in _from_pretrained
     lst(cls.vocab_files_names.values()),
    OSError: Model name '.../input/bert_based_uncased/' was not found in tokenizers model name list
    (bert-base-uncased, bert-large-uncased, bert-base-cased, bert-large-cased, bert-base-
    multilingual-uncased, bert-base-multilingual-cased, bert-base-chinese, bert-base-german-cased,
    bert-large-uncased-whole-word-masking, bert-large-cased-whole-word-masking, bert-large-
    uncased-whole-word-masking-finetuned-squad, bert-large-cased-whole-word-masking-finetuned-
    squad, bert-base-cased-finetuned-mrpc, bert-base-german-dbmdz-cased, bert-base-german-
    dbmdz-uncased, bert-base-finnish-cased-v1, bert-base-finnish-uncased-v1, bert-base-dutch-
    cased). We assumed '..input//bert_based_uncased/' was a path, a model identifier, or url to a
    directory containing vocabulary files named ['vocab.txt'] but couldn't find such vocabulary files at
    this path or url.


Regarding distilbert-multilingual-uncased model

I am using pretrained distilbert-multilingual-uncased model to get the embeddings of a sentence. I  want to ask which layer would be great for taking the semantic embedding of a sentence


Create README.md

model card

Related to pretrained model cards

added documentation for our fine-tuned BERT model


Add tiny-bert-bahasa-cased model card

Related to pretrained model cards


BertJapaneseTokenizer accept options for mecab

Now we can pass `mecab_kwargs` to `BertJapaneseTokenizer.__init__` and set  tokenizer more accurately.

Changes:

1. `BertJapaneseTokenizer.__init__` accepts `mecab_kwargs` keyword argument. It is directly passed to `MeCabTokenizer.__init__`
2. Now we can disable `normalize_text` in `MeCabTokenizer` through `mecab_kwargs`
3. Also we can pass the argument to `MeCab.Tagger.__init__`  through `mecab_kwargs["mecab_option"]`. It is useful to customize mecab's dictionary.


Language model fine tuning using scibert as the base model

Usage

General questions about the library

# 🐛 Bug

## Information
I am trying to finetune the Scibert model on Covid dataset. Features for the train data is not being calculated properly.
Model I am using (Bert, XLNet ...): allenai/scibert_scivocab_uncased

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: Language Modelling
* [ ] my own task or dataset: my own dataset in text files

## To reproduce

Steps to reproduce the behavior:

1. python examples/run_language_modeling.py --output_dir=./lm_finetune --model_name_or_path=allenai/scibert_scivocab_uncased --do_train --train_data_file=lm_data/train.txt --do_eval --eval_data_file=lm_data/dev.txt --mlm --tokenizer_name allenai/scibert_scivocab_uncased  --model_type  bert

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior
Finetuned model and perplexity score on evaluation data
<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.6
- Platform: "CentOS Linux 7"
- Python version: 3.6.9
- PyTorch version (GPU?): yes
- Tensorflow version (GPU?): No
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

What should I provide the `model_type` as?


Tokenizers: setting bos_token_id = 0 and adding language_pair_codes

Core: Tokenization

Internals of the library; Tokenization.

I am unable to set bos_token_id=0 for a new SentencePiece tokenizer (MBART).
Here is what I'm doing?

```bash
wget https://s3.amazonaws.com/models.huggingface.co/bert/facebook/mbart-large-en-ro/sentence.bpe.model
```

```python
from transformers import T5Tokenizer
vocab_file = 'sentence.bpe.model'
t2 = T5Tokenizer(vocab_file, bos_token='<s>', bos_token_id=0)
t2.bos_token_id # => 1
```

The following also returns 1
```python
t2 = T5Tokenizer(vocab_file, bos_token='<s>', bos_token_id=0,
                 additional_special_tokens=['<s>'])
t2.bos_token_id
```

Help much appreciated!


update run_language_modeling.py for high efficiency in Multi GPUs

The code "model(inputs, masked_lm_labels=labels)" will return all outputs which causes out of GPU memory in train() function. After modifying the code, batch size per GPU increases from 4 to 32 in Multi GPUs.


can not init tokenizers from third party model , on albert model

# 🐛 Bug

## Information

Model I am using (albert.):

Language I am using the model on (English, Chinese ...):

The problem arises when using:
* [ *] the official example scripts: (give details below)

follow the instructions on :

`https://huggingface.co/models`

such as use "voidful/albert_chinese_tiny" model,

`AutoTokenizer.from_pretrained('voidful/albert_chinese_tiny')`

will raise
` Model name 'voidful/albert_chinese_tiny' was not found in tokenizers model name list (albert-base-v1, albert-large-v1, albert-xlarge-v1, albert-xxlarge-v1, albert-base-v2, albert-large-v2, albert-xlarge-v2, albert-xxlarge-v2). We assumed 'voidful/albert_chinese_tiny' was a path, a model identifier, or url to a directory containing vocabulary files named ['spiece.model'] but couldn't find such vocabulary files at this path or url.`


Evaluation of labelled test set?

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->

## Details
<!-- Description of your issue -->
Hi,

I have a labelled  test set of QQP data set. What arguments do i need to input if i want to report accuracy and F1 on test set not just the development set?
<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:


Mean reduce over last hidden state

One of the outputs of [TFBert](https://huggingface.co/transformers/model_doc/bert.html#transformers.TFBertModel.call) is the `last_hidden_state` which is a tensor of shape `(batch_size, sequence_length, hidden_size)`.

How someone could proceed to compute the `mean pooling` of the valid embeddings? I mean, as the `attention_mask` avoids performing attention on padding token indices, it can be used as weight to average only over the real input embeddings ignoring the pad embeddings.


How to trace the BertForQuestionAnswering

I followed the example [here](https://huggingface.co/transformers/model_doc/bert.html#bertforquestionanswering) and want to convert BertForQuestionAnswering to TorchScript.
Here is my code
```
from transformers import BertTokenizer, BertForQuestionAnswering
import torch

tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertForQuestionAnswering.from_pretrained('bert-large-uncased-whole-word-masking-finetuned-squad', torchscript=True)

question, text = "Who was Jim Henson?", "Jim Henson was a nice puppet"
input_ids = tokenizer.encode(question, text)
token_type_ids = [0 if i <= input_ids.index(102) else 1 for i in range(len(input_ids))]

input_tensor = torch.tensor([input_ids])
token_type_ids_tensor = torch.tensor([token_type_ids])

# The way I traced the model could be wrong here
traced_model = torch.jit.trace(model, (input_tensor, token_type_ids_tensor))
traced_model.eval()
start_scores, end_scores = traced_model(input_tensor, token_type_ids_tensor)

all_tokens = tokenizer.convert_ids_to_tokens(input_ids)
answer = ' '.join(all_tokens[torch.argmax(start_scores) : torch.argmax(end_scores)+1])

print(answer)
```
The answer I got is `jim henson was a nice puppet [SEP]`
Do you have any idea how to make it correct?
Thanks


Metrics are coupled to the run_glue.py tasks.

The metrics used for evaluating a run_glue task is coupled to the task itself (i.e. regression or classification). We wrote a `TrinarySentimentProcessor` which grabs a positive, neutral, or negative sentiment from a text, but we found that the simple accuracy measure was not the right one. We wanted to use log loss (cross entropy) and so we added an log loss function to the metrics (`__init__.py`) package and added another elif to the chain.

Why is the metric so coupled to the task itself? What if you wanted to use a different metric for any particular task? I understand that `run_glue.py` is an "example" but we've been building upon the architecture to reduce the workload of training for tasks outside of GLUE and SQUAD.  Maybe you could add a flag to the `run_glue.py` file like `--metric=cross-entropy`. Any thoughts? Are we just misusing the library by extending `glue.py`?


Create model card

model card

Related to pretrained model cards

Create model card for: distilbert-multi-finetuned-for-xqua-on-tydiqa


[T5, examples] replace heavy t5 models with tiny random models

As first done by @sshleifer in #3488 , this PR puts a tiny T5 model on S3 to save testing time.


T5 for summarization: pipeline x T5ForConditionalGeneration different results

I've been doing experiments with text summarization and got some different results between pipeline and T5ForConditionalGeneration.

First I use the model.generate() to generate the resume. It runs very fast (even on CPU) and give poors results.

Second I use the pipeline, passing the same model I build on the first step. This runs slower and give a very good results.

Third, I re-run the first model.generate(). Now, the the model runs slower and produces the same result as pipeline.

I did a colab so that you can check.

I'm loosing any point in using the model x pipeline?

https://colab.research.google.com/drive/15HOerw3mYVCsjedW_dVGeyRX5cYWiNvS


resize_token_embeddings error for Transformer-XL

# 🐛 Bug

## Information

Model I am using : Transformer-XL

Language I am using the model on : English

The problem arises when using:
* [ ] my own modified scripts: a fine-tuning script for TransfoXLLMHeadModel

## To reproduce

The following code aims to add two new tokens to the vocabulary, 'wug' and 'wugs'. After doing so to the tokenizer, we call `resize_token_embeddings` with the model in order to update its input embeddings to have correct dimension to account for the new tokens.

``` python
import torch
from transformers import TransfoXLTokenizer, TransfoXLLMHeadModel

model = TransfoXLLMHeadModel.from_pretrained('transfo-xl-wt103')
tokenizer = TransfoXLTokenizer.from_pretrained('transfo-xl-wt103')

tokenizer.add_tokens(['wug', 'wugs'])
model.resize_token_embeddings(len(tokenizer))
```

Running the above gives the following error
```
Traceback (most recent call last):
  File "bug.py", line 9, in <module>
    model.resize_token_embeddings(len(tokenizer))
  File "/home/AD/rdsie/anaconda3/envs/lign251/lib/python3.7/site-packages/transformers/modeling_utils.py", line 198, in resize_token_embeddings
    model_embeds = base_model._resize_token_embeddings(new_num_tokens)
  File "/home/AD/rdsie/anaconda3/envs/lign251/lib/python3.7/site-packages/transformers/modeling_utils.py", line 213, in _resize_token_embeddings
    new_embeddings = self._get_resized_embeddings(old_embeddings, new_num_tokens)
  File "/home/AD/rdsie/anaconda3/envs/lign251/lib/python3.7/site-packages/transformers/modeling_utils.py", line 234, in _get_resized_embeddings
    old_num_tokens, old_embedding_dim = old_embeddings.weight.size()
  File "/home/AD/rdsie/anaconda3/envs/lign251/lib/python3.7/site-packages/torch/nn/modules/module.py", line 576, in __getattr__
    type(self).__name__, name))
AttributeError: 'AdaptiveEmbedding' object has no attribute 'weight'
```
It seems that the function `resize_token_embeddings()` does not currently account for the particulars of the input embeddings used for the TransformerXLLMHeadModel.

## Expected behavior

We expect that `resize_token_embeddings` should handle the appropriate updating of the embedding layers for the new vocabulary size, so that the model can be correctly used with the new tokens.

Thank you in advance


unable to completely load T5 pretrained model; missing/unexpected keys

# 🐛 Bug

## Information

Model I am using: T5

## To reproduce

```
model, info = T5ForConditionalGeneration.from_pretrained('t5-small',output_loading_info=True)
```
info is
`
{'missing_keys': ['encoder.embed_tokens.weight', 'decoder.embed_tokens.weight', 'lm_head.weight'], 'unexpected_keys': ['encoder.block.0.layer.0.layer_norm.bias', 'encoder.block.0.layer.1.layer_norm.bias', 'encoder.block.1.layer.0.layer_norm.bias', 'encoder.block.1.layer.1.layer_norm.bias', 'encoder.block.2.layer.0.layer_norm.bias', 'encoder.block.2.layer.1.layer_norm.bias', 'encoder.block.3.layer.0.layer_norm.bias', 'encoder.block.3.layer.1.layer_norm.bias', 'encoder.block.4.layer.0.layer_norm.bias', 'encoder.block.4.layer.1.layer_norm.bias', 'encoder.block.5.layer.0.layer_norm.bias', 'encoder.block.5.layer.1.layer_norm.bias', 'encoder.final_layer_norm.bias', 'decoder.block.0.layer.0.layer_norm.bias', 'decoder.block.0.layer.1.layer_norm.bias', 'decoder.block.0.layer.2.layer_norm.bias', 'decoder.block.1.layer.0.layer_norm.bias', 'decoder.block.1.layer.1.layer_norm.bias', 'decoder.block.1.layer.2.layer_norm.bias', 'decoder.block.2.layer.0.layer_norm.bias', 'decoder.block.2.layer.1.layer_norm.bias', 'decoder.block.2.layer.2.layer_norm.bias', 'decoder.block.3.layer.0.layer_norm.bias', 'decoder.block.3.layer.1.layer_norm.bias', 'decoder.block.3.layer.2.layer_norm.bias', 'decoder.block.4.layer.0.layer_norm.bias', 'decoder.block.4.layer.1.layer_norm.bias', 'decoder.block.4.layer.2.layer_norm.bias', 'decoder.block.5.layer.0.layer_norm.bias', 'decoder.block.5.layer.1.layer_norm.bias', 'decoder.block.5.layer.2.layer_norm.bias', 'decoder.final_layer_norm.bias'], 'error_msgs': []}
`
<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

No keys should be missing or unexpected
<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.7.0
- Platform: Ubuntu
- Python version: 3.6
- PyTorch version (GPU?): 1.2.0 (yes)
- Tensorflow version (GPU?): nope
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: nope


Update README.md

Related to pretrained model cards

- Show that the last uploaded version was trained on more data (custom_license files)


Recommended preprocessing steps for english sentences in GPT2

# ❓ Questions & Help

To run inferences in english sentences... I'm not really sure what preprocessing steps I need to do before sending the tokenized text to gpt-2 in order to get the predictions. Any advice?

How to handle non-english words that can be found in an english sentences? Extra punctuation, space, new lines, user's mentions, hashtags, urls, alternative apostrophes, etc...?

## Details
<!-- Description of your issue -->

**A link to original question on Stack Overflow**:  https://stackoverflow.com/questions/60807799/gpt2-huggingfaces-transformer-preprocessing


[T5, Testst] Add extensive hard-coded integration tests and make sure PT and TF give equal results

A direct comparison to google's official model seems quite hard - not sure if that's absolutely needed @thomwolf, @craffel

But some integration tests for T5 would be very nice, to be sure that changes in the future will not break T5.
This PR adds hard-coded integration tests, where the input for summarization is copied from Bart's summarization tests and the input for translation is taken from Appendix D of the official [paper](https://arxiv.org/pdf/1910.10683.pdf)

Checking the expected output for PT, one can see that the output looks quite good!

- [x] Add PyTorch integration tests
- [x] Verify quality (subjectively for the moment)
- [x] Add TF integration tests
- [x] Same output for PT and TF

UPDATE:
- Found a big bug in TF Beam Search generation (see comment) -> this PR fixes it


model name '../data/bert_models/chinese_finetuned_lm/pytorch_model.bin' was not found in model name list . Creating an empty model card.

nlp = pipeline('fill-mask',
                   # model=args.bert_model_path,
                   # config=args.bert_config_path,
                   # tokenizer=args.bert_model_dir
                    model = '../data/bert_models/chinese_finetuned_lm/pytorch_model.bin',
                    config = '../data/bert_models/chinese_finetuned_lm/config.json',
                    tokenizer = "../data/bert_models/chinese_finetuned_lm/"
                   )
i use fine-tune model (chinese bert) ,when i end fine-tune  i can not load model  !


How to extract "contiguous tokens" from `NerPipeline` results?

Using `NerPipeline`, I want to be able to input a string (sequence of tokens), and extract *entity groups*, where an entity group is a contiguous series of tokens, having the same *entity type*.

**Example:**

For the ner code below:

```
from transformers import pipeline, AutoModelForTokenClassification, AutoTokenizer

# Allocate a pipeline for sentiment-analysis
model = AutoModelForTokenClassification.from_pretrained("dbmdz/bert-large-cased-finetuned-conll03-english")
tokenizer = AutoTokenizer.from_pretrained("bert-base-cased")
nlp = pipeline('ner', model=model, tokenizer=tokenizer)
nlp('Enzo works at the Australian National University (AUN)')
```

This returns:

```
[{'entity': 'I-PER', 'score': 0.9983270168304443, 'word': 'En'},
 {'entity': 'I-PER', 'score': 0.9952995777130127, 'word': '##zo'},
 {'entity': 'I-ORG', 'score': 0.9984350204467773, 'word': 'Australian'},
 {'entity': 'I-ORG', 'score': 0.9967807531356812, 'word': 'National'},
 {'entity': 'I-ORG', 'score': 0.9959043264389038, 'word': 'University'},
 {'entity': 'I-ORG', 'score': 0.9900023937225342, 'word': 'AU'},
 {'entity': 'I-ORG', 'score': 0.9763911366462708, 'word': '##N'}]
```

When I want it to return something like:

```
[{'entity_group': 'I-PER', 'score': 0.9983270168304443, 'word': 'Enzo'},
 {'entity_group': 'I-ORG', 'score': 0.9984350204467773, 'word': 'Australian National University'},
 {'entity_group': 'I-ORG', 'score': 0.9900023937225342, 'word': 'AUN'}]
```

I should be able to write a function that performs the above transformation if the indices of the word pieces are also indicated in the dictionary output of `NerPipeline`. Is this currently possible? Please advise me on if there's already an easy way to do this.

If not, I can fork the repo and introduce an `index` key to the dictionary output. I can send a PR for this if the use case is general enough.


[T5, TF 2.2] change tf t5 argument naming

**Problem**:
As shown in #3539, in TF 2.2 errors occur due to the naming of the first argument in the `keras.layer.__call__` function of TF T5.
Previously `decoder_input_ids` was used as the first argument - which did not produce any errors in TF <= 2.1. In TF 2.2, it produces the error:

```
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py in __call__(self, *args, **kwargs)
    797     else:
    798       raise ValueError(
--> 799           'The first argument to `Layer.call` must always be passed.')
    800
    801     call_context = base_layer_utils.call_context()

ValueError: The first argument to `Layer.call` must always be passed.
```

**Conclusion**
It seems that we have to change to a consistent naming, being `inputs` for the first argument of every `keras.layer.__call__` function


Impossible to use T5 11b

Core: Modeling

Internals of the library; Models.

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...):
T5 11B param
Language I am using the model on (English, Chinese ...):
English
The problem arises when using:
* [ x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:
```
summarizer = pipeline(task="summarization", model="t5-11b", tokenizer="t5-11b")
summary = summarizer(
	article,
	min_length=5,
	max_length=100
	)
print("The Summary, ",summary[0]['summary_text'])
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

As indicated in [T5's repo](https://github.com/google-research/text-to-text-transfer-transformer), they used Mesh Tensorflow, which is (according to them) the only way to run inference T5. This means the default CPU setting and even 1 GPU device setting would result in the following

1. CPU setting: run forever (my experience)
2. GPU setting: OOM (although I haven't tested this).

Meaning the current implementations of 3b and 11b would most likely render useless.

More testing needs to be done on whether this is the same for smaller models.

Get an output
<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.7.0
- Platform: GCP
- Python version: 3.7.7
- PyTorch version (GPU?): 1.1.0 and no GPU
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?: no


[T5, pipeline] fix bug in warnings

Warnings only took `self.model.config.max_length` in consideration, but not the actual passed `max_length` parameter.

This PR fixes this.


[examples] unit test for run_bart_sum

- add lightning to `examples/requirements.txt`
- first lightning unittest!


[testing] add timeout_decorator

This is a simple way to make sure code doesn't get slower over time. Since it is a new dependency, I wanted to show a tiny PR before I use it more.


KeyError: 'answers' error when using BioASQ dataset using Huggingface Transformers

# 🐛 Bug

I am using Bert on BioASQ question answering dataset using the script run_squad.py from Huggingface Transformers.

##To reproduce

Steps to reproduce the behavior:

1.I am using run_squad.py https://github.com/huggingface/transformers/blob/master/examples/run_squad.py from Huggingface Transformers for fine-tuning on BioASQ Question Answering dataset.

2.I have converted the tensorflow weights provided by the authors of BioBERT https://github.com/dmis-lab/bioasq-biobert to Pytorch as discussed here https://github.com/huggingface/transformers/issues/312.

3.Further, I am using the preprocessed data of BioASQ [(https://github.com/dmis-lab/bioasq-biobert)] which is converted to the SQuAD form. However, when I am running the run_squad.py script with the below parameters

python3 run_squad.py \
 --model_type bert \
  --model_name_or_path /scratch/oe7/uk1594/BioBERT/BioBERT-PyTorch/BioBERTv1.1-SQuADv1.1-Factoid-PyTorch/ \
  --do_train \
  --do_eval \
  --save_steps 1000 \
  --train_file $data/BioASQ-train-factoid-6b.json \
  --predict_file $data/BioASQ-test-factoid-6b-1.json \
  --per_gpu_train_batch_size 12 \
  --learning_rate 3e-5 \
  --num_train_epochs 2.0 \
  --max_seq_length 384 \
  --doc_stride 128 \
  --output_dir /scratch/oe7/uk1594/BioBERT/BioBERT-PyTorch/QA_output_squad/BioASQ-factoid-6b/BioASQ-factoid-6b-1-issue-23mar/

I get the below error:

  0%|          | 0/1 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "run_squad.py", line 856, in <module>
    main()
  File "run_squad.py", line 845, in main
    result = evaluate(args, model, tokenizer, prefix=global_step)
  File "run_squad.py", line 299, in evaluate
    dataset, examples, features = load_and_cache_examples(args, tokenizer, evaluate=True, output_examples=True)
  File "run_squad.py", line 475, in load_and_cache_examples
    examples = processor.get_dev_examples(args.data_dir, filename=args.predict_file)
  File "/scratch/oe7/uk1594/lib/python3.7/site-packages/transformers/data/processors/squad.py", line 522, in get_dev_examples
    return self._create_examples(input_data, "dev")
  File "/scratch/oe7/uk1594/lib/python3.7/site-packages/transformers/data/processors/squad.py", line 549, in _create_examples
    answers = qa["answers"]
KeyError: 'answers'

- `transformers` version: Latest
- Platform:
- Python version: python3.7.4
- PyTorch version (GPU?): pytorch 1.4.
- Tensorflow version (GPU?):tensorflow/2.0.0
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?:No

Appreciate your help.
Thanks!!


Usage

General questions about the library

I am getting an error such as forward () got an unexpected keyword argument 'output_all_encoded_layers', how can I fix it?

class BertBinaryClassifier(nn.Module):
    def __init__(self, dropout=0.1):
        super(BertBinaryClassifier, self).__init__()

        self.bert = BertModel.from_pretrained('bert-base-uncased')

        self.dropout = nn.Dropout(dropout)
        self.linear = nn.Linear(768, 1)
        self.sigmoid = nn.Sigmoid()

    def forward(self, tokens, masks=None):
        _, pooled_output = self.bert(tokens, attention_mask=masks, output_all_encoded_layers=False)
        dropout_output = self.dropout(pooled_output)
        linear_output = self.linear(dropout_output)
        proba = self.sigmoid(linear_output)
        return proba

bert_clf = BertBinaryClassifier()
bert_clf = bert_clf.cuda()

x = torch.tensor(train_tokens_ids[:3]).to(device)
y, pooled = bert_clf.bert(x, output_all_encoded_layers=False)
x.shape, y.shape, pooled.shape

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-22-def915600e8d> in <module>()
      1 x = torch.tensor(train_tokens_ids[:3]).to(device)
----> 2 y, pooled = bert_clf.bert(x, output_all_encoded_layers=False)
      3 x.shape, y.shape, pooled.shape

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    530             result = self._slow_forward(*input, **kwargs)
    531         else:
--> 532             result = self.forward(*input, **kwargs)
    533         for hook in self._forward_hooks.values():
    534             hook_result = hook(self, input, result)

TypeError: forward() got an unexpected keyword argument 'output_all_encoded_layers'


Quick Tour TF2.0 error: dataclasses.FrozenInstanceError: cannot assign to field 'label'

# 🐛 Bug

## Information

Model I am using: bert-base-cased

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. install TensorFlow 2 with conda  `conda install tensorflow`
2. install Transformers either from source or using pip `pip install transformers`
3. run the Quick Tour TF 2 example with the following content:
```python
import tensorflow as tf
import tensorflow_datasets
from transformers import *

# Load dataset, tokenizer, model from pretrained model/vocabulary
tokenizer = BertTokenizer.from_pretrained('bert-base-cased')
model = TFBertForSequenceClassification.from_pretrained('bert-base-cased')
data = tensorflow_datasets.load('glue/mrpc')

# Prepare dataset for GLUE as a tf.data.Dataset instance
train_dataset = glue_convert_examples_to_features(data['train'], tokenizer, max_length=128, task='mrpc')
valid_dataset = glue_convert_examples_to_features(data['validation'], tokenizer, max_length=128, task='mrpc')
train_dataset = train_dataset.shuffle(100).batch(32).repeat(2)
valid_dataset = valid_dataset.batch(64)

# Prepare training: Compile tf.keras model with optimizer, loss and learning rate schedule
optimizer = tf.keras.optimizers.Adam(learning_rate=3e-5, epsilon=1e-08, clipnorm=1.0)
loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
metric = tf.keras.metrics.SparseCategoricalAccuracy('accuracy')
model.compile(optimizer=optimizer, loss=loss, metrics=[metric])

# Train and evaluate using tf.keras.Model.fit()
history = model.fit(train_dataset, epochs=2, steps_per_epoch=115,
                    validation_data=valid_dataset, validation_steps=7)

# Load the TensorFlow model in PyTorch for inspection
model.save_pretrained('./save/')
```
## Expected behavior
```
Traceback (most recent call last):
  File "quick_tour_tf2.py", line 11, in <module>
    train_dataset = glue_convert_examples_to_features(data['train'], tokenizer, max_length=128, task='mrpc')
  File "C:\Users\Anh Minh\.conda\envs\transformers\lib\site-packages\transformers\data\processors\glue.py", line 86, in glue_convert_examples_to_features
    example = processor.tfds_map(example)
  File "C:\Users\Anh Minh\.conda\envs\transformers\lib\site-packages\transformers\data\processors\utils.py", line 115, in tfds_map
    example.label = self.get_labels()[int(example.label)]
  File "<string>", line 4, in __setattr__
dataclasses.FrozenInstanceError: cannot assign to field 'label'
```
### Update:
I have recently installed Pytorch and tried out `examples/run_tf_glue.py` and the same error occured.
```
(transformers) C:\Users\Anh Minh\Workspace\transformers_my_codes>python run_tf_glue.py
2020-03-31 10:43:55.555102: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_101.dll
2
2020-03-31 10:44:02.576281: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library nvcuda.dll
2020-03-31 10:44:02.669572: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1555] Found device 0 with properties:
pciBusID: 0000:01:00.0 name: GeForce RTX 2080 computeCapability: 7.5
coreClock: 1.59GHz coreCount: 46 deviceMemorySize: 8.00GiB deviceMemoryBandwidth: 417.29GiB/s
2020-03-31 10:44:02.679337: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_101.dll
2020-03-31 10:44:02.683708: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cublas64_10.dll
2020-03-31 10:44:02.689044: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cufft64_10.dll
2020-03-31 10:44:02.693280: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library curand64_10.dll
2020-03-31 10:44:02.762552: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cusolver64_10.dll
2020-03-31 10:44:02.767982: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cusparse64_10.dll
2020-03-31 10:44:02.773095: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudnn64_7.dll
2020-03-31 10:44:02.779069: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1697] Adding visible gpu devices: 0
2020-03-31 10:44:02.789070: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2
2020-03-31 10:44:02.799782: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1555] Found device 0 with properties:
pciBusID: 0000:01:00.0 name: GeForce RTX 2080 computeCapability: 7.5
coreClock: 1.59GHz coreCount: 46 deviceMemorySize: 8.00GiB deviceMemoryBandwidth: 417.29GiB/s
2020-03-31 10:44:02.809292: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_101.dll
2020-03-31 10:44:02.813862: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cublas64_10.dll
2020-03-31 10:44:02.818889: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cufft64_10.dll
2020-03-31 10:44:02.823516: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library curand64_10.dll
2020-03-31 10:44:02.828140: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cusolver64_10.dll
2020-03-31 10:44:02.833958: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cusparse64_10.dll
2020-03-31 10:44:02.839710: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudnn64_7.dll
2020-03-31 10:44:02.845469: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1697] Adding visible gpu devices: 0
2020-03-31 10:44:05.483986: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1096] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-03-31 10:44:05.489238: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102]      0
2020-03-31 10:44:05.492138: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] 0:   N
2020-03-31 10:44:05.499953: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1241] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6269 MB memory) -> physical GPU (device: 0, name: GeForce RTX 2080, pci bus id: 0000:01:00.0, compute capability: 7.5)
2020-03-31 10:44:06.412558: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cublas64_10.dll
INFO:absl:Overwrite dataset info from restored data version.
INFO:absl:Reusing dataset glue (C:\Users\Anh Minh\tensorflow_datasets\glue\mrpc\1.0.0)
INFO:absl:Constructing tf.data.Dataset for split None, from C:\Users\Anh Minh\tensorflow_datasets\glue\mrpc\1.0.0
Traceback (most recent call last):
  File "run_tf_glue.py", line 51, in <module>
    train_dataset = glue_convert_examples_to_features(data["train"], tokenizer, 128, TASK)
  File "C:\Users\Anh Minh\.conda\envs\transformers\lib\site-packages\transformers\data\processors\glue.py", line 86, in glue_convert_examples_to_features
    example = processor.tfds_map(example)
  File "C:\Users\Anh Minh\.conda\envs\transformers\lib\site-packages\transformers\data\processors\utils.py", line 115, in tfds_map
    example.label = self.get_labels()[int(example.label)]
  File "<string>", line 4, in __setattr__
dataclasses.FrozenInstanceError: cannot assign to field 'label'
```

The issue has been resolved by reinstalling Transformers 2.5.0

<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.7.0
- Platform: Windows 10
- Python version: 3.7.7
- PyTorch version (GPU?): 1.4.0 on GPU
- Tensorflow version (GPU?): 2.1 on GPU
- Using GPU in script?: yes, RTX 2080
- Using distributed or parallel set-up in script?: Unavailable


T5 Summarization

# 🐛 Bug
T5 summarization code in pipelines.py file gives an error.

## Information

Model I am using: T5

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
pipelines.py official documentation example at line 1146
```
# use t5 in tf
summarizer = pipeline("summarization", model="t5-base", tokenizer="t5-base", framework="tf")
summarizer("Sam Shleifer writes the best docstring examples in the whole world.", min_length=5, max_length=20)
```

## To reproduce
In google colab, I did the following:
```
!pip install transformers
from transformers import pipeline

summarizer = pipeline("summarization", model="t5-base", tokenizer="t5-base", framework="tf")

summarizer("Sam Shleifer writes the best docstring examples in the whole world.", min_length=5, max_length=10)
```

And this gets the following error:
```
Your max_length is set to 200, but you input_length is only 18. You might consider decreasing max_length manually, e.g. summarizer('...', max_length=50)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-23-0fc603a01733> in <module>()
----> 1 summarizer("Sam Shleifer writes the best docstring examples in the whole world.", min_length=5, max_length=10)

3 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py in __call__(self, *args, **kwargs)
    797     else:
    798       raise ValueError(
--> 799           'The first argument to `Layer.call` must always be passed.')
    800
    801     call_context = base_layer_utils.call_context()

ValueError: The first argument to `Layer.call` must always be passed.
```

- `transformers` version: 2.7.0
- Platform: Google Colab
- PyTorch version (GPU?): 1.4.0 GPU enabled
- Tensorflow version (GPU?): 2.2.0-rcl GPU enabled
- Using GPU in script?:
- Using distributed or parallel set-up in script?: No


[Docs] Add usage examples for translation and summarization

Adds docs. Fastest way to check is the changes in "rich diff" format.


Add model cards

model card

Related to pretrained model cards

Add IMDB tuned classifier and LMs model cards.


[Encoder-Decoder] Force models outputs to always have batch_size as their first dim

This PR remove the hard-coded variable `encoder_outputs_batch_dim_idx` from Bart and T5 by transposing BART's `encoder_outputs` dimensions before returning them.

**Reasons:**
- When adding more encoder-decoder models, we would always force the newly added model to have this variable
- When adding the modeling_encoder_decoder.py file, models that could be used in an encoder-decoder structure would also need to have this attribute, e.g. we would have to add it to Bert for example
- `encoder_outputs_batch_dim_idx` is a hard-coded variable that I don't think is very pretty

**Trade-off:**
- Now every encoder output in a encoder-decoder model has to have `batch_size` as their first dimension.

This PR is related to a question that already came up before (see #3120):
*Should we force all model outputs to have `batch_size` as their first dimension* ?

I think it would be good to always have `batch_size` as the first dimension (to the user) exposed output @thomwolf @LysandreJik @sshleifer @julien-c


Error on fine-tuning XLM like model on SQUaD like dataset

# 🐛 Bug

## Information

I am trying to fine-tune the model [xlm-mlm-100-1280](https://huggingface.co/xlm-mlm-100-1280) on SQuAD v1 dataset like (tydiQA) with the script provided for this task (transformers/examples/run_squad.py) and I get the following error:

```python
Epoch:   0% 0/5 [00:00<?, ?it/s]
Iteration:   0% 0/2383 [00:00<?, ?it/s]Traceback (most recent call last):
  File "/content/transformers/examples/run_squad.py", line 829, in <module>
    main()
  File "/content/transformers/examples/run_squad.py", line 768, in main
    global_step, tr_loss = train(args, train_dataset, model, tokenizer)
  File "/content/transformers/examples/run_squad.py", line 204, in train
    outputs = model(**inputs)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
TypeError: forward() got an unexpected keyword argument 'cls_index'

Epoch:   0% 0/5 [00:00<?, ?it/s]
Iteration:   0% 0/2383 [00:00<?, ?it/s]
```

## To reproduce
```bash
git clone https://github.com/huggingface/transformers
pip install -q ./transformers

# wget "your SQuAD v1 like dataset

python /content/transformers/examples/run_squad.py \
  --model_type xlm \
  --model_name_or_path xlm-mlm-100-1280 \
  --do_lower \
  --do_train \
  --do_eval \
  --train_file /content/dataset/tydiqa-goldp-v1.0-train.json \
  --predict_file /content/dataset/tydiqa-goldp-v1.0-dev.json \
  --per_gpu_train_batch_size 24 \
  --per_gpu_eval_batch_size 128 \
  --learning_rate 3e-5 \
  --num_train_epochs 5 \
  --max_seq_length 384 \
  --doc_stride 128 \
  --output_dir /content/model_output \
  --overwrite_output_dir \
  --save_steps 2000 \
  --threads 400
```

## Environment info
<!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.7.0
- Platform: Linux-4.14.137+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.4.0 (True)
- Tensorflow version (GPU?): 2.2.0-rc1 (True)
- Using GPU in script?: Yes. Nvidia Tesla P100
- Using distributed or parallel set-up in script?: No


pretrained EsperBERTo

Ex: LM (Pretraining)

Related to language modeling pre-training

Usage

General questions about the library

Hi,

I am trying to replicate the pretraining process mentioned in this blog post: https://huggingface.co/blog/how-to-train

I have time-restricted access to the GPU I'm currently working on and so I wanted to know how to save the checkpoint and resume the pretraining process from the latest checkpoint.

Thanks!


Error when training with distributed training on 4/8 Nvidia v100.

# 🐛 Bug

## Information

I'm getting the following error while training the official implementation in [examples/run_language_modeling.py](https://github.com/huggingface/transformers/blob/master/examples/run_language_modeling.py) on [WikiText-2 dataset](https://blog.einstein.ai/the-wikitext-long-term-dependency-language-modeling-dataset/) using multi-gpu setting (4/8 Nvidia GPU's).

```python
Traceback (most recent call last):
  File "run_language_modeling.py", line 976, in <module>
    main()
  File "run_language_modeling.py", line 926, in main
    global_step, tr_loss = train(args, train_dataset, model, tokenizer)
  File "run_language_modeling.py", line 513, in train
    outputs = model(inputs, masked_lm_labels=labels) if args.mlm else model(inputs, labels=labels)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/parallel/distributed.py", line 457, in forward
    self.reducer.prepare_for_backward(list(_find_tensors(output)))
RuntimeError: Expected to have finished reduction in the prior iteration before starting a new one. This error indicates that your module has parameters that were not used in producing loss. You can enable unused parameter detection by (1) passing the keyword argument `find_unused_parameters=True` to `torch.nn.parallel.DistributedDataParallel`; (2) making sure all `forward` function outputs participate in calculating loss. If you already have done the above two steps, then the distributed data parallel module wasn't able to locate the output tensors in the return value of your module's `forward` function. Please include the loss function and the structure of the return value of `forward` of your module when reporting this issue (e.g. list, dict, iterable). (prepare_for_backward at /pytorch/torch/csrc/distributed/c10d/reducer.cpp:518)
frame #0: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x33 (0x7f66375cb273 in /usr/local/lib/python3.6/dist-packages/torch/lib/libc10.so)
frame #1: c10d::Reducer::prepare_for_backward(std::vector<torch::autograd::Variable, std::allocator<torch::autograd::Variable> > const&) + 0x734 (0x7f66822b09e4 in /usr/local/lib/python3.6/dist-packages/torch/lib/libtorch_python.so)
frame #2: <unknown function> + 0x691a4c (0x7f668229fa4c in /usr/local/lib/python3.6/dist-packages/torch/lib/libtorch_python.so)
frame #3: <unknown function> + 0x1d3ef4 (0x7f6681de1ef4 in /usr/local/lib/python3.6/dist-packages/torch/lib/libtorch_python.so)
frame #4: _PyCFunction_FastCallDict + 0x35c (0x5674fc in /usr/bin/python)
frame #5: /usr/bin/python() [0x50abb3]
frame #6: _PyEval_EvalFrameDefault + 0x449 (0x50c5b9 in /usr/bin/python)
frame #7: /usr/bin/python() [0x508245]
```
Model I am using (Bert, XLNet ...): GPT-2

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [x] the official example scripts: [examples/run_language_modeling.py](https://github.com/huggingface/transformers/blob/master/examples/run_language_modeling.py)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: [WikiText-2 dataset](https://blog.einstein.ai/the-wikitext-long-term-dependency-language-modeling-dataset/)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Run [examples/run_language_modeling.py](https://github.com/huggingface/transformers/blob/master/examples/run_language_modeling.py) Using this command:
```
python -m torch.distributed.launch --nproc_per_node 4 run_language_modeling.py --output_dir=./output/ --model_type=gpt2 --model_name_or_path=gpt2 --do_train --train_data_file=./data/wiki.train.raw --per_gpu_train_batch_size 2 --num_train_epochs 10 --fp16
```

## Expected behavior

Should run distributed training without any errors.

## Environment info

I'm using this docker:
```
docker pull deepspeed/deepspeed:latest
```

- `transformers` version: 2.6.0
- Platform: Linux
- Python version:
- PyTorch version (GPU?):
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?: Yes


Resizing embedding matrix before sending it to the optimizer.

Hi there,

This is a minor bug when fine-tuning a pre-trained language model with a larger vocabulary using the run_lm_finetuning.py script.
Nothing major but I spent a couple of hours trying to figure out why my token embeddings were not being updated with their corresponding gradient :)

This bug will rise when you add new token types in the Tokenizer.
Since the resizing was done after passing the params to the optimizer, the wrong set of params for the embedding table were optimized.

Cheers


[T5, docs] remove useless and confusing lm_labels line

Remove useless  docstring


TensorFlow

Anything TensorFlow

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): bert-base-multilingual-cased

Language I am using the model on (English, Chinese ...): German

The problem arises when using:
* [X] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [X] an official GLUE/SQUaD task: NER
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Google Colab Notebook
2. Run: `!python3 run_tf_ner.py --data_dir ./data --model_type bert --labels labels.txt --model_name_or_path bert-base-multilingual-cased --output_dir germeval-model --max_seq_length 128 --num_train_epochs 3 --per_device_train_batch_size 32 --save_steps 750 --seed 1 --do_train --do_eval --do_predict`
3.
```
I0330 12:15:45.104261 140536926144384 modeling_tf_utils.py:388] loading weights file germeval-model/tf_model.h5
I0330 12:15:46.316838 140536926144384 modeling_tf_utils.py:428] Layers of TFBertForTokenClassification not initialized from pretrained model: ['dropout_75']
I0330 12:15:46.317042 140536926144384 modeling_tf_utils.py:432] Layers from pretrained model not used in TFBertForTokenClassification: ['dropout_37']
I0330 12:15:46.317251 140536926144384 run_tf_ner.py:420] Loading features from cached file ./data/cached_dev_bert-base-multilingual-cased_128.tf_record
I0330 12:15:46.483210 140536926144384 run_tf_ner.py:318] ***** Running evaluation *****
I0330 12:15:46.483375 140536926144384 run_tf_ner.py:319]   Num examples = 2200
I0330 12:15:46.483478 140536926144384 run_tf_ner.py:320]   Batch size = 8
Traceback (most recent call last):
  File "run_tf_ner.py", line 644, in <module>
    app.run(main)
  File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 299, in run
    _run_main(main, args)
  File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 250, in _run_main
    sys.exit(main(argv))
  File "run_tf_ner.py", line 579, in main
    args, strategy, model, tokenizer, labels, pad_token_label_id, mode="dev"
  File "run_tf_ner.py", line 322, in evaluate
    for eval_features, eval_labels in eval_iterator:
  File "/usr/local/lib/python3.6/dist-packages/fastprogress/fastprogress.py", line 39, in __iter__
    if self.total != 0: self.update(0)
  File "/usr/local/lib/python3.6/dist-packages/fastprogress/fastprogress.py", line 56, in update
    self.update_bar(0)
  File "/usr/local/lib/python3.6/dist-packages/fastprogress/fastprogress.py", line 76, in update_bar
    else: self.on_update(val, f'{100 * val/self.total:.2f}% [{val}/{self.total} {elapsed_t}<{remaining_t}{end}]')
  File "/usr/local/lib/python3.6/dist-packages/fastprogress/fastprogress.py", line 126, in on_update
    elif self.parent is not None: self.parent.show()
  File "/usr/local/lib/python3.6/dist-packages/fastprogress/fastprogress.py", line 167, in show
    self.html_code = '\n'.join([getattr(self.inner_dict[n], 'progress', self.inner_dict[n]) for n in to_show])
TypeError: sequence item 0: expected str instance, NBProgressBar found
```

## Expected behavior

Evaluation of test.txt

## Environment info

- `transformers` version: 2.6.0
- Platform: Google  Colab
- Python version: 3.6
- PyTorch version (GPU?): -
- Tensorflow version (GPU?): 2.2.0rc1
- Using GPU in script?: -
- Using distributed or parallel set-up in script?: using example setup


[T5] fix lm lables in docstring

Add better explanation to T5 `lm_labels` dosctring.


Unexpected ZeroDivisionError when calling model.prune_heads

# 🐛 Bug
Traceback (most recent call last):
  File "/Users/user/anaconda3/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/Users/user/anaconda3/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/user/Desktop/org/text_models_branches/section_extraction/text_models/bert_text_classification/train_bert_ml_mc.py", line 609, in <module>
    masking_amount=args.masking_amount,
  File "/Users/user/Desktop/org/text_models_branches/section_extraction/text_models/bert_text_classification/train_bert_ml_mc.py", line 288, in train_model
    local_rank=transformer_args["local_rank"],
  File "/Users/user/Desktop/org/text_models_branches/section_extraction/text_models/bert_text_classification/prune_attention_heads.py", line 488, in prune_model_and_return
    metric=metric,
  File "/Users/user/Desktop/org/text_models_branches/section_extraction/text_models/bert_text_classification/prune_attention_heads.py", line 396, in prune_heads
    model.prune_heads(heads_to_prune)
  File "/Users/user/Desktop/org/text_models_branches/section_extraction/text_models/env/lib/python3.7/site-packages/transformers/modeling_utils.py", line 234, in prune_heads
    self.base_model._prune_heads(heads_to_prune)
  File "/Users/user/Desktop/org/text_models_branches/section_extraction/text_models/env/lib/python3.7/site-packages/transformers/modeling_bert.py", line 635, in _prune_heads
    self.encoder.layer[layer].attention.prune_heads(heads)
  File "/Users/user/Desktop/org/text_models_branches/section_extraction/text_models/env/lib/python3.7/site-packages/transformers/modeling_bert.py", line 301, in prune_heads
    self.output.dense = prune_linear_layer(self.output.dense, index, dim=1)
  File "/Users/user/Desktop/org/text_models_branches/section_extraction/text_models/env/lib/python3.7/site-packages/transformers/modeling_utils.py", line 824, in prune_linear_layer
    new_layer = nn.Linear(new_size[1], new_size[0], bias=layer.bias is not None).to(layer.weight.device)
  File "/Users/user/Desktop/org/text_models_branches/section_extraction/text_models/env/lib/python3.7/site-packages/torch/nn/modules/linear.py", line 81, in __init__
    self.reset_parameters()
  File "/Users/user/Desktop/org/text_models_branches/section_extraction/text_models/env/lib/python3.7/site-packages/torch/nn/modules/linear.py", line 84, in reset_parameters
    init.kaiming_uniform_(self.weight, a=math.sqrt(5))
  File "/Users/user/Desktop/org/text_models_branches/section_extraction/text_models/env/lib/python3.7/site-packages/torch/nn/init.py", line 325, in kaiming_uniform_
    std = gain / math.sqrt(fan)
ZeroDivisionError: float division by zero

Basically, the error is being thrown on calling model.prune_heads(heads_to_prune). This error is not coming every time I run the script. Not sure what is causing the error.

## Information

Model I am using (Bert, XLNet ...): BERT

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below) - No
* [x] my own modified scripts: (give details below) - Yes
My script calls mask_heads and then prune_heads similar to the original bertology script

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)
A multi-class classification task on a proprietary dataset

## To reproduce

Steps to reproduce the behavior:

Still don't know as the error seems to be unexpected. I don't get it every time I run the script.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
model.prune_heads(heads_to_prune) where heads_to_prune -> Dict[int, List] where key is the layer number and value is the list of heads to prune (Calculated by calling the mask_heads function). Expected behaviour is pruning off heads present in heads_to_prune for each layer

## Environment info
<!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.2.2
- Platform: macOS Catalina 10.15
- Python version: Python 3.7.3
- PyTorch version (GPU?): 1.3.1 (No GPU)
- Tensorflow version (GPU?): 1.14.0
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


Bart.generate requires config.output_past=True

Is there a way to generate using pre-trained BART like one in
https://huggingface.co/blog/how-to-generate

I am currently using BART for a generation task but finetuning it
I was wondering if it's possible to see generation result from pre-trained BART


bug in run_glue.py

# 🐛 Bug

## Information

Model I am using (FlauBert ...):

Language I am using the model on (French ...):

The problem arises when using:
* [x] the official example scripts: (give details below)

the script "run_glue" (I did not modified anything) I took it as it is

The tasks I am working on is:
* [x] an official FLUE task: (CLS = classification tast ) :  finetuning model

## To reproduce

Steps to reproduce the behavior:

1. Just run the script run_glue.py from a bash script following commands in the FlauBert tuto (https://github.com/getalp/Flaubert/tree/master/flue).

config='flue/examples/cls_books_lr5e6_hf_base_cased.cfg'
source $config
python ~/transformers/examples/run_flue.py \
                                        --data_dir $data_dir \
                                        --model_type flaubert \
                                        --model_name_or_path $model_name_or_path \
                                        --task_name $task_name \
                                        --output_dir $output_dir \
                                        --max_seq_length 512 \
                                        --do_train \
                                        --do_eval \
                                        --learning_rate $lr \
                                        --num_train_epochs $epochs \
                                        --save_steps $save_steps \
                                        --fp16 \
                                        --fp16_opt_level O1 \
                                        |& tee output.log

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

`

03/27/2020 11:54:48 - WARNING - main - Process rank: -1, device: cuda, n_gpu: 2, distributed training: False, 16-bits training: True
Traceback (most recent call last):
File "/home/getalp/kelodjoe/transformers/examples/run_glue.py", line 693, in
main()
File "/home/getalp/kelodjoe/transformers/examples/run_glue.py", line 613, in main
config_class, model_class, tokenizer_class = MODEL_CLASSES[args.model_type]
KeyError: 'flaubert'

`

Code of line 613

`

    # Training
    if args.do_train:
        train_dataset = load_and_cache_examples(args, args.task_name, tokenizer, evaluate=False)
        global_step, tr_loss = train(args, train_dataset, model, tokenizer)
        logger.info(" global_step = %s, average loss = %s", global_step, tr_loss)

    # Saving best-practices: if you use defaults names for the model, you can reload it using from_pretrained()  #613
    if args.do_train and (args.local_rank == -1 or torch.distributed.get_rank() == 0):
        # Create output directory if needed
        if not os.path.exists(args.output_dir) and args.local_rank in [-1, 0]:
            os.makedirs(args.output_dir)

        logger.info("Saving model checkpoint to %s", args.output_dir)
        # Save a trained model, configuration and tokenizer using `save_pretrained()`.
        # They can then be reloaded using `from_pretrained()`
        model_to_save = (
            model.module if hasattr(model, "module") else model
        )  # Take care of distributed/parallel training
        model_to_save.save_pretrained(args.output_dir)
        tokenizer.save_pretrained(args.output_dir)
`

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

I would Have expected it to work. and train the model.

## Environment info
<!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:   => how do I know I git pull before running the script
- Platform: linux
- Python version: 3.6
- PyTorch version (GPU?): 1.4
- Tensorflow version (GPU?):
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: no


Issue loading custom tokenizer for fine-tuning gpt2

Core: Tokenization

Internals of the library; Tokenization.

I'm trying to fine-tune gpt2 with a custom tokenizer. It was working fine just over 10 days ago, with  --tokenizer_name=/path to vocab and merges folder/  and now it cannot load, asking to check if it's a correct model identifier or contains a config.json file. As if instead of a tokenizer it is now trying to load a model? It also asked for an extra model identifier in the config file of my model, which before was not required.

I suppose there was a library update? What would be the workaround? Thanks in advance.


Add shoarora/electra and alectra model cards

model card

Related to pretrained model cards

Add model cards for recently uploaded models:

- shoarora/electra-small-owt (BERT)
- shoarora/alectra-small-owt (ALBERT)


Why GPT2 train loss and topK accuracy both decrease?

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->

## Details
<!-- Description of your issue -->
Hi,

I use GPT2 LMHeadModel training from scratch. The training loss decreases, however, I test it on the same train dataset, I get top3 accuracy decreasing as well. Moreover, is it normal that my top3 accuracy goes down sharply like from 0.2 to 0.05 only one or two epochs? Because it seems to be stable when it converges. Did anyone meet the same problem?

![loss](https://user-images.githubusercontent.com/62126666/77870461-14441380-7274-11ea-9406-c8c56bb8143f.jpg)

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:


why isn't AlbertForMultipulChioce in modeling_albert?

Core: Modeling

Internals of the library; Models.

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->

## Details
<!-- Description of your issue -->
I just copy code from RobertaForMultipleChoice to modeling_albert and change all 'roberta' to 'albert', but the loss didn't goes dowm evidently. And the result is even worse than articles.
<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:


[T5] make decoder input ids optional for t5 training

- [x] Make `decoder_input_ids` optional when supplying `lm_labels` for `T5ForConditionalGeneration`
- [x] Add test


WIP: haiku bert implementation

Still a work in progress but the contextual embeddings line up with the pytorch version so this is roughly at parity with jax-bert

TODO (mostly notes to myself):
- [x] Add `save_pretrained`
- [ ] Make `from_pretrained` work with names
- [ ] Add dropout at training time, pass through training flag
- [ ] Make sure weight initializations line up when pre-trained state isn't passed
- [ ] Gradually work towards parity with the pytorch version if desired? (target models, BERT variants, etc.)
- [ ] Write HaikuPretrainedModel to take advantage of archive resolution / make saving + loading compatible with pytorch bins?

To use the pre-trained weights cleanly I ended up subclassing `hk.Module` -- unsure how I feel about this decision but I couldn't think of a better method at the time.  Feel free to suggest an alternative if you have ideas.


This bug will rise when you add new token types in the Tokenizer.
Since the resizing was done **after** passing the params to the optimizer, the wrong set of params for the embedding table were optimized.


Argument “never_split” not working on bert tokenizer

Core: Tokenization

Internals of the library; Tokenization.

I used the ```never_split``` option and tried to retain some tokens. But the tokenizer still divide them into wordpieces.

```
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased', never_split=['lol'])
tokenizer.tokenize("lol That's funny")
['lo', '##l', 'that', "'", 's', 'funny']
```
**A link to original question on Stack Overflow**:
https://stackoverflow.com/posts/60914793/edit


[Tokenization] fix edge case for bert tokenization

This PR fixes #3502 .

The reason why the tests fail in #3502 is because of an edge case.

If the input to `tokenizer.batch_encode_plus()` consists of a tokenized string that results in a list of exactly two strings (``[[16], [.]]`` in issue #3502) then it is treated as a pair of input sequences (=> [CLS] input_sequence_1 [SEP] input_sequence_2 [SEP]) but this behavior should only happen if the input list consists of two **untokenized** strings.


[Docs] examples/summarization/bart: Simplify CNN/DM preprocessing steps

I added the preprocessed data in S3.

Evidence that it is the correct size:
![image](https://user-images.githubusercontent.com/6045025/77852114-d29b6600-71aa-11ea-979c-1547007a0d28.png)


Isort installed from github branch does not correspond to circle ci isort

# 🐛 Bug

## Information

Installing isort via:

`$ pip install -U git+git://github.com/timothycrosley/isort.git@e63ae06ec7d70b06df9e528357650281a3d3ec22#egg=isort`

does not correspond to the circle ci version anymore, so that `make style` formatting leads to circle-ci make the code quality test fail.

- `transformers` version: 2.6.0
- Platform: Linux-5.3.0-42-generic-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.4.0+cpu (False)
- Tensorflow version (GPU?): 2.1.0 (False)


[Examples] Clean summarization and translation example testing files for T5 and Bart

- Only create temporary files instead of "real" files so that each `test_file` in `examples/summarization` and `examples/translations` creates unique testing output files that cannot be overridden by other test files.


Adding mbart-large-cc25

translation

machine translation utilities and models

# 🌟 New model addition
Multilingual BART model implemented in fairseq introduced by FAIR

## Model description
This issue is to request adding mBART model existing as a part of fairseq lib.

[Link to the fairseq description of the model](https://github.com/pytorch/fairseq/tree/master/examples/mbart
)
[Link to the mBART paper](https://arxiv.org/abs/2001.08210)

Multilingually pretrained BART checkpoint.

<!-- Important information -->
The model code follows the original BART model code which is already a part of ```transformers``` repo. However, it introduces a couple more features like multilingual denoising and translation from pretrained BART.

## Open source status

- [x] _the model implementation is available: (give details)_
[Link to the PR adding mBART to the fairseq](https://github.com/pytorch/fairseq/commit/5e79322b3a4a9e9a11525377d3dda7ac520b921c)
This PR shows the main pieces that were added to the fairseq to make mBART work considering BART  which is already existing in the codebase. However, a few additional mBART commits were added afterward.

- [x] _the model weights are available: (give details)_
[Link to the weights](https://github.com/pytorch/fairseq/tree/master/examples/mbart#pre-trained-models)

- [x] _who are the authors: (mention them, if possible by @gh-username)_
Facebook AI Research (@MultiPath)


# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->

## Details
<!-- Description of your issue -->
how to get activation weights of a pretrained model?

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:


Update the NER TF script

PR to remove the softmax and make the pad token label id to -1.


reproducing the performance of XLM-ROBERTA on MLQA dataset on the zh language

# ❓ Questions & Help
## Details
<!-- Description of your issue -->

I have trouble in reproducing the result of XLM-ROBERTA on MLQA dataset for Chinese languages. The result of the rest of the language seems fine, however on the zh section, I have very low  {'exact_match': 4.263188631496982, 'f1': 17.451059178461946})!


Fix for continuing training

If `args.should_continue` is true, then there is no way in the current example to reload a checkpoint since it is assumed it exists within the output_dir. Checking here fixes everything as expected.


[Bart] when output_paste=False BartForConditionalGeneration raises confusing error

# 🐛 Bug

## Information

i am using BART

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [x ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

Summarization

## To reproduce

Steps to reproduce the behavior:

```python
tokenizer = BartTokenizer.from_pretrained('bart-large-mnli')
model = BartForConditionalGeneration.from_pretrained('bart-large-mnli')

article_input_ids = tokenizer.batch_encode_plus([LONG_BORING_TENNIS_ARTICLE], return_tensors='pt', max_length=1024)['input_ids'].to(torch_device)
summary_ids = model.generate(article_input_ids,
                             num_beams=4,
                             length_penalty=2.0,
                             max_length=100,
                             early_stopping=True)

print([tokenizer.decode(g, skip_special_tokens=True, clean_up_tokenization_spaces=False) for g in summary_ids])
```

i get error

```
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-27-2df1c6607426> in <module>()
      4                              length_penalty=2.0,
      5                              max_length=100,
----> 6                              early_stopping=True)
      7
      8 print([tokenizer.decode(g, skip_special_tokens=True, clean_up_tokenization_spaces=False) for g in summary_ids])

3 frames
/usr/local/lib/python3.6/dist-packages/transformers/modeling_bart.py in _reorder_cache(past, beam_idx)
    921     @staticmethod
    922     def _reorder_cache(past, beam_idx):
--> 923         ((enc_out, enc_mask), decoder_cached_states) = past
    924         reordered_past = []
    925         for layer_past in decoder_cached_states:

ValueError: too many values to unpack (expected 2)
```

this works with bart-large-cnn
but gives error with other models?


[T5] Add training documenation

- Fixes T5 docstring regarding pretraining
- Add detailed description on how to process input and target for T5 training


No grad feature in model parameters

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->

## Details
<!-- Description of your issue -->
I wonder if parameters in model.named_parameters() have the feature grad? If none, how can I add to it ?

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:


Add clear description of how to train T5


[Docs] Update usage doc regarding generate fn

Update `model.generate()` docs


Distil-BART?

Distillation

Related to model distillation


Bert Batch Encode Plus adding an extra [SEP]

# 🐛 Bug

## Information

I'm using `bert-base-multilingual-cased` tokenizer and model for creating another model. However, the `batch_encode_plus` is adding an extra `[SEP]` token id in the middle.

The problem arises when using:
* Specific strings to encode, e.g. `16.`, `3.`, `10.`,
* The `bert-base-multilingual-cased` tokenizer is used beforehand to tokenize the previously described strings and
* The `batch_encode_plus` is used to convert the tokenized strings

In fact, `batch_encode_plus` will generate an `input_ids` list containing two `[SEP]`, such as in `[101, 10250, 102, 119, 102]`

I have seen similar issues, but they don't indicate the version of transformers:

https://github.com/huggingface/transformers/issues/2658
https://github.com/huggingface/transformers/issues/3037

Thus, I'm not sure if it is related to transformers version `2.6.0`

## To reproduce

Steps to reproduce the behavior (simplified steps):

1. Have a string of type `16.` or `6.`
2. Use `tokens = bert_tokenizer.tokenize("16.")`
3. Use `bert_tokenizer.batch_encode_plus([tokens])`

You can reproduce the error with this code

```python
from transformers import BertTokenizer
import unittest

class TestListElements(unittest.TestCase):

    def setUp(self):

        bert_tokenizer = BertTokenizer.from_pretrained('bert-base-multilingual-cased')

        problematic_string = "16."

        tokens = bert_tokenizer.tokenize(problematic_string)

        self.encoded_batch_1 = bert_tokenizer.batch_encode_plus([tokens])    #list[list[]]
        self.encoded_batch_2 = bert_tokenizer.batch_encode_plus([problematic_string]) #list[]
        self.encoded_tokens_1 = bert_tokenizer.encode_plus(problematic_string)
        self.encoded_tokens_2 = bert_tokenizer.encode_plus(tokens)

    def test_tokens_vs_tokens(self):
        self.assertListEqual(self.encoded_tokens_1["input_ids"], self.encoded_tokens_2["input_ids"])

    def test_tokens_vs_batch_string(self):
        self.assertListEqual(self.encoded_tokens_1["input_ids"], self.encoded_batch_2["input_ids"][0])

    def test_tokens_vs_batch_list_tokens(self):
        self.assertListEqual(self.encoded_tokens_1["input_ids"], self.encoded_batch_1["input_ids"][0])

if __name__ == "__main__":
    unittest.main(verbosity=2)
```
The code will break at test `test_tokens_vs_batch_list_tokens`, with the following summarized output:

```
- [101, 10250, 119, 102]
+ [101, 10250, 102, 119, 102]
```

## Expected behavior

The `batch_encode_plus` should always produce the same `input_ids` no matter whether we pass them a list of tokens or a list of strings.

For instance, for the string `16.` we should get always `[101, 10250, 119, 102]`. However, using `batch_encode_plus` we get `[101, 10250, 102, 119, 102]` if we pass them an input already tokenized.

## Environment info

- `transformers` version: 2.6.0
- Platform: Linux (Manjaro)
- Python version: Python 3.8.1 (default, Jan  8 2020, 22:29:32)
- PyTorch version (GPU?): 1.4.0 (True)
- Tensorflow version (GPU?): ---
- Using GPU in script?: False
- Using distributed or parallel set-up in script?: False


[BART] Update encoder and decoder on set_input_embedding

Since  _resize_token_embeddings will create a new embedding layer,
resizing the input embeddings for BART will currently break, as the
model.shared will refer to the new embedding created but the
model.encoder.embed_tokens and the model.decoder.embed_tokens will
still refer to the old embedding created.

We need to re-assign the encoder/decoder or just their weights, I opted for the second option.
Unfortunately can't see how to write a test in the test_resize_tokens_embeddings to capture this without putting a BART-specific if statement there, but this is also related to https://github.com/huggingface/transformers/issues/3378

Run tests:
 733 passed, 319 skipped, 80 warnings in 269.62s (0:04:29)


[Wait to merge] [Bart] Rename lm_labels argument to masked_lm_labels

While the docstring correctly lists masked_lm_labels, forward
was expecting lm_labels. Renaming to match BERT and the rest of
the MLM-based models


masked_lm_loss in BertForMaskedLM model

Altought I've read the documentation related to BertForMaskedLM class, I still cannot understand how to properly calculate loss for my problem.

Let's suppose that my target sentence is:
"_I will be writing when you arrive._"
I want to calculate loss for all words except 'arrive'.

The documentation says:
> **masked_lm_labels (torch.LongTensor of shape (batch_size, sequence_length), optional, defaults to None)** Labels for computing the masked language modeling loss. Indices should be in [-100, 0, ..., config.vocab_size] (see input_ids docstring) Tokens with indices set to -100 are ignored (masked), the loss is only computed for the tokens with labels in [0, ..., config.vocab_size]

Due to the way I understood, I should pass to the _masked_lm_labels_ argument a tensor that contains following indices:
`tensor([[ 101, 1045, 2097, 2022, 3015, 2043, -100, 7180, 1012,  101]])`
It returns error:
`RuntimeError: Assertion 'cur_target >= 0 && cur_target < n_classes' failed.`

Can you help me and point out what is wrong in my thinking?


XLM-ROBERTA

Core: Modeling

Internals of the library; Models.

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...):  XLM-ROBERTA

In order to fine-tune XLM-ROBERTA from pre_trained file, since the config file does not have the model_type field, all the required files should be in a folder which contains "xlm-robert" in its name! otherwise the "XLM" config will be assigned !