# ❓ Transformers' trainer sequence classification problem
## Details

I wanted to use `XLMRobertaForSequenceClassification` to classify a sequence into `1` or `0`.

```python
MODEL_NAME = 'xlm-roberta-base'

def multilingual_model(max_seq_length=SEQUENCE_LENGTH, trainable=False):
    """Build and return a multilingual BERT model and tokenizer."""
    model = XLMRobertaForSequenceClassification.from_pretrained(
        MODEL_NAME,
        num_labels = 2,
        output_attentions = False,
        output_hidden_states = False,
    )
    return model
```

The trainer is

```python
from transformers import Trainer

model = multilingual_model()

trainer = Trainer(
        model=model,
        args=training_args,
        train_dataset=part_train_dataset,
        eval_dataset=part_valid_dataset,
        compute_metrics=compute_metrics)
```

`training_args`
```python
from transformers import TrainingArguments
BATCH_SIZE = 32
DEVICE = torch.device("cpu")
training_args = TrainingArguments("/kaggle/working")
training_args.do_train = True
training_args.evaluate_during_training = True
training_args.adam_epsilon = 1e-8
training_args.learning_rate = 1e-5
training_args.per_gpu_train_batch_size = BATCH_SIZE
training_args.num_train_epochs=TRAIN_EPOCH
```

`compute_metrics`
```python
from transformers import EvalPrediction
from typing import Dict
import numpy as np

def compute_metrics(p: EvalPrediction) -> Dict:
    preds = np.argmax(p.predictions, axis=1)
    return metrics.roc_auc_score(preds, p.label_ids)
```

An exerpt of `part_train_dataset`
```
[InputFeatures(input_ids=[0, 99070, 1159, 11050, 8108, 398, 6244, 7, 10932, 98, 759, 4488, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], attention_mask=[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], token_type_ids=None, label=1),
 InputFeatures(input_ids=[0, 28192, 2367, 83, 442, 22120, 2367, 83, 442, 142, 97629, 21115, 111, 3060, 102172, 20397, 761, 7, 2750, 621, 4127, 99, 163684, 214, 15970, 6, 140545, 297, 7398, 1419, 2750, 2], attention_mask=[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], token_type_ids=None, label=1)
```

Similarly, one of `part_valid_dataset`:
```
[InputFeatures(input_ids=[0, 99070, 1159, 11050, 8108, 398, 6244, 7, 10932, 98, 759, 4488, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], attention_mask=[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], token_type_ids=None, label=1),
 InputFeatures(input_ids=[0, 28192, 2367, 83, 442, 22120, 2367, 83, 442, 142, 97629, 21115, 111, 3060, 102172, 20397, 761, 7, 2750, 621, 4127, 99, 163684, 214, 15970, 6, 140545, 297, 7398, 1419, 2750, 2], attention_mask=[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], token_type_ids=None, label=1),
```

When running `trainer.train()`, I received the following error:
```
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-11-3435b262f1ae> in <module>
----> 1 trainer.train()

/opt/conda/lib/python3.7/site-packages/transformers/trainer.py in train(self, model_path)
    380                     continue
    381
--> 382                 tr_loss += self._training_step(model, inputs, optimizer)
    383
    384                 if (step + 1) % self.args.gradient_accumulation_steps == 0 or (

/opt/conda/lib/python3.7/site-packages/transformers/trainer.py in _training_step(self, model, inputs, optimizer)
    465             inputs[k] = v.to(self.args.device)
    466
--> 467         outputs = model(**inputs)
    468         loss = outputs[0]  # model outputs are always tuple in transformers (see doc)
    469

/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    548             result = self._slow_forward(*input, **kwargs)
    549         else:
--> 550             result = self.forward(*input, **kwargs)
    551         for hook in self._forward_hooks.values():
    552             hook_result = hook(self, input, result)

/opt/conda/lib/python3.7/site-packages/transformers/modeling_roberta.py in forward(self, input_ids, attention_mask, token_type_ids, position_ids, head_mask, inputs_embeds, labels)
    355             else:
    356                 loss_fct = CrossEntropyLoss()
--> 357                 loss = loss_fct(logits.view(-1, self.num_labels), labels.view(-1))
    358             outputs = (loss,) + outputs
    359

/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    548             result = self._slow_forward(*input, **kwargs)
    549         else:
--> 550             result = self.forward(*input, **kwargs)
    551         for hook in self._forward_hooks.values():
    552             hook_result = hook(self, input, result)

/opt/conda/lib/python3.7/site-packages/torch/nn/modules/loss.py in forward(self, input, target)
    930     def forward(self, input, target):
    931         return F.cross_entropy(input, target, weight=self.weight,
--> 932                                ignore_index=self.ignore_index, reduction=self.reduction)
    933
    934

/opt/conda/lib/python3.7/site-packages/torch/nn/functional.py in cross_entropy(input, target, weight, size_average, ignore_index, reduce, reduction)
   2315     if size_average is not None or reduce is not None:
   2316         reduction = _Reduction.legacy_get_string(size_average, reduce)
-> 2317     return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
   2318
   2319

/opt/conda/lib/python3.7/site-packages/torch/nn/functional.py in nll_loss(input, target, weight, size_average, ignore_index, reduce, reduction)
   2113                          .format(input.size(0), target.size(0)))
   2114     if dim == 2:
-> 2115         ret = torch._C._nn.nll_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
   2116     elif dim == 4:
   2117         ret = torch._C._nn.nll_loss2d(input, target, weight, _Reduction.get_enum(reduction), ignore_index)

RuntimeError: expected scalar type Long but found Float
```

which does not exist if `num_labels` is 1. From `transformers`'s github, it seems that 2 labels is standard for binary classification.

Beside how to fix the error, I wanted to ask why there are zeroes in `attention_mask` in `part_train`/`valid_dataset`

[**Link to original question on Stack Overflow**:](https://stackoverflow.com/questions/61987904/transformers-trainer-sequence-classification-problem)


Typo in GPT2 documentation

# 🐛 Bug

In the GPT2 documentation [page](https://huggingface.co/transformers/model_doc/gpt2.html#gpt2lmheadmodel), in the parameters of both LMHead and DoubleHeads model, 'inputs_embeds' is registered as 'input_embeds' which leads to an error upon implementation.


# 🐛 Bug

hi, thanks for your nice tool for NLP.
However, when I use transformers on MacOs to load Electra model I get an import error

> ImportError: cannot import name 'TFElectraModel' from 'transformers'

How can I fix this issue?


Model card: Updated the link to the paper

model card

Related to pretrained model cards

The conference has changed the link to the paper, so I updated it.


bert embedding make OOM in albert

# ❓ Questions & Help

## Details
hi, i made item collaborative filtering model by albert model.
i set vocab size to 600k, it made OOM.
(after initialized model, memory used 3 GB. but During initialization, it takes 11 GB)

i tracked source code and i found this:
albert model initialize albert word embedding layer after initializing bert word embedding layer.
embedding layer is initialized twice. so it made OOM.
- bert word embedding : vocab size x hidden layer size(4048)
- albert word embedding : vocab size x embedding size(128) => bert word embedding free

is there any problem if i fix AlbertEmbeddings to nn.Module?

thanks

`
class AlbertEmbeddings(BertEmbeddings):
    """
    Construct the embeddings from word, position and token_type embeddings.
    """

    def __init__(self, config):
        super().__init__(config)

        self.word_embeddings = nn.Embedding(config.vocab_size, config.embedding_size, padding_idx=config.pad_token_id)
        self.position_embeddings = nn.Embedding(config.max_position_embeddings, config.embedding_size)
        self.token_type_embeddings = nn.Embedding(config.type_vocab_size, config.embedding_size)
        self.LayerNorm = torch.nn.LayerNorm(config.embedding_size, eps=config.layer_norm_eps)

`
`
class BertEmbeddings(nn.Module):
    """Construct the embeddings from word, position and token_type embeddings.
    """

    def __init__(self, config):
        super().__init__()
        self.word_embeddings = nn.Embedding(config.vocab_size, config.hidden_size, padding_idx=config.pad_token_id)
        self.position_embeddings = nn.Embedding(config.max_position_embeddings, config.hidden_size)
        self.token_type_embeddings = nn.Embedding(config.type_vocab_size, config.hidden_size)

        # self.LayerNorm is not snake-cased to stick with TensorFlow model variable name and be able to load
        # any TensorFlow checkpoint file
        self.LayerNorm = BertLayerNorm(config.hidden_size, eps=config.layer_norm_eps)
        self.dropout = nn.Dropout(config.hidden_dropout_prob)
`


❓ [BART] Different embedding sizes between pre-trained / fine-tuned checkpoint

# ❓ Questions & Help

Running this code :

```python
from transformers import BartModel

x = BartModel.from_pretrained('bart-large')
x2 = BartModel.from_pretrained('bart-large-cnn')
print(x.shared)
print(x2.shared)
```

Gives :

>Embedding(50265, 1024, padding_idx=1)
Embedding(50264, 1024, padding_idx=1)

---

Why the vocabulary size is different ? Isn't it supposed to be the same ? Is it just from the original authors' checkpoint ?

@sshleifer


❓ [BART] Why using bias for LM head if not trained ?

# ❓ Questions & Help

As I understood, BART is not using a regular Linear layer as LM head, but instead reuse the weights of the shared embeddings.

As show here, biases are added :
https://github.com/huggingface/transformers/blob/a34a9896ac2a4a33ff9cd805c76eed914c8d8965/src/transformers/modeling_bart.py#L876

But these biases are registered as buffer, not as parameter. **Since they are not trained, they will always stay 0 ?**

If they stay 0, what's the point of having bias at all ?

@sshleifer


variable name changes for Issue #4141

Hi

Let me know if additional changes are required for Issue #4141. Thank you for this awesome repository.


changing config.axial_pos_shape for 'ReformerModelWithLMHead' when fine-tuning

reformer

Everything related to the reformer model

# ❓ Questions & Help

I'm trying to fine-tune the Reformer for the language generation task and I padded the sequence lengths to be a multiple of least common multiple chunk_length 64, and now I'm asked to pad the sequence to 524288(512 * 1024), which will give me an out of memory error.

I would like to know a workaround for this, since the error message also gives an alternative to 'pad_to_max_length', which is 'changing config.axial_pos_shape' and specially since this is known to be a memory efficient transformer. Thank you.
**A link to original question on Stack Overflow**: [https://stackoverflow.com/questions/61986452/fine-tuning-reformer-gives-out-of-memory-error-when-sequence-length-is-padded-t](https://stackoverflow.com/questions/61986452/fine-tuning-reformer-gives-out-of-memory-error-when-sequence-length-is-padded-t)


[Reformer] fix reformer num buckets

Fix automatic setting of `num_buckets` by making sure `num_buckets` is always a power of 2 and set as default.

The idea behind this whole function is that `num_buckets` should not be set by the user, but calculated on the fly to a good value before training (`num_buckets` ~ 2 * sequence length / chunk length as recommend in the paper). This value will then be saved in the config and can be reapplied for inference.


Decoding with DistilmBERT to generate text in different languages

Good day and congrats for your great library

If I want to decode and get new generated text with the GPT2 heads, that works great like you suggest:

```py
tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
input_ids = torch.tensor(tokenizer.encode("Once upon a time there was")).unsqueeze(0)
model = GPT2LMHeadModel.from_pretrained("gpt2", pad_token_id=tokenizer.eos_token_id)
greedy_output = model.generate(input_ids, max_length=50)
print("Output:\n" + 100 * '-')
print(tokenizer.decode(greedy_output[0], skip_special_tokens=True))
```

but my issue is that now I want to do the same but with the smaller simpler DistilmBERT model which is also multilingual in 104 languages, so I want to generate text in for example Spanish and English and with this lighter model. So I do this:

```py
tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-multilingual-cased')
model = DistilBertForMaskedLM.from_pretrained('distilbert-base-multilingual-cased')
input_ids = torch.tensor(tokenizer.encode("Hello, my dog is cute", add_special_tokens=True)).unsqueeze(0)  # Batch size 1
outputs = model(input_ids, masked_lm_labels=input_ids)
loss, prediction_scores = outputs[:2]
```

but now, how do I get the continuation of the phrase at that point? I tried to apply tokenizer.decode with no luck there, thank you


implementation of transformers for abstractive summarization task

Hello, I am new to whole NLP world and PyTorch. I am trying to learn the concepts and that is taking some time for a rookie. I have a project to finish and I want to implement transformers & BERT on my abstractive summarization project. I tried to find some implementation tutorial on this topic but I could not find any. Do you guys have any suggestions about clear implementation of any pre-trained model that I can fine-tune on my dataset to get some solid results. I am not looking for this just for finishing up the project but also learn how to implement. Therefore, I need a clear tutorial.

Data:
I am using 0.25 of XSum dataset so I have 45k news and their one-sentence summary.

Thank you in advance.


Fix the example command for SQuAD

Issue #4549: added the missing argument `--do_lower_case` for reproducing the intended results.


Albert Tokenizer hangs

Core: Tokenization

Internals of the library; Tokenization.

# 🐛 Bug

## Information
I am following the language modeling tutorial to train a LM on a simple wikipedia corpus from scratch. I am trying to use Albert instead of Roberta. As I couldn't find information on how to train an Albert Tokenizer from scratch, I'm loading the albert-base-v2 tokenizer. The Dataset creation doesn't work, it hangs for ages and when I stop it, I can see that it is always stuck in tokenization_albert.py, line 193:

```python
outputs = "".join([c for c in outputs if not unicodedata.combining(c)])
```

A week ago, it crashed consistently in this line due to large RAM allocations, but I can't reproduce that behaviour right now.

## To reproduce

Steps to reproduce the behavior:
```python
from transformers import AlbertTokenizer
tokenizer = AlbertTokenizer.from_pretrained("albert-base-v2")
from transformers import TextDataset
train_set = TextDataset(
    tokenizer=tokenizer,
    file_path="./drive/My Drive/datasets/simplewiki_train.txt",
    block_size=128,
)
```
## Expected behavior

I expected the tokenizer to run through in less than an hour for 100MB input.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.10.0
- Platform: Colab
- Python version: 3.6

Is anyone else experiencing this? I read in another issue that Albert should work with run_language_modeling out of the box.


XLnet loss and accuracy not decreasing

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): XLnet base cased

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)
Can check the notebook below:
https://colab.research.google.com/drive/132r5kb1G5oG0yi-qnymBsMBPCGP5Gu85
Can only give access to a few people.

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)
Binary classification.
## To reproduce

Steps to reproduce the behavior:
Preprocessing:
```
from transformers import XLNetTokenizer
from keras.preprocessing.sequence import pad_sequences
tokenizer = XLNetTokenizer.from_pretrained('xlnet-base-cased')

 encoded_sent = tokenizer.encode(
             text, # Sentence to encode.
             add_special_tokens = True,
             )
MAX_LEN = 128
encoded_sent = pad_sequences([encoded_sent], maxlen=MAX_LEN, dtype="long",
         value=0, truncating="post", padding="post")
attention_masks=[]
att_mask = [int(token_id > 0) for token_id in encoded_sent[0]]
attention_masks.append(att_mask)
```

Model definition:
```
from transformers import BertForSequenceClassification, XLNetForSequenceClassification, AdamW, BertConfig

# bert = BertForSequenceClassification.from_pretrained(
#  "bert-base-uncased",
#  num_labels = 2, .
#  output_attentions = False,
#  output_hidden_states = False,

xlnet = XLNetForSequenceClassification.from_pretrained('xlnet-base-cased',
num_labels = 2,
output_attentions = False,
output_hidden_states = False,
)
class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        self.features2 = xlnet
        self.softmax = nn.LogSoftmax(dim=1)
    def forward(self, x2, x3):
        x2 = x2.to(torch.int64)
        x2 = self.features2(x2,x3)[0]
        x = self.softmax(x2)
        return x

model = MyModel()
torch.cuda.empty_cache()
model.to('cuda')
criterion = nn.CrossEntropyLoss()

# Observe that all parameters are being optimized
optimizer = optim.AdamW(model.parameters(), lr=0.0005)

# Decay LR by a factor of 0.1 every 7 epochs
exp_lr_scheduler = lr_scheduler.StepLR(optimizer, step_size=5, gamma=0.1)
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
Loss and accuracy should decrease but are not changing at all( both training and valid).
This script worked while training BERT model.
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: Latest
- Platform: Colab
- Python version: 3.6
- PyTorch version (GPU?): Latest
- Tensorflow version (GPU?):
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No


Add DistilBERT to supported run_language_modeling models

As per the code, distilbert is indeed supported by the `run_language_modeling.py` script, even though the README states otherwise.


Cleaner warning when loading pretrained models

Give more explicit logging messages when using the various `from_pretrained` methods in the lib.
Also makes these messages as `logging.warning` because it's a common source of silent mistakes.

cc @BramVanroy

Happy to improve the language further if people have advice.


Added reference to use for citing this model

model card

Related to pretrained model cards


Example script for SQuAD question answering unable to reproduce the claimed performance

# 🐛 Bug

## Information

The example script for SQuAD question answering (`examples/question-answering/run-squad.py`) fails to produce the correct results as claimed in the tutorial.
The correct performance is around f1 = 88.52, exact_match = 81.22 on SQuAD v1.1, but the script produces f1 = 81.97 and exact match = 73.80 instead.

## To reproduce

Steps to reproduce the behavior:

1. Install with the latest commit (a34a989)
2. Download the SQuAD v1.1 dataset.
3. Run `examples/question-answering/run-squad.py`. with the exact same arguments as seen in the tutorial.

```
export SQUAD_DIR=/path/to/SQUAD

python run_squad.py \
  --model_type bert \
  --model_name_or_path bert-base-uncased \
  --do_train \
  --do_eval \
  --train_file $SQUAD_DIR/train-v1.1.json \
  --predict_file $SQUAD_DIR/dev-v1.1.json \
  --per_gpu_train_batch_size 12 \
  --learning_rate 3e-5 \
  --num_train_epochs 2.0 \
  --max_seq_length 384 \
  --doc_stride 128 \
  --output_dir /tmp/debug_squad/
```

Following is the final result.

05/24/2020 16:10:09 - INFO - __main__ -   ***** Running evaluation  *****
05/24/2020 16:10:09 - INFO - __main__ -     Num examples = 10789
05/24/2020 16:10:09 - INFO - __main__ -     Batch size = 8
Evaluating: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████| 1349/1349 [01:31<00:00, 14.81it/s]
05/24/2020 16:11:41 - INFO - __main__ -     Evaluation done in total 91.079697 secs (0.008442 sec per example)
05/24/2020 16:11:41 - INFO - transformers.data.metrics.squad_metrics -   Writing predictions to: out-noamp/predictions_.json
05/24/2020 16:11:41 - INFO - transformers.data.metrics.squad_metrics -   Writing nbest to: out-noamp/nbest_predictions_.json
05/24/2020 16:12:09 - INFO - __main__ -   Results: {'exact': 73.80321665089878, 'f1': 81.96651715123286, 'total': 10570, 'HasAns_exact': 73.80321665089878, 'HasAns_f1': 81.96651715123286, 'HasAns_total': 10570, 'best_exact': 73.80321665089878, 'best_exact_thresh': 0.0, 'best_f1': 81.96651715123286, 'best_f1_thresh': 0.0}

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

The script should produce f1 = 88.52, exact_match = 81.22.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.10.0
- Platform: Linux-4.15.0-99-generic-x86_64-with-debian-buster-sid
- Python version: 3.7.7
- PyTorch version (GPU?): 1.5.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: True
- Using distributed or parallel set-up in script?: False


[WIP] Replace instances of  `config.output_hidden_states` with function argument  `output_hidden_states` in all possible models.

Attempts to close #3879 by refactoring `config.output_hidden_states` as an argument,`output_hidden_states`, to functions `forward()` and `call()`. Affects all PT and TF models that output hidden states.

Currently it is failing the following tests: `run_tests_tf`, `run_tests_torch` and `run_tests_torch_and_tf` because they are still using `config.output_hidden_states`. Please advise on how I should go about testing this.


LongformerTokenizerFast

This PR adds `LongformerTokenizerFast` by sub-classing `RobertaTokenizerFast`.

@patrickvonplaten


Fix two bugs on MNLI dataset and SST-2 respectively.

The text index of test data of SST-2 are 1 rather than 0.

The label of MNLI task has a tricky swap on MNLI dataset, which should also be involved in get_labels() of dataset for correctness.


pass lowercase to fast tokenizer

https://github.com/huggingface/transformers/blob/a34a9896ac2a4a33ff9cd805c76eed914c8d8965/src/transformers/tokenization_gpt2.py#L338-L343
only passes four parameters without `lowercase` to
https://github.com/huggingface/tokenizers/blob/704cf3fdd2f607ead58a561b892b510b49c301db/bindings/python/tokenizers/implementations/byte_level_bpe.py#L15


seems that run_ner.py cannot handle the situation when example length exceed max_length?

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...):

Language I am using the model on (English, Chinese ...):

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1.
2.
3.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- Platform:
- Python version:
- PyTorch version (GPU?):
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:


Automatically setting number of LSH buckets in Reformer may give invalid value

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): Reformer
transformers version: 2.9.0

When using a Reformer model such that `config.num_buckets` is set to `None` (as recommended), the model automatically determines the number of necessary buckets. However, depending on some hyperparameters, it may compute an odd number of buckets, which is invalid.

It happens at this line, because of the +1 in the second element:
https://github.com/huggingface/transformers/blob/a34a9896ac2a4a33ff9cd805c76eed914c8d8965/src/transformers/modeling_reformer.py#L541

This triggers the assertion in `_hash_vectors`: https://github.com/huggingface/transformers/blob/a34a9896ac2a4a33ff9cd805c76eed914c8d8965/src/transformers/modeling_reformer.py#L454

I think a simple fix is just to check if the number is odd, and add one in that case.


RuntimeError: The size of tensor a (1025) must match the size of tensor b (1024) at non-singleton dimension 3

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): gpt2

Language I am using the model on (English, Chinese ...): english

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)
I want to generate samples with length = 2000
## To reproduce

Steps to reproduce the behavior:
1. run the command
python run_generation.py     --model_type=gpt2     --model_name_or_path=<output_dir_of_finetuned_model>     --length=2000     --num_return_sequences=10     --stop_token='<|endoftext|>
'

## Expected behavior
The error that I am getting is
w = torch.where(mask.bool(), w, self.masked_bias.to(w.dtype))
RuntimeError: The size of tensor a (1024) must match the size of tensor b (1025) at non-singleton dimension 3

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- Platform: Python 3.8
- Python version:
- PyTorch version (GPU?): 10.1
- Tensorflow version (GPU?):
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No


'use_fast=True' results in 'TypeError' when trying to save tokenizer via AutoTokenizer

# 🐛 Bug

## Information

Model I am using: all/any

Language I am using the model on: English

The problem arises when using:
* [x] the official example scripts: `AutoTokenizer.from_pretrained([model], use_fast=True)`

After updating to Transformers v2.10.0, when setting `use_fast=True` as in `tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased', use_fast=True)`, when trying to save the model by using `tokenizer.save_pretrained(path)` I get the following error and the process quits:
```
File "../python3.6/site-packages/transformers/tokenization_utils.py", line 1117,
  in save_pretrained
    vocab_files = self.save_vocabulary(save_directory)
  File "../python3.6/site-packages/transformers/tokenization_utils.py", line 2657,
  in save_vocabulary
    files = self._tokenizer.save(save_directory)
  File "../python3.6/site-packages/tokenizers/implementations/base_tokenizer.py",
  line 328, in save
    return self._tokenizer.model.save(directory, name=name)
TypeError
```
When I omit the `use_fast=True` flag, the tokenizer saves fine.

The tasks I am working on is:
* [x] my own task or dataset: Text classification

## To reproduce

Steps to reproduce the behavior:

1. Upgrade to `transformers==2.10.0` (requires `tokenizers==0.7.0`)
2. Load a tokenizer using `AutoTokenizer.from_pretrained()` with flag `use_fast=True`
3. Train for one epoch on any dataset, then try to save the tokenizer.

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
The tokenizer/file should save into the chosen path, as it does with the regular tokenizer.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.10.0
- Platform: Linux-5.0.0-37-generic-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.8
- PyTorch version (GPU?): 1.4.0+cu100 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no


InvalidArgumentError while using GRU layer in custom training loop

TensorFlow

Anything TensorFlow

**System information**
- TensorFlow version `2.1.0`
- Python version: `3`
- GPU model and memory: `NVIDIA Tesla P100`
- CUDA Version: `10.1`
- Environment: This happens both on Kaggle and Colab

**Describe the current behavior**
I'm trying to train a Hugging face transformer model (roBERTa base) with a custom training loop, and got the error below:

```
InvalidArgumentError: 2 root error(s) found.
  (0) Invalid argument:  InstantiateOptions.input_devices must have the same length as the number of arguments: input_devices length = 23 number of arguments = 24
	 [[{{node while/body/_1/StatefulPartitionedCall}}]]
  (1) Invalid argument:  InstantiateOptions.input_devices must have the same length as the number of arguments: input_devices length = 23 number of arguments = 24
	 [[{{node while/body/_1/StatefulPartitionedCall}}]]
	 [[while/body/_1/Adam/Cast_6/ReadVariableOp/_30]]
0 successful operations.
0 derived errors ignored. [Op:__inference_train_step_35635]

Function call stack:
train_step -> train_step
```
The thing is I can run the same model using `model.fit()` API, and this error only happens when I use a LSTM or GRU layer on top of the transformer

**Describe the expected behavior**
Training should go normal


Add BART fine-tuning summarization community notebook

An example of how to train, evaluate, deploy a BART summarization model with fastai using the blurr library


[All models] Extend config.output_attentions with output_attentions function arguments

Attempts to close #3880 by refactoring `config.output_attentions` => `output_attentions` in `forward()`/`call()` functions

**UPDATE** from Patrick

@thomwolf @LysandreJik
This PR adds the argument `output_attentions` to every forward function for more flexibility. This PR makes it possible to easily switch the output attention on/off without having to instantiate a new model every time.

The logic is the following. If 'output_attentions` is configured in the `forwrd()` fn => use that. If not => use `config.output_attentions`.

IMPORTANT: This PR does **not** change backward compatibility since output_attentions can still be configured using the config.

TESTS:
An additional test is added to the `test_output_attentions()` common test.

FUTURE PR:
- [ ] Clean the documentation. We still need to add this argument to the docs of all models and make sure the docs are clean. Lemme know @Bharat123rox if you want to tackle this in a new PR or if I should do it :-) It's not the most interesting PR, so I fully understand if you don't feel like doing it anymore ;-)


DOC: Make `import torch` explicit for "Quick tour TF 2.0" example

I tried to run the Quick Tour example with only the `tensorflow` and the `transformers` imports (as shown literally in the code snippet), and _obviously_ (in hint sight) this fails with:

```
pytorch_model = BertForSequenceClassification.from_pretrained('./models/', from_tf=True)
NameError: name 'BertForSequenceClassification' is not defined
```

The trivial fix was to add `import torch` to the snippet.

When running all examples in sequence, this is not an issue, but I
was running the `tensorflow 2` example in a separate project.

Adding this line may avoid this confusion for the next newcomer :-)


resize_token_embeddings not implemented for TFGPT2LMHeadModel

I am using a TFGPT2LMHeadModel pretrained model and new special tokens to the gpt tokenizer.
However, the method resize_token_embeddings is not implemented in all gpt2 tf models.
Will it be added? Or are there any workarounds?

Thank you!


How to speed up inference step in BertQuestionAnswering?

Distillation

Related to model distillation

External

Using the library with external tools (onnx, tflite, ...)

I'm working on a QA system which uses a pre-trained BertQA model. At this point, even if I use GPU, the step generating start_scores and end_scores  for a set of 20 candidate passages till takes a few seconds which is the bottleneck of my application.

I just wonder do we have any strategies/tricks to speed up this step? So far, it seems using multiple GPUs at the inference step does not help at all. Any advice greatly appreciated!


DOC: Fix typos in modeling_auto

Fix typo of word `dictionnary` => `dictionary`


Add nn.Module as superclass

Add nn.Module as superclass of `MMBTModel` to fix #4532


MMBT doesn't inherit from nn.Module

Core: Modeling

Internals of the library; Models.

PyTorch

Anything PyTorch

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): MMBT

Language I am using the model on (English, Chinese ...): not related

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

Minimal reproduction:
```python
from transformers import MMBTConfig, MMBTModel, AutoConfig, AutoModel

electra_config = AutoConfig.from_pretrained("google/electra-small-discriminator")
mmbt_config = MMBTConfig(electra_config)

model = AutoModel.from_config(electra_config)

mmbt = MMBTModel(mmbt_config, model, None)

mmbt()
```

output:
```
Traceback (most recent call last):
  File "mmbt_debug.py", line 11, in <module>
    mmbt()
TypeError: 'MMBTModel' object is not callable
```

You can see in the [source code](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_mmbt.py#L152) that it's currently only inheriting from `ModuleUtilsMixin`, but not `torch.nn.Module`

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

We should be seeing a downstream error since I didn't pass in a real modal encoder or any input.  It should at least call `forward()`

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.9.1 (also tried 2.10.0)
- Platform: Darwin-19.4.0-x86_64-i386-64bit
- Python version: 3.6.10
- PyTorch version (GPU?): 1.5.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: no (doesn't matter)
- Using distributed or parallel set-up in script?: no (doesn't matter)


Fix add_special_tokens on fast tokenizers

Fix #4457

By using `flatten`, the following
```
dict_values(['[EOS]', '[BOS]'])
```
was being transformed into this:
```
['[', 'E', 'O', 'S', ']', '[', 'B', 'O', 'S', ']']
```


Tensorflow improvements

Hello,

Here a quite big PR that propose the following updates:

- Loss computation is now attached to their respective class, such as PyTorch.
- Remove useless `mode` and `loss_name` parameters for the TF Trainer.
- Add missing task models to different Transformers
- Bugfix on T5 keras serialization + tests
- Add tests for TF Flaubert and XLM-Roberta
- Bugfix in TF Trainer for Tensorflow 2.2

Reviews are welcome :)

/cc @julien-c @LysandreJik @thomwolf


Minor correction in Roberta Model docs, Roberta doesn't use NSP

In the roberta model docs

https://huggingface.co/transformers/model_doc/roberta.html

>pooler_output (tf.Tensor of shape (batch_size, hidden_size)):
Last layer hidden-state of the first token of the sequence (classification token) further processed by a Linear layer and a Tanh activation function. The Linear layer weights are trained from the next sentence prediction (classification) objective during Bert pretraining. This output is usually not a good summary of the semantic content of the input, you’re often better with averaging or pooling the sequence of hidden-states for the whole input sequence.

Roberta uses something else though

>FULL-SENTENCES: Each input is packed with
full sentences sampled contiguously from one
or more documents, such that the total length is
at most 512 tokens. Inputs may cross document
boundaries. When we reach the end of one document, we begin sampling sentences from the
next document and add an extra separator token
between documents. We remove the NSP loss.


Warn the user about max_len being on the path to be deprecated.

Makes it clear the parameter `model_max_length` is preferred over `max_len` by writting a warning to the logger.

https://github.com/huggingface/transformers/issues/4527


Tokenizers bug: version 2.10 doesn't honor `max_len` when instantiating a pretrained model

# 🐛 Bug

## Information

Hello! I've just upgraded from Transformers 2.8 to Transformers 2.10, and noticed that parameter `max_len` is not properly honored when instantiating a pretrained model. For example, in Transformer 2.8.0, I was able to limit the length of a tokenized sequence as follows:

```python
import transformers

>>> tok = transformers.RobertaTokenizer.from_pretrained('roberta-base', max_len=16)
>>> tok.encode('This is a sentence', pad_to_max_length=True)

[0, 152, 16, 10, 3645, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]

>>>print(tok.max_len)

16
```

However, on version 2.10, `max_len` is ignored when loading a pretrained tokenizer:

```python
import transformers

>>> tok = transformers.RobertaTokenizer.from_pretrained('roberta-base', max_len=16)
>>> tok.encode('This is a sentence', pad_to_max_length=True)

[0, 152, 16, 10, 3645, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...] # 512 tokens

>>>print(tok.max_len)

512
```

This bug can be temporary solved by using `model_max_length` instead of `max_len`, but it broke the all my scripts that relied on that attribute. It seems that this issue was introduced in a recent change in [`tokenization_utils.py`](https://github.com/huggingface/transformers/blob/master/src/transformers/tokenization_utils.py) (line 825):

```python
# For backward compatibility we fallback to set model_max_length from max_len if provided
model_max_length = model_max_length if model_max_length is not None else kwargs.pop("max_len", None)
```

This compatibility is not guaranteed if the pretrained model contains `model_max_length` among its parameters, but `max_len` is specified in `from_pretrained`.

Model I am using (Bert, XLNet ...): As far as I can tell, this affects all pretrained models. Observed on BERT, RoBERTa, and DistilBERT.

Language I am using the model on (English, Chinese ...): As far as I can tell, this affects all pretrained models. Observed on English.

The problem arises when using:
* [ ] the official example scripts
* [x] my own modified scripts: See above.

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task:
* [x] my own task or dataset: It's a classification task.

## To reproduce

See above.

## Expected behavior

See above.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.10.0
- Platform: Linux-4.15.0-1060-aws-x86_64-with-debian-buster-sid
- Python version: 3.6.5
- PyTorch version (GPU?): 1.4.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: yes; 4 x Tesla V100
- Using distributed or parallel set-up in script?: parallel, but not relevant


link to paper was broken

changed from https://https://arxiv.org/abs/2001.04451.pdf to https://arxiv.org/abs/2001.04451.pdf


Error in Longformer attention mask using apex mixed precision

Anything PyTorch

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): Longformer

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Install latest transformers (2.10.0) and apex (0.1)
2. Code:
```
import torch
from transformers import LongformerTokenizer, LongformerModel, LongformerConfig
from apex import amp

tokenizer = LongformerTokenizer.from_pretrained('longformer-base-4096')
config = LongformerConfig.from_pretrained('longformer-base-4096')
model = LongformerModel.from_pretrained('longformer-base-4096').cuda()
optimizer = torch.optim.Adam(model.parameters(), lr=1e-5)

model, optimizer = amp.initialize(model, optimizer, opt_level='O1')

# toy input example
inputs = torch.randint(config.vocab_size, (1, 1024)).cuda() # randomly select tokens with sequence length 1024
mask = torch.ones(1, 1024).cuda() # set mask for every token to local attention
mask[0] = 2. # global attention for first token

outputs = model(inputs, attention_mask=mask)
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
Error message:
```
File "/home/miniconda3/envs/test_transformer/lib/python3.8/site-packages/transformers/modeling_longformer.py", line 374, in
forward
    attn[extra_attention_mask_nonzeros[::-1]] = nonzero_selected_attn.view(
RuntimeError: expected dtype Half but got dtype Float
```
`attn` is half precision but is assigned a tensor that is casted into single precision `.type_as(hidden_states)` in line 376 .

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.10.0
- Platform: Linux-4.15.0-65-generic-x86_64-with-glibc2.10
- Python version: 3.8.3
- PyTorch version (GPU?): 1.5.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no


Codecov migration to marketplace app

Hi, Tom from Codecov here.

We noticed that you are using our app with high frequency, and we’re so excited to see that! However, because you are not using our app, you may have experienced issues with uploading reports or viewing coverage information. This is due to rate-limiting from GitHub.

**In order to prevent any future outages, we ask that you move over to our GitHub marketplace app: https://github.com/marketplace/codecov.**

Let me know if you have any questions, or if I can help at all with this process.


Can't reproduce export to onnx with custom bert model

# 🐛 Bug

I try to run onnx export on a custom bert model, but during inference I get the following error.
I share a google colab with the minimum changes to reproduce. All changes are marked with a `# CHANGE` comment. https://colab.research.google.com/drive/1eiqyQmvhwGih6IHrOg7MkLSc2q0zMHmH?usp=sharing

```
InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Non-zero status code returned while running Gather node. Name:'Gather_32' Status Message: indices element out of data bounds, idx=1 must be within the inclusive range [-1,0]
```

## Information

Model I am using (Bert, XLNet ...): Bert

Language I am using the model on (English, Chinese ...):
None, I'm using a custom bert model, and for this bug report I'm using a random bert model.

The problem arises when using:
The official example notebook: https://github.com/huggingface/transformers/blob/master/notebooks/04-onnx-export.ipynb

## To reproduce

Steps to reproduce the behavior:

Run the convert to onxx script with a custom bert model. I've made a copy of the official notebook with the minimum changes required to illustrate the problem here: https://colab.research.google.com/drive/1eiqyQmvhwGih6IHrOg7MkLSc2q0zMHmH?usp=sharing

```python
---------------------------------------------------------------------------

InvalidArgument                           Traceback (most recent call last)

<ipython-input-12-1d032f1e9ad0> in <module>()
      9
     10 # Run the model (None = get all the outputs)
---> 11 sequence, pooled = cpu_model.run(None, inputs_onnx)
     12
     13 # Print information about outputs

/usr/local/lib/python3.6/dist-packages/onnxruntime/capi/session.py in run(self, output_names, input_feed, run_options)
    109             output_names = [output.name for output in self._outputs_meta]
    110         try:
--> 111             return self._sess.run(output_names, input_feed, run_options)
    112         except C.EPFail as err:
    113             if self._enable_fallback:

InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Non-zero status code returned while running Gather node. Name:'Gather_32' Status Message: indices element out of data bounds, idx=1 must be within the inclusive range [-1,0]
```

## Expected behavior

Get pooled and sequence output of bert model.

## Environment info

- `transformers` version: 2.10.0
- Platform: Linux-4.19.104+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.5.0+cu101 (True)
- Tensorflow version (GPU?): 2.2.0 (True)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no


Added huseinzol05/t5-small-bahasa-cased README.md

model card

Related to pretrained model cards


Using DistillBert to train Bert (run_languge_modeling.py) for some languge from scratch

Distillation

Related to model distillation

Ex: LM (Pretraining)

Related to language modeling pre-training

# ❓ Questions & Help

Is it a right thing to use 'distilbert-base-cased'  for training that with more data for some language ( which exists in the list of multilingual languages , and doesn't have a separate model still)

Will we reach good results as good as doing that with 6 layer BERT ?
Is there any differences between  6 layer BERT and distillBert (to use as the model for run_languge_modeling.py)


How to use its own custom Optimizer (GLUE Example)

# ❓ Questions & Help
I am referring to the example on GLUE text-classification/run_glue.py.
I would like to change the default optimizer (which I believe it to be ADAMW) to my own one.
Should be in the imported Trainer or TrainingArguments, but I have not found an example doing so.

Is it possible with any optimizer? (as long as they are written as a Torch Optimizer of course)
Thanks a lot!


Specify device in DataCollator

By setting `device` parameter in `DataCollator` we're able to allocate tensors directly on the right device at tensor creation and avoid moving data around afterwards.

This effectively avoid some snippet like this in the Trainer:

```python

for k, v in some_dict.items():
     v.to(device)
```


[marian] possible memory leak problem while translating & extracting internal representations

# 🐛 Bug

## Information
I am extracting the internal representations of some of the Marian models.
There seems to be a memory leak problem. In this issue, you will find code for running the model sentence by sentence (bsz = 1) just to keep it simple. When I use batching, the problem persists and arises earlier.

Model I am using: MarianMT
` modelnames=[f'Helsinki-NLP/opus-mt-en-{tgt}'  for tgt in ['de', 'fr', 'ee', 'sv', 'el', 'fi', 'cs', 'ru' ]]`

Language I am using the model on: en-{tgt]

The problem arises when using:
* [ ] a mix of official example scripts and my own:
on this code, I keep the lines used to see if it is a memory problem. Hence the `empty_cache()`, keeping track of the memory usage with `memory_stats()`, and passing things to 'cpu' (but this has not solved the problem for me)
```
import torch
import transformers
config_overrider={'output_attentions':True, 'output_hidden_states':True}
model = transformers.MarianMTModel.from_pretrained(modelname, **config_overrider)
tokenizer = transformers.MarianTokenizer.from_pretrained(modelname)
model.eval()
encoded_sentences = []
memdict=[]
for sent in tqdm(sentences):
        tokdsent = self.tokenizer.prepare_translation_batch(src_texts=[' '.join(sent)])
        tokdsent = {k:v.to(self.device) for k,v in tokdsent.items()}
        model_outputs = self.model.forward(**tokdsent)
        encoded_sentences.append( [x.to('cpu') for x in model_outputs[4]+model_outputs[1]]  )
        torch.cuda.empty_cache()

        memdict.append(torch.cuda.memory_stats(self.device))
        print(memdict[-1]['active.all.current'],memdict[-1]['active.all.peak']) # comment out this part
```

The tasks I am working on is:
* [ ] Using a dataset from an official task:
semantic textual similarity - STS 2012, 2013, 2014, 2015 and 2016 (can use [this one](https://github.com/Helsinki-NLP/Geometry/blob/refactorize/data/STS/allSTS.txt))

## To reproduce

Steps to reproduce the behavior:

1. Load and tokenize the sentences (need this for what I am doing, even when I detok when passing it to the tokenizer)
```
STS_path = "path/to/allSTS.txt"
with open(STS_path, 'r') as f:
    samples = f.readlines()

sentences = []
for sent in samples:
    sent = sent.strip()
    sent = re.findall(r'[\w]+|\.|,|\?|\!|;|:|\'|\(|\)|/',sent)
    sentences.append(sent)
```
2. Run the code above. The one on _"The problem arises when using:"_
3. For me, around line 3750 I get OOM:
```
RuntimeError('CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 31.75 GiB total capacity; 30.67 GiB already allocated; 17.69 MiB free; 30.67 GiB reserved in total by PyTorch)')
```
Here I copy some of the linesprinted from `active.all.current ` and `active.all.peak` (it never changes the upwards trend):
```
533 535
779 811
1025 1057
1271 1303
1517 1549
1763 1795
2009 2041
2255 2287
2501 2533
2747 2779
2993 3025
...
9635 9667
9881 9913
10127 10159
10373 10405
10619 10651
...
921311 921343
921557 921589
921803 921835
922049 922081
922295 922327
922541 922573
```
^-- these are the first 10 lines, somewhere around 40 sentences, and the last lines before running out of mem - close to 3750 sents.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

I would expect the memory on the cuda device to be freed after every iteration since I am overwriting the variable there and what I append to the list I want to keep is sent to cpu.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- Platform: Linux 3.10.0-1062.7.1.el7.x86_64 x86_64, Red Hat Enterprise Linux Server 7.7 (Maipo)
- Python version: 3.7.3
- PyTorch version (GPU?): 1.5.0 for cuda 10.2 (Nvidia Volta V100 GPU with 32 GB of memory)
- Tensorflow version (GPU?):  not using tf
- Using GPU in script?: yes (but I have seen the same problem on CPU)
- Using distributed or parallel set-up in script?: no


How to train a custom seq2seq model with BertModel

How to train a custom seq2seq model with `BertModel`,

I would like to use some Chinese pretrained model base on `BertModel`

so I've tried using `Encoder-Decoder Model`, but it seems the`Encoder-Decoder Model` is not used for conditional text generation

and I saw that BartModel seems to be the model I need, but I cannot load pretrained BertModel weight with BartModel.

by the way, could I finetune a BartModel for seq2seq with custom data ?

any suggestion, thanks


sometimes loss starts with nan when running "Quick tour TF 2.0 training and PyTorch interoperability" script

TensorFlow

Anything TensorFlow

# 🐛 Bug

## Information
The problem arises when using:
* [x] the official example scripts: (give details below)
"Quick tour TF 2.0 training and PyTorch interoperability"

## To reproduce

Steps to reproduce the behavior:

1.run the code:
```
import tensorflow as tf
import tensorflow_datasets
from transformers import *

# Load dataset, tokenizer, model from pretrained model/vocabulary
tokenizer = BertTokenizer.from_pretrained('bert-base-cased')
model = TFBertForSequenceClassification.from_pretrained('bert-base-cased')
data = tensorflow_datasets.load('glue/mrpc')

# Prepare dataset for GLUE as a tf.data.Dataset instance
train_dataset = glue_convert_examples_to_features(data['train'], tokenizer, max_length=128, task='mrpc')
valid_dataset = glue_convert_examples_to_features(data['validation'], tokenizer, max_length=128, task='mrpc')
train_dataset = train_dataset.shuffle(100).batch(32).repeat(10)
valid_dataset = valid_dataset.batch(64)

# Prepare training: Compile tf.keras model with optimizer, loss and learning rate schedule
optimizer = tf.keras.optimizers.Adam(learning_rate=3e-5, epsilon=1e-08, clipnorm=1.0)
loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
metric = tf.keras.metrics.SparseCategoricalAccuracy('accuracy')
model.compile(optimizer=optimizer, loss=loss, metrics=[metric])

# Train and evaluate using tf.keras.Model.fit()
history = model.fit(train_dataset, epochs=10, steps_per_epoch=115,
                    validation_data=valid_dataset, validation_steps=7)
```
2. sometimes the loss is nan and sometimes it works well:
### nan case:
Train for 115 steps, validate for 7 steps
Epoch 1/10
115/115 [==============================] - 863s 8s/step - loss: nan - accuracy: 0.3255 - val_loss: nan - val_accuracy: 0.3162
Epoch 2/10
115/115 [==============================] - 854s 7s/step - loss: nan - accuracy: 0.3255 - val_loss: nan - val_accuracy: 0.3162

### normal case:
Train for 115 steps, validate for 7 steps
Epoch 1/10
 27/115 [======>.......................] - ETA: 11:37 - loss: 0.6249 - accuracy: 0.6609

## Environment info
- `transformers` version: 2.9.1
- Platform:ubuntu 16.04
- Python version:3.6
- PyTorch version (GPU?):1.2.0 GPU
- Tensorflow version (GPU?):2.0.0 gpu
- Using GPU in script?:yes
- Using distributed or parallel set-up in script?:no


Allow BatchEncoding to be pickled

Overrides the methods ___get_state()___ & ___set_state()___ to (respectively) export the content of the underlying `data` dictionnary and - if defined - the content of `encodings`.

Unittests added to covert the serialization & deserialization of all the exported properties.

Block until new release of **tokenizers**


❓ How Linear layer difference between TF2 and PT are handled ?

PyTorch

Anything PyTorch

TensorFlow

Anything TensorFlow

# ❓ Questions & Help

There is a difference between TF2 and Pytorch on how to store the weights of a Linear Layer.

As shown in [this Colab notebook](https://colab.research.google.com/drive/1zLWONO3wo09-PImo0kg1bwARsh2jKnpQ?usp=sharing), in order to get the same output for both TF2 and PT when using `torch.nn.Linear` and `tf.keras.layers.Dense`, we need to transpose the weights in PT.

**I couldn't find in this library where this is handled** (when loading a Pytorch checkpoint to TF2 for example).
Can someone point me out where and how this is handled ?


Couldn't reach server GPT-2

I have tried to use gpt2 using ubuntu and vagrant. This is the code:
`
import torch
from lm_scorer.models.auto import AutoLMScorer as LMScorer
scorer = LMScorer.from_pretrained("gpt2")
`
I get this error:
> AH01215: OSError: Couldn't reach server at 'https://s3.amazonaws.com/models.huggingface.co/bert/gpt2-config.json' to download pretrained model configuration file.

It has worked before but I had to reset my virtual environment and now it no longer worked. I think it has something to do with apache configurations.

Also, It works in the terminal but not in python script.


Need more information

Further information is requested

PyTorch

Anything PyTorch

# 🐛 Bug

## Information

Language I am using the model on English:

## To reproduce
Steps to reproduce the behavior:

1.run the "Quick tour" code:
```
import torch
from transformers import *

# Transformers has a unified API
# for 10 transformer architectures and 30 pretrained weights.
#          Model          | Tokenizer          | Pretrained weights shortcut
MODELS = [(BertModel,       BertTokenizer,       'bert-base-uncased'),
          (OpenAIGPTModel,  OpenAIGPTTokenizer,  'openai-gpt'),
          (GPT2Model,       GPT2Tokenizer,       'gpt2'),
          (CTRLModel,       CTRLTokenizer,       'ctrl'),
          (TransfoXLModel,  TransfoXLTokenizer,  'transfo-xl-wt103'),
          (XLNetModel,      XLNetTokenizer,      'xlnet-base-cased'),
          (XLMModel,        XLMTokenizer,        'xlm-mlm-enfr-1024'),
          (DistilBertModel, DistilBertTokenizer, 'distilbert-base-cased'),
          (RobertaModel,    RobertaTokenizer,    'roberta-base'),
          (XLMRobertaModel, XLMRobertaTokenizer, 'xlm-roberta-base'),
         ]

# To use TensorFlow 2.0 versions of the models, simply prefix the class names with 'TF', e.g. `TFRobertaModel` is the TF 2.0 counterpart of the PyTorch model `RobertaModel`

# Let's encode some text in a sequence of hidden-states using each model:
for model_class, tokenizer_class, pretrained_weights in MODELS:
    # Load pretrained model/tokenizer
    tokenizer = tokenizer_class.from_pretrained(pretrained_weights)
    model = model_class.from_pretrained(pretrained_weights)

    # Encode text
    input_ids = torch.tensor([tokenizer.encode("Here is some text to encode", add_special_tokens=True)])  # Add special tokens takes care of adding [CLS], [SEP], <s>... tokens in the right way for each model.
    with torch.no_grad():
        last_hidden_states = model(input_ids)[0]  # Models outputs are now tuples

# Each architecture is provided with several class for fine-tuning on down-stream tasks, e.g.
BERT_MODEL_CLASSES = [BertModel, BertForPreTraining, BertForMaskedLM, BertForNextSentencePrediction,
                      BertForSequenceClassification, BertForTokenClassification, BertForQuestionAnswering]

# All the classes for an architecture can be initiated from pretrained weights for this architecture
# Note that additional weights added for fine-tuning are only initialized
# and need to be trained on the down-stream task
pretrained_weights = 'bert-base-uncased'
tokenizer = BertTokenizer.from_pretrained(pretrained_weights)
for model_class in BERT_MODEL_CLASSES:
    # Load pretrained model/tokenizer
    model = model_class.from_pretrained(pretrained_weights)

    # Models can return full list of hidden-states & attentions weights at each layer
    model = model_class.from_pretrained(pretrained_weights,
                                        output_hidden_states=True,
                                        output_attentions=True)
    input_ids = torch.tensor([tokenizer.encode("Let's see all hidden-states and attentions on this text")])
    all_hidden_states, all_attentions = model(input_ids)[-2:]

    # Models are compatible with Torchscript
    model = model_class.from_pretrained(pretrained_weights, torchscript=True)
    traced_model = torch.jit.trace(model, (input_ids,))

    # Simple serialization for models and tokenizers
    model.save_pretrained('./directory/to/save/')  # save
    model = model_class.from_pretrained('./directory/to/save/')  # re-load
    tokenizer.save_pretrained('./directory/to/save/')  # save
    tokenizer = BertTokenizer.from_pretrained('./directory/to/save/')  # re-load

    # SOTA examples for GLUE, SQUAD, text generation...
```
2. Encountered the bug:

  File "/home/**/anaconda3/envs/dl/lib/python3.6/site-packages/torch/jit/__init__.py", line 1860, in check_unique
    raise ValueError("TracedModules don't support parameter sharing between modules")

ValueError: TracedModules don't support parameter sharing between modules

## Environment info
- `transformers` version:2.9.1
- Platform: ubuntu 16.04
- Python version: 3.6
- PyTorch version (GPU?):1.2.0 GPU
- Tensorflow version (GPU?):2.0.0 GPU
- Using GPU in script?:yes
- Using distributed or parallel set-up in script?:no


Need more information

Further information is requested

# 🐛 Bug

## Information

Traceback (most recent call last):
  File "F:/Kaggle/Hug/Colab/main.py", line 105, in <module>
    trainer.train()
  File "c:\programdata\anaconda3\lib\site-packages\transformers\trainer.py", line 359, in train
    self.tb_writer.add_hparams(self.args.to_sanitized_dict(), metric_dict={})
AttributeError: 'SummaryWriter' object has no attribute 'add_hparams'


[HUGE] Refactoring tokenizers backend - padding - truncation - pre-tokenized pipeline - fast tokenizers - tests

Fix #4015

Edit @thomwolf: I morphed this in a large refactoring of the tokenizer code and test to make it more flexible and have a better API. Here is a summary of the changes.

## Breaking change
There is no breaking change in the user-facing methods (`encode`, `encode_plus`, `batch_encode_plus`, `tokenize`, `convert_XXX`).

There is a breaking change in the internal method`prepare_for_model` which is now a private method `_prepare_for_model` with a simplified signature.

## A new main user-facing method: `__call__` i.e. `model_input = tokenizer(text, **kwargs)`

The extended encoding methods `encode_plus` and `batch_encode_plus` methods had names that could be intimidating for first-time users.

A new main entry point is created as `tokenizer.__call__` which wraps both methods. You can feed `__call__` with single examples, a pair of sentence to encode together or batches of single/pair sentences.

The signature of `__call__` is also a better fit for the 🤗nlp library when it comes to batches of pairs of sequences since the first and second elements in pair of sentences are supplied as separate arguments (see below) instead of a zipped list of pairs like in `batch_encode_plus`.

While all the previously provided methods (`encode`, `encode_plus`, `batch_encode_plus`, `tokenize`, `convert_XXX`) are still supported without breaking changes, `__call__` is now the recommended way to encode all types of inputs when `tokenizer.encode` (which only return the list of input indices for a single sentence) is not enough i.e. for every case beside simple demo purposes.

Here is how you should use this new entry point for encoding text in all the main use-cases:
```python
from transformers import BertTokenizer

tokenizer = BertTokenizer.from_pretrained('bert-base-cased')

# 1. When you encode "a single sentence"
encoded_input = tokenizer("Hello I'm a single sentence")
# { 'input_ids': [101, 8667, 146, 112, 182, 170, 1423, 5650, 102],
#   'token_type_ids': [0, 0, 0, 0, 0, 0, 0, 0, 0],
#   'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1]}

# 2. When you encode "a pair of sentences in a single input"
encoded_input = tokenizer("How old are you?", "I'm 6 years old")
# { 'input_ids': [101, 1731, 1385, 1132, 1128, 136, 102, 146, 112, 182, 127, 1201, 1385, 102],
#   'token_type_ids': [0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1],
#   'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]}

# 3. When you encode "a batch of single sentences"
batch_sentences = ["Hello I'm a single sentence",
                   "And another sentence",
                   "And the very very last one"]
encoded_input = tokenizer(batch_sentences)
# { 'input_ids': [[101, 8667, 146, 112, 182, 170, 1423, 5650, 102],
#                 [101, 1262, 1330, 5650, 102],
#                 [101, 1262, 1103, 1304, 1304, 1314, 1141, 102]],
#   'token_type_ids': [[0, 0, 0, 0, 0, 0, 0, 0, 0],
#                      [0, 0, 0, 0, 0],
#                      [0, 0, 0, 0, 0, 0, 0, 0]],
#   'attention_mask': [[1, 1, 1, 1, 1, 1, 1, 1, 1],
#                      [1, 1, 1, 1, 1],
#                      [1, 1, 1, 1, 1, 1, 1, 1]]}

# You can batch (to max sequence size) and truncate (to max model length)
# with `padding`and `truncation` (see more details in the next section on padding/truncation)
encoded_input = tokenizer(batch_sentences, padding=True, truncation=True)
# { 'input_ids': [[101, 8667, 146, 112, 182, 170, 1423, 5650, 102],
#                 [101, 1262, 1330, 5650, 102, 0, 0, 0, 0],
#                 [101, 1262, 1103, 1304, 1304, 1314, 1141, 102, 0]],
#   'token_type_ids': [[0, 0, 0, 0, 0, 0, 0, 0, 0],
#                      [0, 0, 0, 0, 0, 0, 0, 0, 0],
#                      [0, 0, 0, 0, 0, 0, 0, 0, 0]],
#   'attention_mask': [[1, 1, 1, 1, 1, 1, 1, 1, 1],
#                      [1, 1, 1, 1, 1, 0, 0, 0, 0],
#                      [1, 1, 1, 1, 1, 1, 1, 1, 0]]}

# 4. When you encode "a batch of pair of sentences"
batch_of_second_sentences = ["I'm a sentence that goes with the first sentence",
                             "And I should be encoded with the second sentence",
                             "And I go with the very last one"]
encoded_input = tokenizer(batch_sentences,
                          batch_of_second_sentences,
                          padding=True,
                          truncation=True)
# { 'input_ids': [[101, 8667, 146, 112, 182, 170, 1423, 5650, 102, 146, 112, 182, 170, 5650, 1115, 2947, 1114, 1103, 1148, 5650, 102],
#                 [101, 1262, 1330, 5650, 102, 1262, 146, 1431, 1129, 12544, 1114, 1103, 1248, 5650, 102, 0, 0, 0, 0, 0, 0],
#                 [101, 1262, 1103, 1304, 1304, 1314, 1141, 102, 1262, 146, 1301, 1114, 1103, 1304, 1314, 1141, 102, 0, 0, 0, 0]],
#   'token_type_ids': [[0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
#                      [0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0],
#                      [0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0]],
#   'attention_mask': [[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
#                      [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0],
#                      [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0]]}
```

## Padding/truncation

The padding and truncation logic was simplified and improved to cover all the major uses-cases with the simplest possible API.

Here is how to do the two most common use-cases for truncation/padding:
```python
from transformers import BertTokenizer

tokenizer = BertTokenizer.from_pretrained('bert-base-cased')
batch_sentences = ["Hello I'm a single sentence",
                   "And another sentence",
                   "And the very very last one"]

# 1. No truncation and no padding
encoded_input = tokenizer(batch_sentences)

# 2. Pad to the max sequence length inside the provided batch
# while truncating to the max input length acceptable by the model
encoded_input = tokenizer(batch_sentences, truncation=True, padding=True)
```

The new API for padding and truncation uses three arguments to the encoding methods: `padding`, `truncation` and `max_length`. This new way to specify padding/truncation is available in all the user-facing encoding methods: `encode`, `encode_plus`, `batch_ecode_plus` and the newly provided `__call__`.

All the previously provided ways to do padding/truncation (`truncation_strategy`, `max_length`, `pad_to_max_length`) are still supported without breaking changes but we recommend to use the new API.

Here are the details of all the possible inputs to `padding`, `truncation` and `max_length`:

- `padding` to control the padding (can be provided with a boolean or a string for finer-grained control). `padding` accepts the following values:

    * `True` or `'longest'`: pad to the longest sequence in the batch (or no padding if only a single sequence if provided),
    * `'max_length'`: pad to a max length specified in `max_length` or to the max acceptable input length for the model if no length is provided (`max_length=None`)
    * `False` or `'do_not_pad'` (default): No padding (i.e. can output batch with sequences of uneven lengths)
- `truncation` to control truncation (can be provided with a boolean or a string for finer-grained control). `truncation` accepts the following values:

    * `True` or `'only_first'`: truncate to a max length specified in `max_length` or to the max acceptable input length for the model if no length is provided (`max_length=None`). This will only truncate the first sequence of a pair if a pair of sequences (or a batch of pairs) is provided,
    * `'only_second'`: truncate to a max length specified in `max_length` or to the max acceptable input length for the model if no length is provided (`max_length=None`). This will only truncate the second sequence of a pair if a pair of sequences (or a batch of pairs) is provided,
    * `'longest_first'`: truncate to a max length specified in `max_length` or to the max acceptable input length for the model if no length is provided (`max_length=None`). This will truncate token by token, removing a token from the longest sequence in the pair if a pair of sequences (or a batch of pairs) is provided,
    * `False` or `'do_not_truncate'` (default): No truncation (i.e. can output batch with sequences length greater than the model max admissible input size)
- `max_length` to control the length of the padding/truncation (integer or `None`). `max_length` accepts the following values:

    * `None` (default): This will use the predefined model max length if required by one of the truncation/padding parameters. If the model has no specific max input length (e.g. XLNet) truncation/padding to max length is deactivated.
    * `any integer value` (e.g. `42`): Use this specific maximum length value if required by one of the truncation/padding parameters.

Now here is a table summarizing the recommended way to setup `padding` and `truncation` as well as the previously provided way to do it (still supported but not recommended) in all cases.

If you use pair of inputs sequence in any of the following examples, you can replace `truncation=True` by a `STRATEGY` selected in `['only_first', 'only_second', 'longest_first']`, i.e. `truncation='only_second'` or `truncation= 'longest_first'` to control how both sequence in the pair are truncated as detailed just before the table. We don't include all these variants for the sake of keeping the table not too long.

| Truncation | Padding | Recommended way | Previously provided (still supported but not recommended)
| --- | --- | --- | --- |
| no truncation | no padding | `tokenizer(batch_sentences)` | `tokenizer.batch_encode_plus(batch_sentences)`
| no truncation | padding to max sequence in batch | `tokenizer(batch_sentences, padding=True)` or `tokenizer(batch_sentences, padding='longest')`| `tokenizer.batch_encode_plus(batch_sentences, pad_to_max_length=True)`
| no truncation | padding to max model input length | `tokenizer(batch_sentences, padding='max_length')` | Not possible
| no truncation | padding to specific length | `tokenizer(batch_sentences, padding='max_length', max_length=42)` | Not possible
|  |  |  |  |
| truncation to max model input length | no padding | `tokenizer(batch_sentences, truncation=True)` or `tokenizer(batch_sentences, truncation=STRATEGY)` | `tokenizer.batch_encode_plus(batch_sentences, max_length=tokenizer.max_len)`
| truncation to max model input length | padding to max sequence in batch | `tokenizer(batch_sentences, padding=True, truncation=True)` or `tokenizer(batch_sentences, padding=True, truncation=STRATEGY)` | Not possible
| truncation to max model input length | padding to max model input length | `tokenizer(batch_sentences, padding='max_length', truncation=True)` or `tokenizer(batch_sentences, padding='max_length', truncation=STRATEGY)` | `tokenizer.batch_encode_plus(batch_sentences, pad_to_max_length=True, max_length=tokenizer.max_len)`
| truncation to max model input length | padding to specific length | Not possible | Not possible
|  |  |  |  |
| truncation to specific length | no padding | `tokenizer(batch_sentences, truncation=True, max_length=42)` or `tokenizer(batch_sentences, truncation=STRATEGY, max_length=42)` | `tokenizer.batch_encode_plus(batch_sentences, max_length=42)`
| truncation to specific length | padding to max sequence in batch | `tokenizer(batch_sentences, padding=True, truncation=True, max_length=42)` or `tokenizer(batch_sentences, padding=True, truncation=STRATEGY, max_length=42)` | Not possible
| truncation to specific length | padding to max model input length | Not possible | Not possible
| truncation to specific length | padding to specific length | `tokenizer(batch_sentences, padding='max_length', truncation=True, max_length=42)` or `tokenizer(batch_sentences, padding='max_length', truncation=STRATEGY, max_length=42)` | `tokenizer.batch_encode_plus(batch_sentences, pad_to_max_length=True, max_length=42)`

## Pre-tokenized inputs

The tokenizers now accept pre-tokenized inputs, i.e. inputs which are already sliced in words. The main reason for implementing a specific track for this type of inputs is to be able to use the fast mapping methods in `tokenizers` which provide character<=>token<=>words mappings. This can be very handy to easily compute labels and extract predictions for instance for Named-Entity-Recognition (NER) or Part-of-Speech tagging (POS tagging).

If you want to use pre-tokenized inputs, just set `is_pretokenized=True` in any of the encoding methods. Here are some examples:
```python
from transformers import BertTokenizer

tokenizer = BertTokenizer.from_pretrained('bert-base-cased')
batch_sentences = [["Hello", "I'm", "a", "single", "sentence"],
                   ["And", "another", "sentence"],
                   ["And", "the", "very", "very", "last", "one"]]

encoded_input = tokenizer(batch_sentences, is_pretokenized=True)

# Pre-tokenized inputs can be used in all cases (single/pair/batch of single/batch of pairs)
batch_of_second_sentences = ["I'm a sentence that goes with the first sentence".split(),
                             "And I should be encoded with the second sentence".split(),
                             "And I go with the very last one".split()]
encoded_input = tokenizer(batch_sentences,
                          batch_of_second_sentences,
                          is_pretokenized=True,
                          padding=True,
                          truncation=True)
```

## Verbose

A new `verbose` argument is provided in all the encoding methods to silence all the warnings related to the length of the input as well as missing special tokens (e.g. missing padding or unknown token).

## Code organization

`tokenization_utils.py` was starting to grow out of control and is now split into three files:
-  `tokenization_utils.py` hosts the code for the `PreTrainedTokenizers`
-  `tokenization_utils_fast.py` hosts the code for the `PreTrainedTokenizersFast`
-  `tokenization_utils_base.py` hosts the common methods for `PreTrainedTokenizers` and `PreTrainedTokenizersFast` (mostly the front API) in a newly created `PretrainedTokenizerBase` as well as all the common logic for special tokens (in `SpecialMixin`) and for the outputs of the encoding (in `BatchEncoding`).

## Full testing of fast tokenizers

The fast tokenizers provided by the [tokenizers](https://github.com/huggingface/tokenizers)  library are now fully tested and follow the same testing pipeline as the python (slow) tokenizers. Additional consistency tests have been added comparing the outputs of the fast and slow tokenizers under various conditions.

## TODO (following PRs)
- Serialization for Fast tokenizers
- Some edge cases for `add_tokens` on Fast tokenizers are not covered (spaces in tokens for byte-level and lower casing of the added tokens).


Add packaging to setup.py

Running `pip install -e transformers` and then `python -c "import transformers"` fails on a fresh Docker container with the error:

```bash
ModuleNotFoundError: No module named 'packaging'
Thu May 21 21:59:44 2020<stderr>:Traceback (most recent call last):
Thu May 21 21:59:44 2020<stderr>:  File "/.../", line 37, in <module>
Thu May 21 21:59:44 2020<stderr>:    from transformers import (
Thu May 21 21:59:44 2020<stderr>:  File "/fsx/transformers/src/transformers/__init__.py", line 350, in <module>
Thu May 21 21:59:44 2020<stderr>:    from .trainer import Trainer, set_seed, torch_distributed_zero_first, EvalPrediction
Thu May 21 21:59:44 2020<stderr>:  File "/fsx/transformers/src/transformers/trainer.py", line 14, in <module>
Thu May 21 21:59:44 2020<stderr>:    from packaging import version
```

Looks like this dependency was recently added, so adding it to setup.py requirements.


FillMaskPipeline crashes when executed on TPU

PyTorch

Anything PyTorch

# 🐛 Bug

## Information

I am following the tutorial in https://colab.research.google.com/github/huggingface/blog/blob/master/notebooks/01_how_to_train.ipynb#scrollTo=QDNgPls7_l13 and running on Google Colab using the TPU. The Pipeline object creation works fine, but when I try to run it on the example sentence, the Colab runtime crashes immediately with an unclear cause and no error message. If I remove the TPU and do not install xla, the pipeline works fine.

## To reproduce

Steps to reproduce the behavior:

```python3
!pip uninstall transformers
!git clone https://github.com/huggingface/transformers
!pip install ./transformers

VERSION = "nightly"
!curl https://raw.githubusercontent.com/pytorch/xla/master/contrib/scripts/env-setup.py -o pytorch-xla-env-setup.py
!python pytorch-xla-env-setup.py --version $VERSION

from transformers import pipeline

fill_mask = pipeline(
    "fill-mask",
    model="drive/My Drive/models/EsperBERTo/output/checkpoint-15000",
    tokenizer="drive/My Drive/models/EsperBERTo"
)

fill_mask("La suno <mask>.")

```

Is anyone else experiencing this?


Hard-coded force_download in run_squad forces expensive community download

Usage

General questions about the library

# 🐛 Bug

## Information
Using a community-registered model (albert, squad2) I noticed that there's no real caching going on during predict/evaluate. In an application that invokes run_squad dozens to hundreds of times, this adds significantly to processing time.

This is due to at least one of the two force_download hard-codings in the run_squad.py script. It would be best to promote the force_download option into the run_squad arguments and let the user override.

I have tested by manually modifying the force_download to be False and caching does occur (I haven't tested dirty cache refetch).

Model I am using (Bert, XLNet ...): Community-submitted Albert v2 xxlarge fine-tuned for SQuAD2 on Torch, https://huggingface.co/mfeb/albert-xxlarge-v2-squad2

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task:
* [ ] my own task or dataset: (give details below)

## To reproduce
Use run_squad.py more than once on a community-installed model and see the fetch go to a different /tmp copy for each invocation.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior
Expect force_download to be overridable at run_squad invocation, especially for community-registered models.

<!-- A clear and concise description of what you would expect to happen. -->
Update run_squad.py to add a force_download argument (default of your choosing) and use the result in the two places force_download is hard coded.

Better might be for the default value to be determined from the nature of the model location/type (e.g., no forced download for non-local models).

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: v2.2.1
- Platform: ubuntu
- Python version: 3.7.7
- PyTorch version (GPU?): gpu 1.3.1
- Tensorflow version (GPU?): gpu 2.0.0
- Using GPU in script?: Y
- Using distributed or parallel set-up in script?: N


[Summarization Pipeline]: Fix default tokenizer

`pipeline.tokenizer` cannot be a dict!


add 2 colab notebooks


SummarizationPipeline crashes

Core: Pipeline

Internals of the library; Pipeline.

```python
summarize = pipeline("summarization")
summarize("Sam Shleifer writes the best docstring examples in the whole world.")
```
➡
```
/usr/local/lib/python3.6/dist-packages/transformers/pipelines.py in _parse_and_tokenize(self, pad_to_max_length, *args, **kwargs)
    461         # Parse arguments
    462         inputs = self._args_parser(*args, **kwargs)
--> 463         inputs = self.tokenizer.batch_encode_plus(
    464             inputs, add_special_tokens=True, return_tensors=self.framework, pad_to_max_length=pad_to_max_length,
    465         )

AttributeError: 'dict' object has no attribute 'batch_encode_plus'
```


Fix convert_token_type_ids_from_sequences for fast tokenizers

Before this fix, the generic version of `convert_token_type_ids_from_sequences` from `tokenizer_utils` gets called when called on a `PreTrainedTokenizerFast`. The `type_ids` for the special token are thus not included.
There is no way at the moment to get this information from the rust tokenizers, so we just use the implementation from the original python tokenizers. Tests added as well.

Thanks @dirkgr for reporting this.


How to finetune ELECTRA on glue?

Core: Modeling

Internals of the library; Models.

After pretraining my own electra model, I wanted to test it out in Glue using run_glue.py.
However I got this:
```
ValueError: Unrecognized configuration class <class 'transformers.configuration_electra.ElectraConfig'> for this kind of AutoModel: AutoModelForSequenceClassification.
Model type should be one of DistilBertConfig, AlbertConfig, CamembertConfig, XLMRobertaConfig, BartConfig, RobertaConfig, BertConfig, XLNetConfig, FlaubertConfig, XLMConfig.
```
After taking a look at the source code, It seems like ElectraConfig isn't available for sequence classification, is there a reason for that?  Did anyone finetune electra on glue?