Can't load `facebook/mbart-large-cc25` tokenizer

# 🐛 Bug
Trying to use the [example](https://huggingface.co/facebook/mbart-large-cc25) fails to load the tokenizer

```
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("facebook/mbart-large-cc25")
```

## Information

Model I am using `facebook/mbart-large-cc25`

The problem arises when using:
* [ ] the official example scripts: (give details below)
[Example](https://huggingface.co/facebook/mbart-large-cc25) fails to load the tokenizer

```
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("facebook/mbart-large-cc25")
```

## To reproduce

Steps to reproduce the behavior:

`AutoTokenizer.from_pretrained("facebook/mbart-large-cc25")`

we get the following error indicating a missing model.

```
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
<ipython-input-48-29412cfd6509> in <module>
----> 1 tokenizer = AutoTokenizer.from_pretrained("facebook/mbart-large-cc25")

/project/WMT20/opt/miniconda3/envs/HuggingFace-3.0.2_cu101/lib/python3.7/site-packages/transformers/tokenization_auto.py in from_pretrained(cls, pretrained_model_name_or_path, *inputs, **kwargs)
    215                     return tokenizer_class_fast.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
    216                 else:
--> 217                     return tokenizer_class_py.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
    218
    219         raise ValueError(

/project/WMT20/opt/miniconda3/envs/HuggingFace-3.0.2_cu101/lib/python3.7/site-packages/transformers/tokenization_utils_base.py in from_pretrained(cls, *inputs, **kwargs)
   1138
   1139         """
-> 1140         return cls._from_pretrained(*inputs, **kwargs)
   1141
   1142     @classmethod

/project/WMT20/opt/miniconda3/envs/HuggingFace-3.0.2_cu101/lib/python3.7/site-packages/transformers/tokenization_utils_base.py in _from_pretrained(cls, pretrained_model_name_or_path, *init_inputs, **kwargs)
   1244                     ", ".join(s3_models),
   1245                     pretrained_model_name_or_path,
-> 1246                     list(cls.vocab_files_names.values()),
   1247                 )
   1248             )

OSError: Model name 'facebook/mbart-large-cc25' was not found in tokenizers model name list (facebook/mbart-large-en-ro, sshleifer/mbart-large-cc25). We assumed 'facebook/mbart-large-cc25' was a path, a model identifier, or url to a directory containing vocabulary files named ['sentencepiece.bpe.model'] but couldn't find such vocabulary files at this path or url.
```

## Expected behavior

Loading a tokenizer for `facebook/mbart-large-cc25` without failure.

## Environment info

```
transformers-cli env
WARNING:tensorflow:From /project/WMT20/opt/miniconda3/envs/HuggingFace-3.0.2_cu101/lib/python3.7/site-packages/transformers/commands/env.py:36: is_gpu_available (from tensorflow.python.framework.test_util) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.config.list_physical_devices('GPU')` instead.
2020-07-14 15:14:13.475823: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 AVX512F FMA
2020-07-14 15:14:13.495994: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 2100000000 Hz
2020-07-14 15:14:13.507620: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55555a4d87f0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-07-14 15:14:13.507676: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- `transformers` version: 3.0.2
- Platform: Linux-3.10.0-957.el7.x86_64-x86_64-with-centos-7.8.2003-Core
- Python version: 3.7.6
- PyTorch version (GPU?): 1.5.1 (False)
- Tensorflow version (GPU?): 2.2.0 (False)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
```


Update README.md

model card

Related to pretrained model cards


tiny ppl typo fix


fail to run trainer.train() with huggingface transformer

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers
     -->

## Details
<!-- I am trying to set up a TensorFlow fine-tune framework for a question-answering project. Using hugging-face/transformer as the prototype, but cannot run through the trainer.

The experiment is conducted at Databricks, the pre-trained model loaded is base-bert, train and dev sets are downloaded from hugging-face examples SQUAD 2.0 https://github.com/huggingface/transformers/tree/master/examples/question-answering

The error log complains about the unexpected keyword argument 'is_impossible', which is a SQUAD 2 data format feature. -->

<!-- Here is the link to my question on stackoverflow -->
**https://stackoverflow.com/questions/62879960/fail-to-run-trainer-train-with-huggingface-transformer**:


Reintroduce clean_text on BertTokenizer call which was removed by mistake in #4723

Signed-off-by: Morgan Funtowicz <funtowiczmo@gmail.com>

closes #7665


Long BERT TypeError: forward() takes from 2 to 4 positional arguments but 7 were given

I'm having an issue on the pretraining of a BERT-like model. I used the following function twice: the first time with [bert-base-multilingual-cased](https://storage.googleapis.com/bert_models/2018_11_23/multi_cased_L-12_H-768_A-12.zip) and the second time with a simil version, but more efficient for **long documents**, exploiting the class [LongformerSelfAttention](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_longformer.py#L100) to make the normal BERT into a **LongBERT**.

```python
def pretrain_and_evaluate(args, model, tokenizer, eval_only, model_path):
    val_dataset = TextDataset(tokenizer=tokenizer,
                              file_path=args.val_datapath,
                              block_size=tokenizer.max_len)
    if eval_only:
        train_dataset = val_dataset
    else:
        logger.info(f'Loading and tokenizing training data is usually slow: {args.train_datapath}')
        train_dataset = TextDataset(tokenizer=tokenizer,
                                    file_path=args.train_datapath,
                                    block_size=tokenizer.max_len)

    data_collator = DataCollatorForLanguageModeling(tokenizer=tokenizer, mlm=True, mlm_probability=0.15)
    trainer = Trainer(model=model, args=args, data_collator=data_collator,
                      train_dataset=train_dataset, eval_dataset=val_dataset, prediction_loss_only=True,)

    eval_loss = trainer.evaluate()
    eval_loss = eval_loss['eval_loss']
    logger.info(f'Initial eval bpc: {eval_loss/math.log(2)}')

    if not eval_only:
        trainer.train(model_path=model_path)
        trainer.save_model()

        eval_loss = trainer.evaluate()
        eval_loss = eval_loss['eval_loss']
        logger.info(f'Eval bpc after pretraining: {eval_loss/math.log(2)}')
```

With the [bert-base-multilingual-cased](https://storage.googleapis.com/bert_models/2018_11_23/multi_cased_L-12_H-768_A-12.zip) it works well: `model` and `tokenizer` passed as arguments to the function are respectively:

```python
model = BertForMaskedLM.from_pretrained('bert-base-multilingual-cased')
tokenizer = BertTokenizerFast.from_pretrained('bert-base-multilingual-cased')
```

But with the modified version of BERT this error occours:

```
Traceback (most recent call last):
  File "convert_bert_to_long_bert.py", line 172, in <module>
    pretrain_and_evaluate(training_args, model, tokenizer, eval_only=False, model_path=training_args.output_dir)
  File "convert_bert_to_long_bert.py", line 86, in pretrain_and_evaluate
    eval_loss = trainer.evaluate()
  File "/Users/user/Library/Python/3.7/lib/python/site-packages/transformers/trainer.py", line 748, in evaluate
    output = self._prediction_loop(eval_dataloader, description="Evaluation")
  File "/Users/user/Library/Python/3.7/lib/python/site-packages/transformers/trainer.py", line 829, in _prediction_loop
    outputs = model(**inputs)
  File "/Users/user/Library/Python/3.7/lib/python/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/Users/user/Library/Python/3.7/lib/python/site-packages/transformers/modeling_bert.py", line 1098, in forward
    return_tuple=return_tuple,
  File "/Users/user/Library/Python/3.7/lib/python/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/Users/user/Library/Python/3.7/lib/python/site-packages/transformers/modeling_bert.py", line 799, in forward
    return_tuple=return_tuple,
  File "/Users/user/Library/Python/3.7/lib/python/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/Users/user/Library/Python/3.7/lib/python/site-packages/transformers/modeling_bert.py", line 460, in forward
    output_attentions,
  File "/Users/user/Library/Python/3.7/lib/python/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/Users/user/Library/Python/3.7/lib/python/site-packages/transformers/modeling_bert.py", line 391, in forward
    hidden_states, attention_mask, head_mask, output_attentions=output_attentions,
  File "/Users/user/Library/Python/3.7/lib/python/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/Users/user/Library/Python/3.7/lib/python/site-packages/transformers/modeling_bert.py", line 335, in forward
    hidden_states, attention_mask, head_mask, encoder_hidden_states, encoder_attention_mask, output_attentions,
  File "/Users/user/Library/Python/3.7/lib/python/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
TypeError: forward() takes from 2 to 4 positional arguments but 7 were given
```
I did few modifications to a working script to obtain a Long version of RoBERTa given the RoBERTa base model. What could be the mistake?


# 🐛 Bug

## Information

Model I am using : ahotrod/electra_large_discriminator_squad2_512

This problem happened again, when I use ELECTRA on question-answering pipeline. My Transformers version is 2.11.0.

Code:
> from transformers import pipeline, AutoTokenizer, AutoModelForQuestionAnswering
>
> tokenizer = AutoTokenizer.from_pretrained("ahotrod/electra_large_discriminator_squad2_512")
> model = AutoModelForQuestionAnswering.from_pretrained("ahotrod/electra_large_discriminator_squad2_512")

Error:

> Traceback (most recent call last):
>   File "albert_qa.py", line 5, in <module>
>     model = AutoModelForQuestionAnswering.from_pretrained("ahotrod/electra_large_discriminator_squad2_512")
>   File "/home/aim/ANDY-project/sentence-transformers-env/lib/python3.6/site-packages/transformers/modeling_auto.py", line 1004, in from_pretrained
>     ", ".join(c.__name__ for c in MODEL_FOR_QUESTION_ANSWERING_MAPPING.keys()),
> ValueError: Unrecognized configuration class <class 'transformers.configuration_electra.ElectraConfig'> for this kind of AutoModel: AutoModelForQuestionAnswering.

- `transformers` version: 2.11.0
- Platform: Ubuntu
- Python version: 3.6.9
- PyTorch version (GPU?): 1.5.1


Where can I find raw code for char_to_token function.

I understand the function has been defined in tokenization_utils_base.py but in the return statement there is a recursive call to the same function. I am unable to understand where does the actual offset calculation takes place.

Tapan


google/reformer-enwik8 tokenizer was not found in tokenizers model name list

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): Reformer

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Enter https://huggingface.co/google/reformer-enwik8
2. Look at "Hosted inference API"

The model's tokenizer cannot be found; I'm getting the same error in scripts as the one displayed on your webpage:

```⚠ This model could not be loaded by the inference API. ⚠
Error loading tokenizer Model name 'google/reformer-enwik8' was not found in tokenizers model name list (google/reformer-crime-and-punishment). We assumed 'google/reformer-enwik8' was a path, a model identifier, or url to a directory containing vocabulary files named ['spiece.model'] but couldn't find such vocabulary files at this path or url. OSError("Model name 'google/reformer-enwik8' was not found in tokenizers model name list (google/reformer-crime-and-punishment). We assumed 'google/reformer-enwik8' was a path, a model identifier, or url to a directory containing vocabulary files named ['spiece.model'] but couldn't find such vocabulary files at this path or url.")
```

## Expected behavior

Tokenizer loaded without issues

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: latest
- Platform: your own
- Python version: ?
- PyTorch version (GPU?): ?
- Tensorflow version (GPU?): ?
- Using GPU in script?: ?
- Using distributed or parallel set-up in script?: ?


Create README.md for the model card of GPorTuguese-2 model (Portuguese GPT-2 small)

Related to pretrained model cards


Customize inference widget input

Related to pretrained model cards


How to use pytorch_model.bin to classify a single sentence?

Hi!

I fine-turned BERT on my own datasets by using run_glue.py. And I got pytorch_model.bin as output. I want to use pytorch_model.bin in another system to classify a single sentence from web browser.
I would be grateful if you could teach me about usage.


FileNotFoundError: File not found when running run_squad.py to fine-tune the BERT on  SQuAD v1.1.

Hi, I am just following this tutorial https://github.com/huggingface/transformers/tree/master/examples/question-answering and created a folder named SQUAD_DIR under transformers. The train and test file were downloaded and put into the SQUAD_DIR folder. But the error is that such a file can not be found.


[ModelOutput] Proposal to fix compatibility issue with torch.DataParallel

This is a proposal to fix #5693 by making `ModelOutput` inherit from a dictionary and behave like a dictionary on iteration (i.e. iterate over the keys rather than the values).

This could break backward compatibility when users iterates over the output tuple rather than indexing it.

On the other hand, we regain backward compatibility with `torch.DataParallel` and from a more general design point of view the `ModelOutput` class should probably be closer to a dictionary than a tuple in the future.


Trying to run seq2seq example scripts with multiple gpus and wandb logging and I get
```
Traceback (most recent call last):
  File "/home/martongyorgy/projects/tmp/transformers/examples/seq2seq/finetune.py", line 364, in <module>
	xsum_rouge.json
  File "/home/martongyorgy/projects/tmp/transformers/examples/seq2seq/finetune.py", line 342, in main
    logger=logger,
  File "/home/martongyorgy/projects/tmp/transformers/examples/lightning_base.py", line 330, in generic_train
    trainer.fit(model)
  File "/home/martongyorgy/miniconda3/envs/transformers/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 891, in fit
    self.ddp_train(task, model)
  File "/home/martongyorgy/miniconda3/envs/transformers/lib/python3.7/site-packages/pytorch_lightning/trainer/distrib_data_parallel.py", line 530, in ddp_train
    self.run_pretrain_routine(model)
  File "/home/martongyorgy/miniconda3/envs/transformers/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1046, in run_pretrain_routine
    self.configure_checkpoint_callback()
  File "/home/martongyorgy/miniconda3/envs/transformers/lib/python3.7/site-packages/pytorch_lightning/trainer/callback_config.py", line 60, in configure_checkpoint_callback
    "checkpoints"
  File "/home/martongyorgy/miniconda3/envs/transformers/lib/python3.7/posixpath.py", line 94, in join
    genericpath._check_arg_types('join', a, *p)
  File "/home/martongyorgy/miniconda3/envs/transformers/lib/python3.7/genericpath.py", line 153, in _check_arg_types
    (funcname, s.__class__.__name__)) from None
TypeError: join() argument must be str or bytes, not 'NoneType'
```

To reproduce follow the instructions in the seq2seq example Readme to download XSUM then run
```
export PYTHONPATH="../":"${PYTHONPATH}"

python finetune.py \
    --learning_rate=3e-5 \
    --fp16 \
    --gpus 2 \
    --do_train \
    --n_val 1000 \
    --val_check_interval 0.1 \
    --data_dir xsum \
    --output_dir xsum_frozen_embs \
    --model_name_or_path t5-small \
    --train_batch_size 16 --eval_batch_size 16 --freeze_embeds --freeze_encoder \
    --num_train_epochs 6 \
    --max_target_length=60 --val_max_target_length=60 --test_max_target_length=100 \
    --logger wandb
```

Happens in pytorch-lightning 0.8.1, fixed in 0.8.4. But there is another problem in  0.8.4, see #5584


Unicode normalization for bert-cased models

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): Bert

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

Tokenize text containing combining marks with `BertTokenizer`. E.g.
```
In [1]: from transformers.tokenization_bert import *

In [2]: tokenizer = BertTokenizer.from_pretrained("bert-base-cased")

In [3]: tokenizer.tokenize("\u00E1")
Out[3]: ['á']

In [4]: tokenizer.tokenize("a\u0301")
Out[4]: ['a', '##́']

In [5]: tokenizer.tokenize("a\u0300")
Out[5]: ['[UNK]']

In [6]: tokenizer.tokenize("\u00E0")
Out[6]: ['à']
```

Results for `BertTokenizerFast` are the same.

## Expected behavior

`"\u00E1"` and `"a\u0301"` should ideally be tokenized the same way; so should `"\u00E0"` and `"a\u0300"`. If existing behavior should be preserved, maybe add an optional argument.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.0.2
- Platform: Linux-4.4.0-18362-Microsoft-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.4.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


Update model_summary.rst

Add '-' to make the reference of Transformer-XL more accurate and formal.


TypeError: an integer is required (got type NoneType) while using  run_language_modeling.py

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): BERT

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below) Yes
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below) Own task

## To reproduce

Steps to reproduce the behavior:
Train the tokenizer:
```python
from tokenizers import    BertWordPieceTokenizer

paths = 'Clean_merged.txt'

# Initialize a tokenizer
tokenizer =    BertWordPieceTokenizer()

# Customize training
tokenizer.train(files=paths, vocab_size=32_000, min_frequency=4)
tokenizer.save_model('./')
```

```
%env TRAIN_FILE= Clean_merged.txt

!python transformers/examples/language-modeling/run_language_modeling.py \
    --output_dir=output_from_scratch \
    --model_type=bert \
    --do_train \
    --tokenizer_name save_tokenizer \
    --save_steps 2000 \
    --per_gpu_train_batch_size 8 \
    --train_data_file=$TRAIN_FILE \
    --mlm \
    --block_size 510
```
<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior
To train a BERT model from scratch, on an mlm task.

### Stack trace
```
2020-07-14 07:52:19.364573: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
07/14/2020 07:52:21 - INFO - transformers.training_args -   PyTorch: setting up devices
07/14/2020 07:52:21 - WARNING - __main__ -   Process rank: -1, device: cuda:0, n_gpu: 1, distributed training: False, 16-bits training: False
07/14/2020 07:52:21 - INFO - __main__ -   Training/evaluation parameters TrainingArguments(output_dir='output_from_scratch', overwrite_output_dir=False, do_train=True, do_eval=False, do_predict=False, evaluate_during_training=False, per_device_train_batch_size=8, per_device_eval_batch_size=8, per_gpu_train_batch_size=8, per_gpu_eval_batch_size=None, gradient_accumulation_steps=1, learning_rate=5e-05, weight_decay=0.0, adam_epsilon=1e-08, max_grad_norm=1.0, num_train_epochs=3.0, max_steps=-1, warmup_steps=0, logging_dir='runs/Jul14_07-52-21_cf8085fe2205', logging_first_step=False, logging_steps=500, save_steps=2000, save_total_limit=None, no_cuda=False, seed=42, fp16=False, fp16_opt_level='O1', local_rank=-1, tpu_num_cores=None, tpu_metrics_debug=False, debug=False, dataloader_drop_last=False, eval_steps=1000, past_index=-1)
07/14/2020 07:52:21 - WARNING - __main__ -   You are instantiating a new config instance from scratch.
07/14/2020 07:52:21 - INFO - transformers.configuration_utils -   loading configuration file save_tokenizer/config.json
07/14/2020 07:52:21 - INFO - transformers.configuration_utils -   Model config BertConfig {
  "architectures": [
    "BertForMaskedLM"
  ],
  "attention_probs_dropout_prob": 0.1,
  "gradient_checkpointing": false,
  "hidden_act": "gelu",
  "hidden_dropout_prob": 0.1,
  "hidden_size": 768,
  "initializer_range": 0.02,
  "intermediate_size": 3072,
  "layer_norm_eps": 1e-12,
  "max_position_embeddings": 514,
  "model_type": "bert",
  "num_attention_heads": 12,
  "num_hidden_layers": 12,
  "pad_token_id": 0,
  "type_vocab_size": 2,
  "vocab_size": 32000
}

07/14/2020 07:52:21 - INFO - transformers.tokenization_utils_base -   Model name 'save_tokenizer' not found in model shortcut name list (bert-base-uncased, bert-large-uncased, bert-base-cased, bert-large-cased, bert-base-multilingual-uncased, bert-base-multilingual-cased, bert-base-chinese, bert-base-german-cased, bert-large-uncased-whole-word-masking, bert-large-cased-whole-word-masking, bert-large-uncased-whole-word-masking-finetuned-squad, bert-large-cased-whole-word-masking-finetuned-squad, bert-base-cased-finetuned-mrpc, bert-base-german-dbmdz-cased, bert-base-german-dbmdz-uncased, TurkuNLP/bert-base-finnish-cased-v1, TurkuNLP/bert-base-finnish-uncased-v1, wietsedv/bert-base-dutch-cased). Assuming 'save_tokenizer' is a path, a model identifier, or url to a directory containing tokenizer files.
07/14/2020 07:52:21 - INFO - transformers.tokenization_utils_base -   Didn't find file save_tokenizer/added_tokens.json. We won't load it.
07/14/2020 07:52:21 - INFO - transformers.tokenization_utils_base -   Didn't find file save_tokenizer/special_tokens_map.json. We won't load it.
07/14/2020 07:52:21 - INFO - transformers.tokenization_utils_base -   Didn't find file save_tokenizer/tokenizer_config.json. We won't load it.
07/14/2020 07:52:21 - INFO - transformers.tokenization_utils_base -   Didn't find file save_tokenizer/tokenizer.json. We won't load it.
07/14/2020 07:52:21 - INFO - transformers.tokenization_utils_base -   loading file save_tokenizer/vocab.txt
07/14/2020 07:52:21 - INFO - transformers.tokenization_utils_base -   loading file None
07/14/2020 07:52:21 - INFO - transformers.tokenization_utils_base -   loading file None
07/14/2020 07:52:21 - INFO - transformers.tokenization_utils_base -   loading file None
07/14/2020 07:52:21 - INFO - transformers.tokenization_utils_base -   loading file None
07/14/2020 07:52:21 - INFO - __main__ -   Training new model from scratch
/usr/local/lib/python3.6/dist-packages/transformers/modeling_auto.py:709: FutureWarning: The class `AutoModelWithLMHead` is deprecated and will be removed in a future version. Please use `AutoModelForCausalLM` for causal language models, `AutoModelForMaskedLM` for masked language models and `AutoModelForSeq2SeqLM` for encoder-decoder models.
  FutureWarning,
07/14/2020 07:52:26 - INFO - filelock -   Lock 140270211965616 acquired on cached_lm_BertTokenizer_508_Clean_merged.txt.lock
07/14/2020 07:52:31 - INFO - transformers.data.datasets.language_modeling -   Loading features from cached file cached_lm_BertTokenizer_508_Clean_merged.txt [took 3.803 s]
07/14/2020 07:52:31 - INFO - filelock -   Lock 140270211965616 released on cached_lm_BertTokenizer_508_Clean_merged.txt.lock
07/14/2020 07:52:33 - INFO - transformers.trainer -   You are instantiating a Trainer but W&B is not installed. To use wandb logging, run `pip install wandb; wandb login` see https://docs.wandb.com/huggingface.
07/14/2020 07:52:33 - WARNING - transformers.training_args -   Using deprecated `--per_gpu_train_batch_size` argument which will be removed in a future version. Using `--per_device_train_batch_size` is preferred.
07/14/2020 07:52:33 - WARNING - transformers.training_args -   Using deprecated `--per_gpu_train_batch_size` argument which will be removed in a future version. Using `--per_device_train_batch_size` is preferred.
07/14/2020 07:52:33 - INFO - transformers.trainer -   ***** Running training *****
07/14/2020 07:52:33 - INFO - transformers.trainer -     Num examples = 101259
07/14/2020 07:52:33 - INFO - transformers.trainer -     Num Epochs = 3
07/14/2020 07:52:33 - INFO - transformers.trainer -     Instantaneous batch size per device = 8
07/14/2020 07:52:33 - INFO - transformers.trainer -     Total train batch size (w. parallel, distributed & accumulation) = 8
07/14/2020 07:52:33 - INFO - transformers.trainer -     Gradient Accumulation steps = 1
07/14/2020 07:52:33 - INFO - transformers.trainer -     Total optimization steps = 37974
Epoch:   0% 0/3 [00:00<?, ?it/s]
Iteration:   0% 0/12658 [00:00<?, ?it/s]Traceback (most recent call last):
  File "transformers/examples/language-modeling/run_language_modeling.py", line 296, in <module>
    main()
  File "transformers/examples/language-modeling/run_language_modeling.py", line 260, in main
    trainer.train(model_path=model_path)
  File "/usr/local/lib/python3.6/dist-packages/transformers/trainer.py", line 492, in train
    for step, inputs in enumerate(epoch_iterator):
  File "/usr/local/lib/python3.6/dist-packages/tqdm/std.py", line 1104, in __iter__
    for obj in iterable:
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 345, in __next__
    data = self._next_data()
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 385, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/usr/local/lib/python3.6/dist-packages/transformers/data/datasets/language_modeling.py", line 75, in __getitem__
    return torch.tensor(self.examples[i], dtype=torch.long)
TypeError: an integer is required (got type NoneType)

Epoch:   0% 0/3 [00:00<?, ?it/s]
Iteration:   0% 0/12658 [00:00<?, ?it/s]
```

## Environment info
- `transformers` version: 3.0.2
- Platform: Linux-4.19.104+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.5.1+cu101 (True)
- Tensorflow version (GPU?): 2.2.0 (True)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: <No


Create README.md (Model card for Norod78/hewiki-articles-distilGPT2py-il)

Related to pretrained model cards

Model card for Norod78/hewiki-articles-distilGPT2py-il
A tiny GPT2 model for generating Hebrew text


Fix typo (model saving TF)


DataParallel fixes

1. switched to a more precise check as suggested by @thomwolf

```
-        if self.args.n_gpu > 1:
+        if isinstance(model, nn.DataParallel):
```
  discussion: https://github.com/huggingface/transformers/issues/5693#issuecomment-657937349

2. fix tests - require the same fixup under DataParallel as the training module fix merged earlier today:
https://github.com/huggingface/transformers/pull/5685
discussion: https://github.com/huggingface/transformers/issues/5693#issuecomment-657938856


Add `power` argument for TF PolynomialDecay


[fix] mbart_en_ro_generate test now identical to fairseq

violentele -> violenţa
Slow test was previously failing.


Using pipeline('ner'), partial tokens returned when grouped_entities=True

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): pipeline('ner', grouped_entities=True)

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

```python
from transformers import pipeline
ner = pipeline('ner', grouped_entities=True)
ner("the sapodilla tree is native to Central America")
```

## Expected behavior

The output says that "##di" is one of the named entities.  It doesn't seem like partial tokens should possibly be returned as predicted named entities.  Instead, I imagine that the desired result is that either the entire word "sapodilla" is determined to be an entity group or nothing at all.  Is this a bug or was this quirk consciously chosen to be allowed?

As a side note, another similar quirk here is that something like "U.S." occasionally gives just "U" or "S" as individual named entities, where "U.S." is desired.  I consider this related to the above issue.

## Environment info
- `transformers` version: 3.0.2
- Platform: Linux-5.3.0-1032-azure-x86_64-with-glibc2.10
- Python version: 3.8.1
- PyTorch version (GPU?): 1.5.1 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no


[Feature request] Pass any Iterable to tokenizer.__call__()

# 🚀 Feature request

Currently `tokenizer.__call__` accepts `List[List[str]]`, for pre-tokenized inputs. It should also accept `List[np.ndarray[str]]`.

## Motivation

The `nlp` library stores a batch of pre-tokenized strings as `List[np.ndarray[str]]` after creating them with `dset.map(batched=True)`. Currently it requires a memory-intensive `tokenizer([list(ex) for ex in batch])` to pass these pre-tokenized strings into `batch_encode_plus`.

## Your contribution

I could contribute this.


Return tokens from tokenizer.__call__()

Currently, pre-tokenizing (not encode tokens to ids, just generating the tokens) a batch of strings requires a manual for-loop. I'm adding a method to call the underlying Rust batch_encode implementation, which runs about 2x faster than a Python for-loop. I was expecting an even greater speedup, so if there's any way this could be made more efficient I would love to hear it.

Before:
```python
batch = ["Sentence 1", "Sentence 2"]
tokenized_batch = [tokenizer.tokenize(ex) for ex in batch]
# [["Sen", "##tence", "1"], ["Sen", "##tence", "2"]
```

Now:
```python
tokenized_batch = tokenizer.tokenize_batch(batch)
# [["Sen", "##tence", "1"], ["Sen", "##tence", "2"]
```


t5 model card

Add Model Card for all t5 checkpoints.
cc @clmnt


Finetuning GPT2 with Custom Loss

## System Info
- Ubuntu 20.04
- Pytorch: 1.5.1+cpu
- Transformers: 3.0.2
- Python: 3.7.6

## Details
Ultimately, I would like to finetune GPT2 on my dataset using a custom loss from an `NGrams` model I have created. Here is what I for the model:
```python
from transformers import GPT2LMHeadModel
from FeatureExtraction.NGrams import *

class GPT2FinetunedWithNgrams(GPT2LMHeadModel):
    def __init__(self, ngrams_model_path):
        super().from_pretrained('gpt2')
        self.ngrams_model = NGrams(ngrams_model_path)

    def forward(
            self,
            input_ids=None,
            past=None,
            attention_mask=None,
            token_type_ids=None,
            position_ids=None,
            head_mask=None,
            inputs_embeds=None,
            labels=None,
            use_cache=None,
            output_attentions=None,
            output_hidden_states=None,
            return_tuple=None,
    ):

        return_tuple = return_tuple if return_tuple is not None else self.config.use_return_tuple

        transformer_outputs = self.transformer(
            input_ids,
            past=past,
            attention_mask=attention_mask,
            token_type_ids=token_type_ids,
            position_ids=position_ids,
            head_mask=head_mask,
            inputs_embeds=inputs_embeds,
            use_cache=use_cache,
            output_attentions=output_attentions,
            output_hidden_states=output_hidden_states,
            return_tuple=return_tuple,
        )
        hidden_states = transformer_outputs[0]
        lm_logits = self.lm_head(hidden_states)

        #use gpt2 to generate a span of text based off input_ids?
        #gpt2_sent = ???

        loss = self.ngrams_model.sentence_loss(gpt2_sent)

        return (loss, lm_logits)
```

and here is my training script using Transformers `Trainer`:
```python
from text_gen_w_transformers.finetune_gpt2 import GPT2FinetunedWithNgrams
from transformers import Trainer, TrainingArguments

model = GPT2FinetunedWithNgrams('/path/to/ngrams/model.pkl')

training_args = TrainingArguments(
    output_dir='/path/to/finetuned_gpt2',
    do_train=True,
    per_device_train_batch_size=16,
    learning_rate=1e-3,
    num_train_epochs=1,
)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=?????
)

trainer.train()
```

My questions are:
1. You can see from the `#gpt2_sent = ???` comment in the model code that I presume this is the place where I would generate a gpt2 sequence based off this version of gpt2 that is currently being finetuned. However, I am not sure what the best way to go about doing this is. Any recommendations?

2. In the training script, I am using the `Trainer` module. However, I don't understand what the `train_dataset` parameter is in `Trainer`. I have a csv file that contains one sequence per line, but I have a feeling I need to construct a `Dataset` object or something.

3. I haven't tried to run this code because I need to fill in the above 2 parts, but I also think I'm not setting any of the parameters for `transformer_outputs`. It looks like they are set to `None` and I don't know if that will be problematic. Any thoughts on this?

I've been reading through the documentation and really like the library. I'm also new to it and pytorch so I apologize if my questions are pretty basic. Thanks in advance for your help!

**EDIT**
When I run `model = GPT2FinetunedWithNgrams('/path/to/ngrams/model.pkl')`, I just get repeated printouts of the GPT2Config object, so I don't think `super().from_pretrained('gpt2')` is the right approach for loading a pretrained model when you have inherited another class.


TPU CI testing

Run TPU CI testing using CircleCI.

I sent a guide in Slack with steps needed on the owner side for CircleCI and Google Cloud to make this work (setting env vars, creating GKE cluster, and populating the dataset).


T5 ONNX Export Test Failing on GPU

https://github.com/huggingface/transformers/runs/863647649?check_suite_focus=true

![image](https://user-images.githubusercontent.com/6045025/87354370-788d9900-c52c-11ea-8838-286633cd93dc.png)


Fix slow test_enro_generate

see https://user-images.githubusercontent.com/6045025/87226810-6103b400-c364-11ea-875a-dcbd7c1e49ca.png


Cannot preprocess WNUT'17 dataset for token-classification

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...):BERT

Language I am using the model on (English, Chinese ...):

The problem arises when using:
* [x] the official example scripts: I am trying to run the run_ner.py script for the WNUT’17 dataset. I followed the preprocessing steps mentioned in the README. I just downloaded the dev dataset. The command
`python3 scripts/preprocess.py data_wnut_17/dev.txt.tmp $BERT_MODEL $MAX_LENGTH > data_wnut_17/dev.txt`
does not work and my terminal hangs.
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: Token Classification
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Move into the `token-classification` folder in the `examples` directory.
2. Run the commands for the WNUT’17 dataset as mentioned in the README.
```
mkdir -p data_wnut_17
curl -L 'https://github.com/leondz/emerging_entities_17/raw/master/emerging.dev.conll' | tr '\t' ' ' > data_wnut_17/dev.txt.tmp
export MAX_LENGTH=128
export BERT_MODEL=bert-large-cased
```
3. The terminal hangs on the next command
`python3 scripts/preprocess.py data_wnut_17/dev.txt.tmp $BERT_MODEL $MAX_LENGTH > data_wnut_17/dev.txt`

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
I would expect the `preprocess.py` script to execute normally and return the control of the terminal back to me. Once hung, I fail to regain control of my command-line even by `ctrl-d/z/c`.
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- Platform: Ubuntu 18.04
- Python version: 3.7.8
- PyTorch version (GPU?): 1.5 (Yes)
- Tensorflow version (GPU?): -
- Using GPU in script?: The error does not involve GPU usage in the script.
- Using distributed or parallel set-up in script?: No


Unable to finetune BERT on own dataset

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers
     -->

## Details
Hello, I am trying to use my own data to finetune transformers for summarization tasks. I have followed the instructions on the README.md by generating a text file with each article to be summarized on its own line. The text file is located in ...\workspace\hug

Below is the command that I ran and it's resulting error. I couldn't find further instructions on the README or in the stackoverflow/current or closed issues. Can you please provide more guidance with examples on how to train on your data set. I would greatly appreciate this. @sshleifer

Command ran + error:
![image](https://user-images.githubusercontent.com/13383820/87342840-c7bdd480-c500-11ea-8cc7-10751701b519.png)

Finetune.sh
![image](https://user-images.githubusercontent.com/13383820/87343018-12d7e780-c501-11ea-834d-48ed5f4c3b02.png)

Once again, I would really appreciate your help!


INFO:tensorflow:Error reported to Coordinator: in converted code:

    /data0/liuyongkang/.conda/envs/tf2/lib/python3.7/site-packages/transformers/trainer_tf.py:511 _forward  *
        per_example_loss, _ = self._run_model(features, labels, True)
    /data0/liuyongkang/.conda/envs/tf2/lib/python3.7/site-packages/transformers/trainer_tf.py:534 _run_model  *
        outputs = self.model(features, labels=labels, training=training)[:2]
    /data0/liuyongkang/.conda/envs/tf2/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/base_layer.py:778 __call__
        outputs = call_fn(cast_inputs, *args, **kwargs)
    /data0/liuyongkang/.conda/envs/tf2/lib/python3.7/site-packages/transformers/modeling_tf_roberta.py:530 call  *
        loss = self.compute_loss(labels, reshaped_logits)
    /data0/liuyongkang/.conda/envs/tf2/lib/python3.7/site-packages/transformers/modeling_tf_utils.py:135 compute_loss  *
        if shape_list(logits)[1] == 1:
    /data0/liuyongkang/.conda/envs/tf2/lib/python3.7/site-packages/tensorflow_core/python/autograph/operators/control_flow.py:918 if_stmt
        basic_symbol_names, composite_symbol_names)
    /data0/liuyongkang/.conda/envs/tf2/lib/python3.7/site-packages/tensorflow_core/python/autograph/operators/control_flow.py:956 tf_if_stmt
        error_checking_orelse)
    /data0/liuyongkang/.conda/envs/tf2/lib/python3.7/site-packages/tensorflow_core/python/util/deprecation.py:507 new_func
        return func(*args, **kwargs)
    /data0/liuyongkang/.conda/envs/tf2/lib/python3.7/site-packages/tensorflow_core/python/ops/control_flow_ops.py:1174 cond
        return cond_v2.cond_v2(pred, true_fn, false_fn, name)
    /data0/liuyongkang/.conda/envs/tf2/lib/python3.7/site-packages/tensorflow_core/python/ops/cond_v2.py:83 cond_v2
        op_return_value=pred)
    /data0/liuyongkang/.conda/envs/tf2/lib/python3.7/site-packages/tensorflow_core/python/framework/func_graph.py:983 func_graph_from_py_func
        expand_composites=True)
    /data0/liuyongkang/.conda/envs/tf2/lib/python3.7/site-packages/tensorflow_core/python/util/nest.py:568 map_structure
        structure[0], [func(*x) for x in entries],
    /data0/liuyongkang/.conda/envs/tf2/lib/python3.7/site-packages/tensorflow_core/python/util/nest.py:568 <listcomp>
        structure[0], [func(*x) for x in entries],
    /data0/liuyongkang/.conda/envs/tf2/lib/python3.7/site-packages/tensorflow_core/python/framework/func_graph.py:943 convert
        (str(python_func), type(x)))

    TypeError: To be compatible with tf.contrib.eager.defun, Python functions must return zero or more Tensors; in compilation of <function tf_if_stmt.<locals>.error_checking_body at 0x7f55400e3c80>, found return value of type <class 'tensorflow.python.keras.losses.MeanSquaredError'>, which is not a Tensor.


generator` yielded an element that could not be converted to the expected type. The expected type was int32, but the yielded element was None.

# 🐛 Bug

## Information

Model I am using (RoBerta):

Language I am using the model on (English):

The problem arises when using:
* [ ] the official example scripts: (give details below)
inputs = tokenizer(
                text_a,
                text_b,
                add_special_tokens=True,
                max_length=max_length,
                padding="max_length",
                truncation=True,
                return_overflowing_tokens=True,
            )
            if "num_truncated_tokens" in inputs and inputs["num_truncated_tokens"] > 0:
                logger.info(
                    "Attention! you are cropping tokens (swag task is ok). "
                    "If you are training ARC and RACE and you are poping question + options,"
                    "you need to try to use a bigger max seq length!"
                )

            choices_inputs.append(inputs)

        label = label_map[example.label]

        input_ids = [x["input_ids"] for x in choices_inputs]
        attention_mask = (
            [x["attention_mask"] for x in choices_inputs] if "attention_mask" in choices_inputs[0] else None
        )
        token_type_ids = (
            [x["token_type_ids"] for x in choices_inputs] if "token_type_ids" in choices_inputs[0] else None
        )

"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""

self.dataset = tf.data.Dataset.from_generator(
                gen,
                (
                    {
                        "example_id": tf.int32,
                        "input_ids": tf.int32,
                        "attention_mask": tf.int32,
                        "token_type_ids": tf.int32,
                    },
                    tf.int64,
                ),
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
The inputs  dose not include the key"token_type_ids",so a None is return,but a None can't convert to tf.int32 in the tensorflow2,so the code can't work

## To reproduce

Steps to reproduce the behavior:

1.
2.
3.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- Platform:
- Python version:
- PyTorch version (GPU?):
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:


[Don't merge - Bert2Bert] Add training scripts and slight changes to Trainer

Just a draft to keep track of Bert2Bert summary training.


Update tokenization_t5.py

Minor doc fix.


Add generic text classification example in TF

This PR adds a new example script for text classification in TensorFlow with the :hugs:nlp lib. The script allows users to run a text classification task on their own CSV files.


Extending vocabulary by a large size crashes RobertaTokenizerFast

# 🐛 Bug

## Information

Model I am using: RoBERTa

Language I am using the model on: English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

```python
from transformers import RobertaTokenizerFast

tokenizer = RobertaTokenizerFast.from_pretrained('roberta-base')
tokenizer.add_tokens([str(i) for i in range(60000)])
```

Here's the stack trace:
```
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: CompiledTooBig(10485760)', /__w/tokenizers/tokenizers/tokenizers/src/tokenizer/added_vocabulary.rs:299:13
stack backtrace:
   0: backtrace::backtrace::libunwind::trace
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.46/src/backtrace/libunwind.rs:86
   1: backtrace::backtrace::trace_unsynchronized
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.46/src/backtrace/mod.rs:66
   2: std::sys_common::backtrace::_print_fmt
             at src/libstd/sys_common/backtrace.rs:78
   3: <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt
             at src/libstd/sys_common/backtrace.rs:59
   4: core::fmt::write
             at src/libcore/fmt/mod.rs:1069
   5: std::io::Write::write_fmt
             at src/libstd/io/mod.rs:1537
   6: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:62
   7: std::sys_common::backtrace::print
             at src/libstd/sys_common/backtrace.rs:49
   8: std::panicking::default_hook::{{closure}}
             at src/libstd/panicking.rs:198
   9: std::panicking::default_hook
             at src/libstd/panicking.rs:218
  10: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:477
  11: rust_begin_unwind
             at src/libstd/panicking.rs:385
  12: core::panicking::panic_fmt
             at src/libcore/panicking.rs:89
  13: core::option::expect_none_failed
             at src/libcore/option.rs:1272
  14: tokenizers::tokenizer::added_vocabulary::AddedVocabulary::add_tokens
  15: tokenizers::tokenizer::Tokenizer::add_tokens
  16: tokenizers::tokenizer::__init11742626496714830824::__init11742626496714830824::__wrap
  17: method_vectorcall_VARARGS_KEYWORDS
             at /tmp/build/80754af9/python_1593706424329/work/Objects/descrobject.c:332
  18: _PyObject_Vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Include/cpython/abstract.h:127
  19: call_function
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:4963
  20: _PyEval_EvalFrameDefault
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:3486
  21: function_code_fastcall
             at /tmp/build/80754af9/python_1593706424329/work/Objects/call.c:283
  22: _PyFunction_Vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Objects/call.c:410
  23: _PyObject_Vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Include/cpython/abstract.h:127
  24: call_function
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:4963
  25: _PyEval_EvalFrameDefault
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:3486
  26: _PyEval_EvalCodeWithName
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:4298
  27: _PyFunction_Vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Objects/call.c:435
  28: _PyObject_Vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Include/cpython/abstract.h:127
  29: method_vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Objects/classobject.c:60
  30: _PyObject_Vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Include/cpython/abstract.h:127
  31: call_function
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:4963
  32: _PyEval_EvalFrameDefault
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:3515
  33: _PyEval_EvalCodeWithName
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:4298
  34: _PyFunction_Vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Objects/call.c:435
  35: _PyObject_Vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Include/cpython/abstract.h:127
  36: call_function
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:4963
  37: _PyEval_EvalFrameDefault
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:3486
  38: gen_send_ex
             at /tmp/build/80754af9/python_1593706424329/work/Objects/genobject.c:222
  39: _PyGen_Send
             at /tmp/build/80754af9/python_1593706424329/work/Objects/genobject.c:292
  40: _PyEval_EvalFrameDefault
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:2053
  41: gen_send_ex
             at /tmp/build/80754af9/python_1593706424329/work/Objects/genobject.c:222
  42: _PyGen_Send
             at /tmp/build/80754af9/python_1593706424329/work/Objects/genobject.c:292
  43: _PyEval_EvalFrameDefault
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:2053
  44: gen_send_ex
             at /tmp/build/80754af9/python_1593706424329/work/Objects/genobject.c:222
  45: _PyGen_Send
             at /tmp/build/80754af9/python_1593706424329/work/Objects/genobject.c:292
  46: _PyEval_EvalFrameDefault
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:2053
  47: gen_send_ex
             at /tmp/build/80754af9/python_1593706424329/work/Objects/genobject.c:222
  48: _PyGen_Send
             at /tmp/build/80754af9/python_1593706424329/work/Objects/genobject.c:292
  49: task_step_impl
             at /usr/local/src/conda/python-3.8.3/Modules/_asynciomodule.c:2638
  50: task_step
             at /usr/local/src/conda/python-3.8.3/Modules/_asynciomodule.c:2931
  51: _PyObject_MakeTpCall
             at /tmp/build/80754af9/python_1593706424329/work/Objects/call.c:159
  52: context_run
             at /tmp/build/80754af9/python_1593706424329/work/Python/context.c:634
  53: cfunction_vectorcall_FASTCALL_KEYWORDS
             at /tmp/build/80754af9/python_1593706424329/work/Objects/methodobject.c:437
  54: PyVectorcall_Call
             at /tmp/build/80754af9/python_1593706424329/work/Objects/call.c:199
  55: do_call_core
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:4983
  56: _PyEval_EvalFrameDefault
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:3559
  57: function_code_fastcall
             at /tmp/build/80754af9/python_1593706424329/work/Objects/call.c:283
  58: _PyFunction_Vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Objects/call.c:410
  59: _PyObject_Vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Include/cpython/abstract.h:127
  60: call_function
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:4963
  61: _PyEval_EvalFrameDefault
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:3486
  62: function_code_fastcall
             at /tmp/build/80754af9/python_1593706424329/work/Objects/call.c:283
  63: _PyFunction_Vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Objects/call.c:410
  64: _PyObject_Vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Include/cpython/abstract.h:127
  65: call_function
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:4963
  66: _PyEval_EvalFrameDefault
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:3486
  67: function_code_fastcall
             at /tmp/build/80754af9/python_1593706424329/work/Objects/call.c:283
  68: _PyFunction_Vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Objects/call.c:410
  69: _PyObject_Vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Include/cpython/abstract.h:127
  70: call_function
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:4963
  71: _PyEval_EvalFrameDefault
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:3486
  72: function_code_fastcall
             at /tmp/build/80754af9/python_1593706424329/work/Objects/call.c:283
  73: _PyFunction_Vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Objects/call.c:410
  74: _PyObject_Vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Include/cpython/abstract.h:127
  75: call_function
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:4963
  76: _PyEval_EvalFrameDefault
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:3486
  77: function_code_fastcall
             at /tmp/build/80754af9/python_1593706424329/work/Objects/call.c:283
  78: _PyFunction_Vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Objects/call.c:410
  79: _PyObject_FastCallDict
             at /tmp/build/80754af9/python_1593706424329/work/Objects/call.c:96
  80: _PyObject_Call_Prepend
             at /tmp/build/80754af9/python_1593706424329/work/Objects/call.c:887
  81: slot_tp_call
             at /tmp/build/80754af9/python_1593706424329/work/Objects/typeobject.c:6521
  82: _PyObject_MakeTpCall
             at /tmp/build/80754af9/python_1593706424329/work/Objects/call.c:159
  83: _PyObject_Vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Include/cpython/abstract.h:125
  84: call_function
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:4963
  85: _PyEval_EvalFrameDefault
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:3500
  86: function_code_fastcall
             at /tmp/build/80754af9/python_1593706424329/work/Objects/call.c:283
  87: _PyFunction_Vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Objects/call.c:410
  88: _PyObject_Vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Include/cpython/abstract.h:127
  89: call_function
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:4963
  90: _PyEval_EvalFrameDefault
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:3486
  91: _PyEval_EvalCodeWithName
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:4298
  92: _PyFunction_Vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Objects/call.c:435
  93: _PyObject_Vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Include/cpython/abstract.h:127
  94: method_vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Objects/classobject.c:60
  95: _PyObject_Vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Include/cpython/abstract.h:127
  96: call_function
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:4963
  97: _PyEval_EvalFrameDefault
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:3515
  98: _PyEval_EvalCodeWithName
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:4298
  99: _PyFunction_Vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Objects/call.c:435
 100: _PyObject_Vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Include/cpython/abstract.h:127
 101: call_function
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:4963
 102: _PyEval_EvalFrameDefault
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:3486
 103: _PyEval_EvalCodeWithName
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:4298
 104: _PyFunction_Vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Objects/call.c:435
 105: _PyObject_Vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Include/cpython/abstract.h:127
 106: call_function
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:4963
 107: _PyEval_EvalFrameDefault
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:3486
 108: function_code_fastcall
             at /tmp/build/80754af9/python_1593706424329/work/Objects/call.c:283
 109: _PyFunction_Vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Objects/call.c:410
 110: _PyObject_Vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Include/cpython/abstract.h:127
 111: call_function
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:4963
 112: _PyEval_EvalFrameDefault
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:3486
 113: _PyEval_EvalCodeWithName
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:4298
 114: _PyFunction_Vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Objects/call.c:435
 115: _PyObject_Vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Include/cpython/abstract.h:127
 116: method_vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Objects/classobject.c:89
 117: PyVectorcall_Call
             at /tmp/build/80754af9/python_1593706424329/work/Objects/call.c:199
 118: do_call_core
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:5010
 119: _PyEval_EvalFrameDefault
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:3559
 120: _PyEval_EvalCodeWithName
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:4298
 121: _PyFunction_Vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Objects/call.c:435
 122: _PyObject_Vectorcall
             at /tmp/build/80754af9/python_1593706424329/work/Include/cpython/abstract.h:127
 123: call_function
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:4963
 124: _PyEval_EvalFrameDefault
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:3500
 125: _PyEval_EvalCodeWithName
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:4298
 126: PyEval_EvalCodeEx
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:4327
 127: PyEval_EvalCode
             at /tmp/build/80754af9/python_1593706424329/work/Python/ceval.c:718
 128: run_eval_code_obj
             at /tmp/build/80754af9/python_1593706424329/work/Python/pythonrun.c:1125
 129: run_mod
             at /tmp/build/80754af9/python_1593706424329/work/Python/pythonrun.c:1147
 130: PyRun_FileExFlags
             at /tmp/build/80754af9/python_1593706424329/work/Python/pythonrun.c:1063
 131: PyRun_SimpleFileExFlags
             at /tmp/build/80754af9/python_1593706424329/work/Python/pythonrun.c:428
 132: pymain_run_file
             at /tmp/build/80754af9/python_1593706424329/work/Modules/main.c:387
 133: pymain_run_python
             at /tmp/build/80754af9/python_1593706424329/work/Modules/main.c:571
 134: Py_RunMain
             at /tmp/build/80754af9/python_1593706424329/work/Modules/main.c:650
 135: Py_BytesMain
             at /tmp/build/80754af9/python_1593706424329/work/Modules/main.c:1096
 136: __libc_start_main
 137: <unknown>
             at ../sysdeps/x86_64/elf/start.S:103
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
fatal runtime error: failed to initiate panic, error 5
Aborted (core dumped
```
<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior
Tokens should get added normally

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.0.2
- Platform: Linux-5.3.0-1032-azure-x86_64-with-glibc2.10
- Python version: 3.8.3
- PyTorch version (GPU?): 1.5.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


facebook/bart-large-mnli input format

Hi folks,

First off, I've been using you guys since the early days and think the effort and time that you put in is just phenomenal. Thank you. All the postgrads I know at the Uni of Edinburgh love HuggingFace.

My question concerns the usage of the ```facebook/bart-large-mnli``` checkpoint - specifically the input formatting. The paper mentions that inputs are concatenated and appended with an EOS token, which is then passed to the classification head.

Something like below perhaps? If this is the case, the probabilities do not seem right, seeing as the first two sentences are the exact same.

```python
from transformers import AutoModelForSequenceClassification, AutoTokenizer, AutoModel
import torch

t = AutoTokenizer.from_pretrained("facebook/bart-large-mnli")
mc = AutoModelForSequenceClassification.from_pretrained("facebook/bart-large-mnli")

s1 = torch.tensor(t("i am good. [EOS] i am good.", padding="max_length")["input_ids"])
s2 = torch.tensor(t("i am good. [EOS] i am NOT good.", padding="max_length")["input_ids"])
s3 = torch.tensor(t("i am good. [EOS] i am bad.", padding="max_length")["input_ids"])

with torch.no_grad():
  logits = mc(torch.stack((s1,s2,s3)), output_hidden_states=True)[0]

sm = torch.nn.Softmax()
print(sm(logits))
# tensor([[0.2071, 0.3143, 0.4786],      # these sentences are the exact same, so why just 0.47?
#             [0.6478, 0.1443, 0.2080],       # slightly better, but this checkpoint gets ~80% acc on MNLI
#             [0.3937, 0.2987, 0.3076]])      # This distribution is almost random, but the sentences are the exact opposite
```

I note that ```[EOS]``` is not registered with the tokenizer special tokens. When I use the registers ```<s>``` or ```</s>``` I get similar results


ONNX export broken for QA models

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): bert-base-uncased

Language I am using the model on (English, Chinese ...): -

The problem arises when using:
* [x] the official example scripts: convert_graph_to_onnx.py
* [x] my own modified scripts: any script attempting to do a regular torch.onnx.export on a `PreTrainedModel`

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: SQuAD
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Run:
`src/transformers/convert_graph_to_onnx.py --model bert-base-uncased --framework pt --pipeline question-answering /tmp/test_hf_onnx/test_hf.onnx`

2. Observe in console:
```
Error while converting the model: Only tuples, lists and Variables supported as JIT inputs/outputs. Dictionaries and strings are also accepted but their usage is not recommended. But got unsupported type QuestionAnsweringModelOutput
```

## Expected behavior

Successful export of the model to `/tmp/test_hf_onnx/test_hf.onnx`

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: current master ce374ba87767d551f720242d5e64bfa976531079
- Platform: Ubuntu 18.04
- Python version: 3.7
- PyTorch version (GPU?): 1.5.0
- Tensorflow version (GPU?): -
- Using GPU in script?: -
- Using distributed or parallel set-up in script?: -


How to download Pre-trained T5 model?

Hi,

I tried looking for ways to download & use T5-small pre-trained model but didn't get any API mentioned in documentation to download it. Though I found links but don't know will it work if I pass the path of the model?

Thanks in advance..


QA Pipeline: Key Error due to predicting a token outside of allowed context

# 🐛 Bug

## Information

Model: distilbert
Language: English
The problem arises when using: QA inference via `pipeline`

The pipeline throws an exception when the model predicts a token that is not part of the document (e.g. final special token).
In the example below, the model predicts token 13 to be the end of the answer span.
The context however ends at token 12 and token 13 is the final [SEP] token. Therefore, we get a key error when trying to access
`feature.token_to_orig_map[13])` in here:

https://github.com/huggingface/transformers/blob/ce374ba87767d551f720242d5e64bfa976531079/src/transformers/pipelines.py#L1370-L1380

## To reproduce

```
nlp = pipeline("question-answering",model="distilbert-base-uncased-distilled-squad",
                                  tokenizer="distilbert-base-uncased",
                                  device=-1)

nlp(question="test finding", context="My name is Carla and I live in Berlin")
```

results in

```
Traceback (most recent call last):
  File "/home/mp/deepset/dev/haystack/debug.py", line 16, in <module>
    nlp(question="test finding", context="My name is Carla and I live in Berlin")
  File "/home/mp/miniconda3/envs/py37/lib/python3.7/site-packages/transformers/pipelines.py", line 1316, in __call__
    for s, e, score in zip(starts, ends, scores)
  File "/home/mp/miniconda3/envs/py37/lib/python3.7/site-packages/transformers/pipelines.py", line 1316, in <listcomp>
    for s, e, score in zip(starts, ends, scores)
KeyError: 13
```

## Expected behavior

Predictions that are pointing to tokens that are not part of the "context" (here: the last [SEP] token) should be filtered out from possible answers.

## Environment info

- `transformers` version: 3.0.2
- Platform: Ubuntu 18.04
- Python version: 3.7.6
- PyTorch version (GPU?): 1.5.1, CPU
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


Attention heads attend equally after conversion from tensorflow checkpoint

# 🐛 Bug

## Information

Hi.

I'm using notebook https://github.com/jessevig/bertviz/blob/master/head_view_bert.ipynb from https://github.com/jessevig/bertviz for visualizing attention in the Bert model. Given example works fine with pytorch default models. The problem arises when I'm converting pre-trained (with https://github.com/google-research/bert#pre-training-with-bert) on custom dataset Bert-Base Multilingual Uncased model.

Pytorch default model visualization:
![Screenshot from 2020-06-30 12-08-01](https://user-images.githubusercontent.com/22409396/87297669-7bc75b80-c511-11ea-8ca1-a24f8d4c31df.png)

Example of attention values for default model:

         [[8.3652e-03, 6.9530e-02, 6.6828e-02,  ..., 9.5115e-03, 2.7546e-02, 4.5171e-01],
          [3.3669e-03, 1.2537e-02, 9.2709e-03,  ..., 1.5638e-03, 1.0154e-03, 9.1897e-01],
          [6.5795e-03, 3.6612e-03, 5.4454e-02,  ..., 1.1923e-03, 4.1071e-03, 8.4899e-01],
          ...,
          [7.3705e-03, 2.5430e-03, 7.6645e-03,  ..., 1.7184e-02, 4.6256e-02, 8.2301e-01],
          [2.2311e-02, 1.8006e-03, 4.3833e-02,  ..., 9.1167e-03, 1.3746e-01, 6.2295e-01],
          [7.5967e-02, 4.2936e-02, 4.6500e-02,  ..., 4.9925e-02, 6.6538e-02, 5.0721e-02]],

         [[3.5124e-02, 2.2295e-02, 9.2680e-03,  ..., 1.1409e-02, 1.7234e-02, 5.5768e-01],
          [7.0571e-03, 3.7321e-01, 1.7890e-02,  ..., 7.6114e-03, 8.8965e-03, 3.6259e-01],
          [8.8010e-03, 4.9023e-03, 1.4315e-01,  ..., 2.2279e-03, 7.9276e-02, 4.3233e-01],
          ...,

Visualization after model conversion from tf to pytorch:
![Screenshot from 2020-06-30 12-07-42](https://user-images.githubusercontent.com/22409396/87297740-97326680-c511-11ea-8a18-6a33160034ab.png)

Example of attention values after conversion:

          [[0.0716, 0.0686, 0.0556,  ..., 0.0776, 0.0783, 0.0648],
          [0.0893, 0.0513, 0.0641,  ..., 0.0606, 0.0908, 0.0554],
          [0.0868, 0.0663, 0.0621,  ..., 0.0822, 0.0777, 0.0471],
          ...,
          [0.0906, 0.0750, 0.0649,  ..., 0.0807, 0.1011, 0.0444],
          [0.0670, 0.0667, 0.0620,  ..., 0.0877, 0.0739, 0.0515],
          [0.0773, 0.0738, 0.0652,  ..., 0.0787, 0.0856, 0.0518]],

         [[0.0553, 0.0622, 0.0665,  ..., 0.0585, 0.0845, 0.0670],
          [0.0631, 0.0829, 0.0592,  ..., 0.0608, 0.0968, 0.0532],
          [0.0561, 0.0720, 0.0617,  ..., 0.0628, 0.1010, 0.0802],
          ...,

Another experiments:
1. loading tensorflow checkpoint directly without conversion - works fine, not equal attentions;
2. loading pytorch model after saving it from loaded tensorflow checkpoint also works fine;
3. tested with Bert-Base Multilingual Uncased model without pre-training (to be sure that pre-training doesn't cause the problem) - got the same results.

So, I guess that conversion from tf checkpoint works wrong or I'm converting model in wrong way.
Any explanation of described behavior would be appreciated, thank you.

## To reproduce

Steps to reproduce the behavior:

1. Example from https://github.com/jessevig/bertviz/blob/master/head_view_bert.ipynb - works fine.
2. Conversion tensorflow checkpoint to pytorch gives incorrect attentions - not work.

```python
from transformers import convert_bert_original_tf_checkpoint_to_pytorch

convert_bert_original_tf_checkpoint_to_pytorch.convert_tf_checkpoint_to_pytorch(
    'model/multilingual_L-12_H-768_A-12/bert_model.ckpt.index',
    'model/multilingual_L-12_H-768_A-12/bert_config.json',
    'model/multilingual_L-12_H-768_A-12/pytorch_model.bin')
```
Then I copy converted pytorch_model.bin, config.json, vocab.txt to model/multilingual_L-12_H-768_A-12_pytorch/

```python
from transformers import BertTokenizer,  BertForPreTraining

pytorch_bert_model = 'model/multilingual_L-12_H-768_A-12_pytorch/'

model = BertForPreTraining.from_pretrained(pytorch_bert_model)
tokenizer = BertTokenizer.from_pretrained(pytorch_bert_model, do_lower_case=True)
```
No errors, but attention values seems wrong as written above.
```
INFO:transformers.configuration_utils:loading configuration file model/multilingual_L-12_H-768_A-12_pytorch/config.json
INFO:transformers.configuration_utils:Model config BertConfig {
  "attention_probs_dropout_prob": 0.1,
  "directionality": "bidi",
  "gradient_checkpointing": false,
  "hidden_act": "gelu",
  "hidden_dropout_prob": 0.1,
  "hidden_size": 768,
  "initializer_range": 0.02,
  "intermediate_size": 3072,
  "layer_norm_eps": 1e-12,
  "max_position_embeddings": 512,
  "model_type": "bert",
  "num_attention_heads": 12,
  "num_hidden_layers": 12,
  "output_attentions": true,
  "pad_token_id": 0,
  "pooler_fc_size": 768,
  "pooler_num_attention_heads": 12,
  "pooler_num_fc_layers": 3,
  "pooler_size_per_head": 128,
  "pooler_type": "first_token_transform",
  "type_vocab_size": 2,
  "vocab_size": 105879
}

INFO:transformers.modeling_utils:loading weights file model/multilingual_L-12_H-768_A-12_pytorch/pytorch_model.bin
INFO:transformers.modeling_utils:All model checkpoint weights were used when initializing BertForPreTraining.

INFO:transformers.modeling_utils:All the weights of BertForPreTraining were initialized from the model checkpoint at model/multilingual_L-12_H-768_A-12_pytorch/.
If your task is similar to the task the model of the ckeckpoint was trained on, you can already use BertForPreTraining for predictions without further training.

INFO:transformers.tokenization_utils_base:loading file model/multilingual_L-12_H-768_A-12_pytorch/vocab.txt
```
Using ```BertModel``` for loading converted model gives the same result.

3. Loading tensorflow checkpoint directly without conversion - works fine

```python
from transformers import BertTokenizer,  BertForPreTraining

bert_config_file = 'model/multilingual_L-12_H-768_A-12_pytorch/config.json'
bert_vocab_file = 'model/multilingual_L-12_H-768_A-12_pytorch/vocab.txt'
tf_bert_checkpoint ='model/multilingual_L-12_H-768_A-12/bert_model.ckpt.index'

model = BertForPreTraining.from_pretrained(tf_bert_checkpoint, from_tf=True, config=bert_config_file)
tokenizer = BertTokenizer.from_pretrained(bert_vocab_file, do_lower_case=True)
```

4. Loading pytorch model after saving it from loaded tensorflow checkpoint - works fine

```python
from transformers import BertTokenizer,  BertForPreTraining

bert_config_file = 'model/multilingual_L-12_H-768_A-12_pytorch/config.json'
bert_vocab_file = 'model/multilingual_L-12_H-768_A-12_pytorch/vocab.txt'
tf_bert_checkpoint ='model/multilingual_L-12_H-768_A-12/bert_model.ckpt.index'

model = BertForPreTraining.from_pretrained(tf_bert_checkpoint, from_tf=True, config=bert_config_file)

model.save_pretrained(model/test/')

pytorch_bert_checkpoint_path = 'model/test/pytorch_model.bin'
model = BertForPreTraining.from_pretrained(pytorch_bert_checkpoint_path, config=bert_config_file)
tokenizer = BertTokenizer.from_pretrained(bert_vocab_file, do_lower_case=True)
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

Conversion from tensorflow checkpoint do not influence on model attentions (values doesn't seem equal), attentions can be visualized correctly.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.0.2
- Platform: Ubuntu 18.04.3 LTS
- Python version: 3.7.7
- PyTorch version (GPU): 1.4.0
- Tensorflow version (GPU): 1.15.0
- Using GPU in script: No
- Using distributed or parallel set-up in script: No


Run Language Modeling on Colab TPU cores terminates

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): GPT2

Language I am using the model on (English, Chinese ...): English (wikitext-2)

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

I'm trying to test `run_language_modeling.py` on GPT2 using all 8 TPU cores.

Running on 1 core gives the following error:
```bash
Epoch:   0% 0/3 [00:00<?, ?it/s]
Iteration: 0it [00:00, ?it/s]Exception in device=TPU:0: 'NoneType' object cannot be interpreted as an integer
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/torch_xla/distributed/xla_multiprocessing.py", line 119, in _start_fn
    fn(gindex, *args)
  File "/content/transformers/examples/language-modeling/run_language_modeling.py", line 292, in _mp_fn
    main()
  File "/content/transformers/examples/language-modeling/run_language_modeling.py", line 260, in main
    trainer.train(model_path=model_path)
  File "/usr/local/lib/python3.6/dist-packages/transformers/trainer.py", line 519, in train
    self.epoch = epoch + (step + 1) / len(epoch_iterator)
TypeError: 'NoneType' object cannot be interpreted as an integer
```

While running using all 8 cores gives this one:
```bash
/usr/lib/python3.6/multiprocessing/semaphore_tracker.py:143: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
  len(cache))
Traceback (most recent call last):
  File "transformers/examples/xla_spawn.py", line 72, in <module>
    main()
  File "transformers/examples/xla_spawn.py", line 68, in main
    xmp.spawn(mod._mp_fn, args=(), nprocs=args.num_cores)
  File "/usr/local/lib/python3.6/dist-packages/torch_xla/distributed/xla_multiprocessing.py", line 182, in spawn
    start_method=start_method)
  File "/usr/local/lib/python3.6/dist-packages/torch/multiprocessing/spawn.py", line 158, in start_processes
    while not context.join():
  File "/usr/local/lib/python3.6/dist-packages/torch/multiprocessing/spawn.py", line 108, in join
    (error_index, name)
Exception: process 0 terminated with signal SIGKILL
```

I'm running this on a Colab TPU Notebook.

## To reproduce

Steps to reproduce the behavior:

```python
VERSION = "20200325"  #@param ["1.5" , "20200325", "nightly"]
!curl https://raw.githubusercontent.com/pytorch/xla/master/contrib/scripts/env-setup.py -o pytorch-xla-env-setup.py
!python pytorch-xla-env-setup.py --version $VERSION
import torch_xla
import torch_xla.core.xla_model as xm

!pip install git+https://github.com/huggingface/transformers.git

!git clone https://github.com/huggingface/transformers.git

!curl https://s3.amazonaws.com/research.metamind.io/wikitext/wikitext-2-v1.zip --output wikitext-2-v1.zip
!unzip wikitext-2-v1.zip
!rm wikitext-2-v1.zip

!python transformers/examples/xla_spawn.py --num_cores 1 \
	transformers/examples/language-modeling/run_language_modeling.py \
    --output_dir=output \
    --model_type=gpt2 \
    --model_name_or_path=gpt2 \
    --do_train \
    --train_data_file=wikitext-2/wiki.train.tokens \
    --do_eval \
    --eval_data_file=wikitext-2/wiki.test.tokens \
    --per_device_train_batch_size 1
```

## Expected behavior

Finetuning the model and saves it.

## Environment info

- `transformers` version: 3.0.2
- Platform: Linux-4.19.104+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.5.0a0+d6149a7 (False)
- Tensorflow version (GPU?): 2.2.0 (False)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: Yes and No


For Roberta pretraining, how to enable large batch training using gradient accumulation?

In the example code, where can I enable gradient accumulation for large batch size training. Thanks. https://github.com/huggingface/transformers/tree/master/examples/language-modeling


Span Mask Fill

I see that Transformers does not support Ernie, but am in search of a way to [MASK] phrases. Can somebody guide me to an alternative to Ernie, code, or a way I could do this myself?


can't resume training from a saved checkpoint in run_glue

# 🐛 Bug
Hi,
I'm using run_glue.py to train Roberta model. I ran the training for a few hours, but after 2 epochs it crashed due to low disk space. I now want to resume the training, and for that, I replaced the --model_name_or_path from roberta-base to [my checkpoint dir]. But then I get the following error:

"OSError: Model name 'models/tmp/roberta512/checkpoint-27000' was not found in tokenizers model name list (roberta-base, roberta-large, roberta-large-mnli, distilroberta-base, roberta-base-openai-detector, roberta-large-openai-detector). We assumed 'models/tmp/roberta512/checkpoint-27000' was a path, a model identifier, or url to a directory containing vocabulary files named ['vocab.json', 'merges.txt'] but couldn't find such vocabulary files at this path or url."

in the checkpoint dir there are only two files:
pytorch_model.bin and config.json

## Information
This is the command line I used to run initially:
python run_glue.py    --model_name_or_path roberta-base   --task_name MNLI   --do_train   --do_eval   --data_dir /home/nlp/ohadr/PycharmProjects/BERT_classification/glue_data/MNLI   --max_seq_length 512   --per_device_train_batch_size 8   --learning_rate 5e-5   --num_train_epochs 3.0   --output_dir /tmp/MNLI/roberta512

This is the command line I tried to resume training with:
python run_glue.py    --model_name_or_path /tmp/MNLI/roberta512/checkpoint-27000   --task_name MNLI   --do_train   --do_eval   --data_dir /home/nlp/ohadr/PycharmProjects/BERT_classification/glue_data/MNLI   --max_seq_length 512   --per_device_train_batch_size 8   --learning_rate 5e-5   --num_train_epochs 3.0   --output_dir models/tmp/roberta512_cont

Model I am using (Bert, XLNet ...): Roberta

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [v] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [v] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. train the model using the following:
python run_glue.py    --model_name_or_path roberta-base   --task_name MNLI   --do_train   --do_eval   --data_dir /home/nlp/ohadr/PycharmProjects/BERT_classification/glue_data/MNLI   --max_seq_length 512   --per_device_train_batch_size 8   --learning_rate 5e-5   --num_train_epochs 3.0   --output_dir /tmp/MNLI/roberta512

2.
use the following line to resume training from the last saved checkpoint - for that, change the marked directory below with your own directory:
python run_glue.py    --model_name_or_path [/tmp/MNLI/roberta512/checkpoint-27000<<change it!>>]   --task_name MNLI   --do_train   --do_eval   --data_dir /home/nlp/ohadr/PycharmProjects/BERT_classification/glue_data/MNLI   --max_seq_length 512   --per_device_train_batch_size 8   --learning_rate 5e-5   --num_train_epochs 3.0   --output_dir models/tmp/roberta512_cont

Error message:
"OSError: Model name 'models/tmp/roberta512/checkpoint-27000' was not found in tokenizers model name list (roberta-base, roberta-large, roberta-large-mnli, distilroberta-base, roberta-base-openai-detector, roberta-large-openai-detector). We assumed 'models/tmp/roberta512/checkpoint-27000' was a path, a model identifier, or url to a directory containing vocabulary files named ['vocab.json', 'merges.txt'] but couldn't find such vocabulary files at this path or url."

## Expected behavior
I would expect the model to continue training from that checkpoint

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.0.1
- Platform: Linux-3.10.0-1127.13.1.el7.x86_64-x86_64-with-centos-7.8.2003-Core
- Python version: 3.7.7
- PyTorch version (GPU?): 1.5.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: <fill in> YES
- Using distributed or parallel set-up in script?: <fill in> PARALLEL


Any insight to this mystery issue? Using the Keras functional API results in whole deleted weights/layers for transformer layers.

# ❓ Questions & Help

This is probably a Keras, tensorflow bug, but just wanted to check-in here in case I overlooked anything.

I've discovered a bug, where if transformer layers are copied from a transformer model, and used as individual layers, and then a Keras model is made using the functional API, this seems to result in missing weights/layers from the list of trainable layers. However, the issue goes away if I use model subclassing to make the Keras model.

I raised the issue here

https://github.com/tensorflow/tensorflow/issues/40638#event-3468314954

but you can directly checkout the issue in this colab notebook

https://colab.research.google.com/gist/Santosh-Gupta/273361f873e4daf572fddea691b1f325/missingtrainablevars.ipynb

Which copies the layers from one of your transformer models. I also made one where I implemented the transformer layers from (near) scratch, and got the same result

https://colab.research.google.com/gist/ravikyram/0191e12b7c6d9afeb80ccc009870b255/untitled52.ipynb

This likely seems like a bug in Keras, but may take a while to pinpoint what exactly is causing this, since a transformer layer has many parts. But just wanted to pop this in here, in case there was some insight to the issue, or a way to pinpoint the cause.


Make the order of additional special tokens deterministic

In `SpecialTokensMixin.all_special_tokens_extended`, deduplication is performed by `all_toks = list(set(all_toks))`. However, this will change the ordering of additional special tokens, and the order depends on the hash seed of set data structure. This will result in non-deterministic id of additional special tokens added to `AutoTokenizer.from_pretrained` method. Therefore, I changed this problematic line to `all_toks = list(OrderedDict.fromkeys(all_toks))`. This line will deduplicate `all_toks` while still keeping the original ordering.


In `SpecialTokensMixin.all_special_tokens_extended`, deduplication is performed by `all_toks = list(set(all_toks))`. However, this will change the ordering of additional special tokens, and the order depends on the hash seed of set data structure. This will result in non-deterministic id of additional special tokens added to `AutoTokenizer.from_pretrained` method. Therefore, I changed this problematic line to `all_toks = sorted(list(set(all_toks)))`.


help：OSError: Model name 'ctrl' was not found in tokenizers model name list (ctrl). We assumed 'ctrl' was a path, a model identifier, or url to a directory containing vocabulary files named ['vocab.json', 'merges.txt'] but couldn't find such vocabulary files at this path or url.

# ❓ Questions & Help
![image](https://user-images.githubusercontent.com/15426714/87268409-dd30ff80-c4fc-11ea-9e34-4d1c06dc1d23.png)


How to generate sentences from Transformer's sentence embeddings?

Is it possible to use the pre-trained transformer models to generate sentences from sentence embeddings?
I konw I can get the continuous representations of a sentence with for example BertModel or GPT2Model.
But can I reconstruct the sentence directly from the sentence representations generated by BertModel, with hugging-face transformer, especially the pre-trained ones?
I mean sentence embeddings as input, the readable sentence as output.


How to visualize the output of the encoder using T-sne plots?

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers
     -->

## Details
I was thinking of just mean or max pooling (along the sequence length dimension) the output of the encoder and visualizing that, but I was wondering if there were better ways of doing so.


Add beta 1 and beta 2 option in `TrainingArguments` for `AdamW` optimizer.

I want to set the Adam Optimizers beta 2 to 0.98 - this is because I want to train a new RoBERTa LM. The paper sais, that it improves stability. The default is 0.999 and it can not be set in `TrainingArguments`.

Could you please add the option to specify beta 1 and 2 for AdamW in the `TrainingArguments`? `adam_epsilon ` can already be specified. If you want me to I can provide a PR.

What do you think?


Create README.md

Related to pretrained model cards


How can I evaluate on GLUE but without fine-tune BERT. Just train the rest layers?

# ❓ Questions & Help

## Details


Update README.md

Related to pretrained model cards


[Don't merge] Run make style on templates

When run `make style` locally, these files got modified.

I noticed that `black` and `isort` seem to have conflicts on these files. Is there a solution?

```
> make style
black --line-length 119 --target-version py35 examples templates tests src utils
reformatted /Users/canwenxu/transformers/src/transformers/__init__.py
reformatted /Users/canwenxu/transformers/templates/adding_a_new_example_script/run_xxx.py
reformatted /Users/canwenxu/transformers/templates/adding_a_new_example_script/utils_xxx.py
All done! ✨ 🍰 ✨
3 files reformatted, 339 files left unchanged.
isort --recursive examples templates tests src utils
Fixing /Users/canwenxu/transformers/templates/adding_a_new_example_script/run_xxx.py
Fixing /Users/canwenxu/transformers/templates/adding_a_new_example_script/utils_xxx.py
Fixing /Users/canwenxu/transformers/src/transformers/__init__.py
```


# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...):

Language I am using the model on (English, Chinese ...):

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1.  python ./examples/text-classification/run_glue.py     --model_name_or_path bert-base-uncased     --task_name $TASK_NAME     --do_train     --do_eval     --data_dir $GLUE_DIR/$TASK_NAME     --max_seq_length 128     --per_device_eval_batch_size=2       --per_device_train_batch_size=2       --learning_rate 2e-5     --num_train_epochs 3.0     --output_dir /tmp/$TASK_NAME/

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->
File "./examples/text-classification/run_glue.py", line 246, in <module>
    main()
  File "./examples/text-classification/run_glue.py", line 173, in main
    model_path=model_args.model_name_or_path if os.path.isdir(model_args.model_name_or_path) else None
  File "/work/vnhh/anaconda3/lib/python3.6/site-packages/transformers/trainer.py", line 499, in train
    tr_loss += self._training_step(model, inputs, optimizer)
  File "/work/vnhh/anaconda3/lib/python3.6/site-packages/transformers/trainer.py", line 622, in _training_step
    outputs = model(**inputs)
  File "/work/vnhh/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "/work/vnhh/anaconda3/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 153, in forward
    return self.gather(outputs, self.output_device)
  File "/work/vnhh/anaconda3/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 165, in gather
    return gather(outputs, output_device, dim=self.dim)
  File "/work/vnhh/anaconda3/lib/python3.6/site-packages/torch/nn/parallel/scatter_gather.py", line 68, in gather
    res = gather_map(outputs)
  File "/work/vnhh/anaconda3/lib/python3.6/site-packages/torch/nn/parallel/scatter_gather.py", line 63, in gather_map
    return type(out)(map(gather_map, zip(*outputs)))
TypeError: __init__() missing 1 required positional argument: 'logits'
## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
It should be able to run and finish training
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.0.2
- Platform: Linux-4.4.0-165-generic-x86_64-with-debian-stretch-sid
- Python version: 3.6.5
- PyTorch version (GPU?): 1.3.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
-tensorboardX: 1.9.0


rename the functions to match the rest of the test convention

no functional change


Cannot import EvalPrediction from transformers

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...):

Language I am using the model on (English, Chinese ...):

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1.  python3 ./examples/text-classification/run_glue.py     --model_name_or_path bert-base-uncased     --task_name $TASK_NAME     --do_train     --do_eval     --data_dir $GLUE_DIR/$TASK_NAME     --max_seq_length 128     --per_device_eval_batch_size=2       --per_device_train_batch_size=2       --learning_rate 2e-5     --num_train_epochs 3.0     --output_dir /tmp/$TASK_NAME/cd ../..

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
It should run normally but then give error: cannot import module EvalPrediction

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.5.1
- Platform: Linux-4.4.0-165-generic-x86_64-with-debian-stretch-sid
- Python version: 3.6.5
- PyTorch version (GPU?): 1.2.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>


How I can  predict missing letters in a sentence, like " I want to b _ _  the car because it is cheap."

Hi,

I am new on NLP, I want to predict missing letters in a sentence. Here is an example,

```text
I want to b _ _  the car because it is cheap.
```


Is Writing With Transform open source?

I want to use Writing with Transform with my own model?  My first though was that I could just download it and modify the source to point to my model., but I can't find WWT anywhere in the repo. Do I have to publish my model then make a request for you to add it?


doc improvements

a few documentation improvements - one variable name consistency rename, and the rest are small tweaks with one clarification.


Making ONNX conversion directly load the model and tokenizer + adding tests

This is a proposal for an update in the ONNX conversion:
- First, I investigated issue #4906 as I was having the same issue. This is due to a dependency on this [commit](https://github.com/pytorch/pytorch/commit/96989a2a114de9b77e7dd9495d62c4a8a549b40d) from the ONNX team available from version 1.5.0 of PyTorch, I therefore added it to the extra requirements and added to the messages to make it more obvious (along with keras2onnx for TF)
- The bigger part of this PR aims to remove the dependency of the script for specific pipelines. I believe this dependency is not needed as a conversion to ONNX simply requires a model, and a tokenizer. There are a few advantages to doing it this way:
1. This would solve #4788, and help greatly towards #5075. With this update, the conversion to ONNX no longer requires a given pipeline, and therefore any model can be converted (I have tested for instance with T5)
2. It is maybe clearer since the elements of the pipeline are not exported onto ONNX.
Let me know your thoughts on that one.
- I added some fast-running integration testing of the script to the existing tests.
- I made the onnx export compatible with the ModelOutput refactor (I believe previously it wasn't?)

A question I have is whether I should add a message or add the possibility for the user to provide a pipeline even if it is not used in order to make it back-compatible?