old nlp causes error that pip install -e. can't fix.

```
Traceback (most recent call last):
  File "finetune.py", line 15, in <module>
    from lightning_base import BaseTransformer, add_generic_args, generic_train
  File "/home/shleifer/transformers_fork/examples/lightning_base.py", line 10, in <mo
dule>
    from transformers import (
  File "/home/shleifer/transformers_fork/src/transformers/__init__.py", line 23, in <
module>
    from .configuration_albert import ALBERT_PRETRAINED_CONFIG_ARCHIVE_MAP, AlbertConfig
  File "/home/shleifer/transformers_fork/src/transformers/configuration_albert.py", line 18, in <module>
    from .configuration_utils import PretrainedConfig
  File "/home/shleifer/transformers_fork/src/transformers/configuration_utils.py", line 25, in <module>
    from .file_utils import CONFIG_NAME, cached_path, hf_bucket_url, is_remote_url
  File "/home/shleifer/transformers_fork/src/transformers/file_utils.py", line 68, in <module>
    import nlp  # noqa: F401
  File "/home/shleifer/miniconda/envs/torch1.5/lib/python3.8/site-packages/nlp/__init__.py", line 41, in <module>
    raise ImportWarning(
ImportWarning: To use `nlp`, the module `pyarrow>=0.16.0` is required, and the current version of `pyarrow` doesn't match this condition.
If you are running this in a Google Colab, you should probably just restart the runtime to use the right version of `pyarrow`.
```


['encoder.version', 'decoder.version'] are unexpected when loading a pretrained BART model

Using an example from the bart doc:
https://huggingface.co/transformers/model_doc/bart.html#bartforconditionalgeneration

```
from transformers import BartTokenizer, BartForConditionalGeneration
tokenizer = BartTokenizer.from_pretrained('facebook/bart-large')
TXT = "My friends are <mask> but they eat too many carbs."

model = BartForConditionalGeneration.from_pretrained('facebook/bart-large')
input_ids = tokenizer([TXT], return_tensors='pt')['input_ids']
logits = model(input_ids)[0]

masked_index = (input_ids[0] == tokenizer.mask_token_id).nonzero().item()
probs = logits[0, masked_index].softmax(dim=0)
values, predictions = probs.topk(5)

print(tokenizer.decode(predictions).split())
```
gives:

```
Some weights of the model checkpoint at facebook/bart-large were not used
when initializing BartForConditionalGeneration:
['encoder.version', 'decoder.version']

- This IS expected if you are initializing BartForConditionalGeneration from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPretraining model).
- This IS NOT expected if you are initializing BartForConditionalGeneration from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
test:9: UserWarning: This overload of nonzero is deprecated:
        nonzero()
Consider using one of the following signatures instead:
        nonzero(*, bool as_tuple) (Triggered internally at  /opt/conda/conda-bld/pytorch_1597302504919/work/torch/csrc/utils/python_arg_parser.cpp:864.)
  masked_index = (input_ids[0] == tokenizer.mask_token_id).nonzero().item()
['good', 'great', 'all', 'really', 'very']
```

well, there is one more issue of using a weird deprecated `nonzero()` invocation, which has to do with some strange undocumented requirement to pass the `as_tuple` arg, since pytorch 1.5 .https://github.com/pytorch/pytorch/issues/43425

we have `authorized_missing_keys`:
 `authorized_missing_keys = [r"final_logits_bias", r"encoder\.version", r"decoder\.version"]`
https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_bart.py#L942
which correctly updates `missing_keys`  - should there be also an `authorized_unexpected_keys` which would clean up `unexpected_keys`?

(note: I re-edited this issue once I understood it better to save reader's time, the history is there if someone needs it)

And found another variety of it: for `['model.encoder.version', 'model.decoder.version']`
```
tests/test_modeling_bart.py::BartModelIntegrationTests::test_mnli_inference Some weights of the model checkpoint at facebook/bart-large-mnli were not used when initializing BartForSequenceClassification: ['model.encoder.version', 'model.decoder.version']
- This IS expected if you are initializing BartForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPretraining model).
- This IS NOT expected if you are initializing BartForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
PASSED
```


[doc] bart doc examples aren't for bart

If you look at the very end of this section https://huggingface.co/transformers/model_doc/bart.html#transformers.BartForConditionalGeneration.generate
there are 5 examples of using `generate` none of which is for BART. Is this an accidental copy-n-paste issue and they should just be removed?

There are examples of generate for BART in the pre-amble of the class:
https://huggingface.co/transformers/model_doc/bart.html#bartforconditionalgeneration


Add model card for electricidad-base-generator

I works like a charm!
Look at the output of the example code!


[Doc model summary] add MBart model summary

add model summary for MBart

@sshleifer , @sgugger


Remove hard-coded uses of float32 to fix mixed precision use

Remove hard-coded uses of float32 from the tensorflow implementation of BERT and ELECTRA.

Fixes #3320


mbart broken in summarization pipeline

```
summarizer = pipeline("summarization", model="facebook/mbart-large-cc25", tokenizer="facebook/mbart-large-cc25")
```


Error when loading my trained model

Hello,

I tried to train the question-answering model using `bert-base-uncased` on SQUADv1.1. The training process seems to be successfully completed. However, when I load the trained model, it said that `File "h5py/h5f.pyx", line 88, in h5py.h5f.open
OSError: Unable to open file (file signature not found)`

Here is my configuration for the training process:
```
model_name_or_path: bert-base-uncased
do_train: True
do_eval: True
overwrite_output_dir: True
num_train_epochs: 10
per_device_train_batch_size: 12
per_device_eval_batch_size: 12
warmup_steps: 100
weight_decay: 0.01
learning_rate: 3e-5
evaluate_during_training: True
save_steps: 5000
```

And here is what I stored in my model directory:
```
checkpoint-10000  checkpoint-35000  checkpoint-55000  pytorch_model.bin
checkpoint-15000  checkpoint-40000  checkpoint-60000  special_tokens_map.json
checkpoint-20000  checkpoint-45000  checkpoint-65000  tokenizer_config.json
checkpoint-25000  checkpoint-5000   checkpoint-70000  training_args.bin
checkpoint-30000  checkpoint-50000  config.json vocab.txt
```

I tried to load my model by
```
self._model = BertForQuestionAnswering.from_pretrained(`./model/trained_squad/`, from_tf=True)
```

It would be appreciated if anyone can give me a clue about what happens here. Is there anything wrong with my training process?

Best,
Yanchao


New config param for cross-attention dimensionality

# 🚀 Feature request

Add a new config param `n_cross` for each model that has cross-attention. This param will determine key and value matrices dimensionality in cross-attention.

## Motivation

I have pretrained encoder (hidden size 512) and want to combine it with GPT-2 medium (hidden size 1024) by cross-attention. Now I cant do it, because in according with this PR https://github.com/huggingface/transformers/commit/1d6e71e1167dea9e026391ec5a1a2d7ec33d22af query and key matrices of cross-attention have dimensionality same as self-attention.

In code it could looks like:
```
config.is_cross_attention = True
config.n_cross = 512
```
And in doc string:
```
n_inner (:obj:`int`, optional, defaults to None):
                Dimensionality of the inner feed-forward layers. :obj:`None` will set it to 4 times n_embd.
n_cross (:obj:`int`, optional, defaults to None):
                Dimensionality of the cross-attention input. :obj:`None` will set it same as n_embd.
```

## Your contribution

Actually I have already fixed this in my fork, but I didn't make any tests and docs. But I would be able to prepare PR within next two weeks.


Dataset and DataCollator for BERT Next Sentence Prediction (NSP) task

Add `DataCollatorForNextSencencePrediction` and `TextDatasetForNextSencencePrediction` to support mlm and next sentence prediction objectives together.


bert finetuning for multilingual question answering

Hi, I'm just learning BERT now, and wanna use pyTorch and apply BERT for question answering.
I browsed the models, and it seems that run_squad.py is appropriate for question answering task.
But I wanna apply it to other languages' question answering tasks, so what I can do is using run_squad.py and switching the train/dev file paths to other languages datasets?
And what if I wanna apply the task to some small data(written in another language. so it will be actual and specific task), the process will be finetuning the "run_sqaud.py" with that language datasets, and then finetuning it again with the small data?

Thank you,


fix order of input/target of cross_entropy


[Docs model summaries] Add pegasus to docs

Stiched together a short model summary for Pegasus. Would be great if @sshleifer and @sgugger can take a look :-)


Run_glue.py, how can I continue previous fine-tuning training?

My previous training on the fine-tuning of glue's SST had been shut down, and now I want to continue my training on the base of the latest checkpoint, how can I implement this ?
<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:


Running squad_convert_examples_to_features causes warnings.

Running function squad_convert_examples_to_features from data/processors/squad.py causes warning. Very annoying to disable warnings when running function many times :)

Warned that tokenization_base_utils.py, max_len is depricated.
`tokenization_utils_base.py:1320: FutureWarning: The max_len attribute has been deprecated and will be removed in a future version, use model_max_length instead.`

Code to reproduce.
```python
from transformers import AutoTokenizer
from transformers.data import SquadExample, squad_convert_examples_to_features
tokenizer =  AutoTokenizer.from_pretrained('a-ware/roberta-large-squadv2')
example = SquadExample(None,'what is test','test is good',None,None,None)
features = squad_convert_examples_to_features(
            examples=[example],
            tokenizer=tokenizer,
            max_seq_length=512,
            doc_stride=128,
            max_query_length=64,
            is_training=False,
            tqdm_enabled=False
        )
```


Add typing.overload for convert_ids_tokens

The annotation of `convert_ids_tokens` is not sufficient. When `ids` are `List[str]` and `str`, the return type are always `List[int]` and `int` respectively.
It can be solved with [typing.overload](https://docs.python.org/3/library/typing.html#typing.overload)


Pre-training a language model on a large dataset

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers
     -->

## Details
<!-- Description of your issue -->

Hi,

I'm getting a memory error when I run the example code for language modeling. I'm interested in pre-training a RoBERTa model using a 25GB text data on a virtual machine with a `v3-8` TPU on Google Cloud Platform.

I'm using the following command with `transformers/examples/xla_spawn.py` and `transformers/examples/run_language_modeling.py`.

```
python xla_spawn.py --num_cores 8 \
run_language_modeling.py \
    --output_dir=[*****] \
    --config_name=[*****] \
    --tokenizer_name=[*****] \
    --do_train \
    --per_device_train_batch_size 8 \
    --gradient_accumulation_steps 128 \
    --learning_rate 6e-4 \
    --weight_decay 0.01 \
    --adam_epsilon 1e-6 \
    --adam_beta1 0.9 \
    --adam_beta2 0.98 \
    --max_steps 500_000 \
    --warmup_steps 24_000 \
    --save_total_limit 5 \
    --save_steps=100_000 \
    --block_size=512 \
    --train_data_file=[*****] \
    --mlm \
    --line_by_line
```

When I run this, I get the following error.
```
08/20/2020 15:21:07 - INFO - transformers.data.datasets.language_modeling -   Creating features from dataset file at [*****]
Traceback (most recent call last):
  File "xla_spawn.py", line 72, in <module>
    main()
  File "xla_spawn.py", line 68, in main
    xmp.spawn(mod._mp_fn, args=(), nprocs=args.num_cores)
  File "/anaconda3/envs/torch-xla-nightly/lib/python3.6/site-packages/torch_xla/distributed/xla_multiprocessing.py", line 395, in spawn
    start_method=start_method)
  File "/anaconda3/envs/torch-xla-nightly/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 158, in start_processes
    while not context.join():
  File "/anaconda3/envs/torch-xla-nightly/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 108, in join
    (error_index, name)
Exception: process 0 terminated with signal SIGKILL
```

It looks like the script gets killed while it's loading the training data [here](https://github.com/huggingface/transformers/blob/573bdb0a5d2897ff6c7520ebb38693c7acfbf17e/src/transformers/data/datasets/language_modeling.py#L89-L92).

```python
with open(file_path, encoding="utf-8") as f:
    lines = [line for line in f.read().splitlines() if (len(line) > 0 and not line.isspace())]
```

When I run the above block of code separately with `transformers/examples/xla_spawn.py`, I get an error.

```
Traceback (most recent call last):
  File "xla_spawn.py", line 72, in <module>
    main()
  File "xla_spawn.py", line 68, in main
    xmp.spawn(mod._mp_fn, args=(), nprocs=args.num_cores)
  File "/anaconda3/envs/torch-xla-nightly/lib/python3.6/site-packages/torch_xla/distributed/xla_multiprocessing.py", line 395, in spawn
    start_method=start_method)
  File "/anaconda3/envs/torch-xla-nightly/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 158, in start_processes
    while not context.join():
  File "/anaconda3/envs/torch-xla-nightly/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 108, in join
    (error_index, name)
Exception: process 0 terminated with signal SIGKILL
```

When I run the above block of code separately using  `n1-highmem-16 (16 vCPUs, 104 GB memory)` without TPU, I still get an error.

```
Traceback (most recent call last):
  File "debug_load.py", line 7, in <module>
    lines = [line for line in f.read().splitlines() if (len(line) > 0 and not line.isspace())]
  File "/anaconda3/envs/torch-xla-nightly/lib/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
MemoryError
```

Has anyone successfully reproduced the original RoBERTa model or pretrained a language model with a large dataset using Huggingface's transformers (with TPU)? If so, what are the specifications of your machine? Has this code (`transformers/examples/run_language_modeling.py`) tested on a large dataset?

<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**: https://discuss.huggingface.co/t/pre-training-a-language-model-on-a-large-dataset/790


How to convert tokenizer output to train_dataset which is required by Trainer API

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers
     -->

## Details
<!-- Description of your issue -->

I tried doing tokenisation using documentation of huggingface transformers.

```
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained('bert-base-cased')
encoded_input = tokenizer(batch_of_sequences)
```

Pre Trained Tokenizer gives output of dictionary containing three keys which are

```
encoded_input = {
'input_ids': [[],[],[]],
'token_type_ids': [[],[],[]],
'attention_mask': [[],[],[]]
}

```

Trainer API requires input of Train & Eval Dataset of type `torch.utils.data.Dataset`.

How can we use this output to create training dataset required for Trainer API?

<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:

https://stackoverflow.com/questions/63519373/how-to-convert-tokenizer-output-to-train-dataset-which-is-required-by-trainer-ap


Fix error class instantiation

The lines I fixed are bugs, causing `TypeError: 'ModuleNotFoundError' object is not callable`


model card for Spanish electra base


Error on `PreTrainedTokenizerBase.batch_encode_plus` with `return_overflowing_tokens=True, truncation=True`

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.0.2 (master branch)
- Platform: macOS-10.14.6-x86_64-i386-64bit
- Python version: 3.8.1
- PyTorch version (GPU?): 1.6.0 (False)
- Tensorflow version (GPU?): 2.3.0 (False)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 blenderbot: @mariamabarham
 Bart: @sshleifer
 Marian: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

 tokenizers: @mfuntowicz

## Information

Model I am using (Bert, XLNet ...): bert-base-uncased

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Run the below code

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

```python
from transformers import BertTokenizer

tokenizer = BertTokenizer.from_pretrained("bert-base-uncased")
tokenizer.batch_encode_plus(
    ["foo", "bar " * 1000], return_overflowing_tokens=True, truncation=True, padding=True
)
```

raises the following error:

```
Traceback (most recent call last):
  File "foo.py", line 4, in <module>
    tokenizer.batch_encode_plus(
  File "/Users/user/work/transformers/src/transformers/tokenization_utils_base.py", line 2121, in batch_encode_plus
    return self._batch_encode_plus(
  File "/Users/user/work/transformers/src/transformers/tokenization_utils.py", line 534, in _batch_encode_plus
    batch_outputs = self._batch_prepare_for_model(
  File "/Users/user/work/transformers/src/transformers/tokenization_utils.py", line 606, in _batch_prepare_for_model
    batch_outputs = self.pad(
  File "/Users/user/work/transformers/src/transformers/tokenization_utils_base.py", line 2305, in pad
    assert all(
AssertionError: Some items in the output dictionnary have a different batch size than others.
```

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

No error


fine tuning with Chinese data LCQMC val_acc not increase

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers
     -->

## Details
<!-- Description of your issue -->
hello everyone
when i use huggingface bert-base-chinese model fine tuning LCQMC dataset ,
the train_acc will raise normal like 0.6->0.8->0.9,but val_acc not increase, it alway same number like 0.56
do you konw what happent?

<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:


Tokenize got an unexpected keyword argument 'pad_to_max_length', 'return_attention_mask'

Core: Tokenization

Internals of the library; Tokenization.

This works fine when I run on my GPU but gives the above error when I try to run on my CPU. They bought have the same environment setup.

Error:
  File "C:\ProgramData\Anaconda3\lib\site-packages\transformers\tokenization_utils.py", line 786, in encode_plus
    first_ids = get_input_ids(text)
  File "C:\ProgramData\Anaconda3\lib\site-packages\transformers\tokenization_utils.py", line 778, in get_input_ids
    return self.convert_tokens_to_ids(self.tokenize(text, **kwargs))
  File "C:\ProgramData\Anaconda3\lib\site-packages\transformers\tokenization_utils.py", line 649, in tokenize
    tokenized_text = split_on_tokens(added_tokens, text)
  File "C:\ProgramData\Anaconda3\lib\site-packages\transformers\tokenization_utils.py", line 646, in split_on_tokens
    else [token] for token in tokenized_text), [])
  File "C:\ProgramData\Anaconda3\lib\site-packages\transformers\tokenization_utils.py", line 646, in <genexpr>
    else [token] for token in tokenized_text), [])
TypeError: _tokenize() got an unexpected keyword argument 'pad_to_max_length'

environment:
Python 3.7
transformers 3.0.2
torch 1.5.1


Remove accidental comment


PreTrainedModel's tie_weights invocation needs to be configurable

`PreTrainedModel` defines `tie_weights` method and then in [one place suggests](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_utils.py#L512)

> Takes care of tying weights embeddings afterwards if the model class has a :obj:`tie_weights()` method.

But since the super-class has it defined, it's always there.

So the only way for a sub-class to avoid this "tying" is to override it with:
```
     def tie_weights(self): pass
```

if nothing else happens that comment needs to be edited to suggest a noop override in the sub-class.

But it took some hunting to get there, so a better solution is needed.

Most likely, currently, most (all?) models in transformers with encoder/decoder share token embed weights, hence the issue didn't come up. I'm working on porting a fairseq transformer and there the enc/dec token embeds aren't shared.

I propose a solution which adds a new param to `PretrainedConfig`, say: `is_enc_dec_sharing_embeds=True` and let the subclass override those, then add at the start of `tie_weights` in `modeling_utils.py`

```
     def tie_weights(self):
         if not self.config.is_enc_dec_sharing_embeds:
             return
```
that way it's easy to quickly become aware that an action needs to be taken and set the desired behavior from within the subclass.

Thoughts?

If the proposed solution is agreeable, please, let me know which config param name it should be `is_enc_dec_sharing_embeds` - or different and I will submit a PR.

Thank you.

**edit:**

OK, having had a closer look:
```
grep -r -A 2  'def tie_weights' src/transformers | grep pass | wc -l
```
we have 5 sub-classes that override it with a no-op so only some rely on the default. Bad superclass, no cookies for you.


BartTokenizerFast cannot decode PyTorch tensors

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.0.2
- Platform: MacOS and Linux
- Python version: 3.6 and 3.7
- PyTorch version (GPU?): 1.6.0 (no and yes)
- Tensorflow version (GPU?): N/A
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 blenderbot: @mariamabarham
 Bart: @sshleifer
 Marian: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

 examples/seq2seq: @sshleifer
(Discovered in #6610.)

## Information

Model I am using (Bert, XLNet ...): Bart.

Any Bart model (reproduced with distilbart-cnn-12-6 and distilbart-xsum-1-1.

## To reproduce

Steps to reproduce the behavior:

```python
In [1]: from transformers import BartTokenizerFast, BartForConditionalGeneration

In [2]: model = BartForConditionalGeneration.from_pretrained("sshleifer/distilbart-xsum-1-1")

In [3]: tokenizer = BartTokenizerFast.from_pretrained("sshleifer/distilbart-xsum-1-1")

In [4]: input_ids = tokenizer("This is a test string.", return_tensors="pt")

In [5]: input_ids
Out[5]: {'input_ids': tensor([[   0,  713,   16,   10, 1296, 6755,    4,    2]]), 'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1]])}

In [6]: summary_ids = model.generate(input_ids['input_ids'], num_beams=4, max_length=5, early_stopping=True)

In [7]: print([tokenizer.decode(g, skip_special_tokens=True, clean_up_tokenization_spaces=False) for g in summary_ids])
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-7-d476aca57720> in <module>
----> 1 print([tokenizer.decode(g, skip_special_tokens=True, clean_up_tokenization_spaces=False) for g in summary_ids])

<ipython-input-7-d476aca57720> in <listcomp>(.0)
----> 1 print([tokenizer.decode(g, skip_special_tokens=True, clean_up_tokenization_spaces=False) for g in summary_ids])

~/.pyenv/versions/finetuning-bart/lib/python3.7/site-packages/transformers/tokenization_utils_fast.py in decode(self, token_ids, skip_special_tokens, clean_up_tokenization_spaces)
    437         self, token_ids: List[int], skip_special_tokens: bool = False, clean_up_tokenization_spaces: bool = True
    438     ) -> str:
--> 439         text = self._tokenizer.decode(token_ids, skip_special_tokens=skip_special_tokens)
    440
    441         if clean_up_tokenization_spaces:

~/.pyenv/versions/finetuning-bart/lib/python3.7/site-packages/tokenizers/implementations/base_tokenizer.py in decode(self, ids, skip_special_tokens)
    265             raise ValueError("None input is not valid. Should be a list of integers.")
    266
--> 267         return self._tokenizer.decode(ids, skip_special_tokens=skip_special_tokens)
    268
    269     def decode_batch(

TypeError:

In [8]:
```

## Expected behavior

Fast tokenizer should be able to decode without producing an error.


Specify config filename in HfArgumentParser

Currently, HfArgumentParser will load arguments from a config file if that config file is the same name as the script being run. So `train.py` would have a corresponding `train.args`. This extends the method to load from any config file that is specified. So `train.py` could use a `bert-large.args` or a `bert-small.args`.


**Specifically for Pegasus-arxiv** -  PegasusForConditionalGeneration - Error in loading state dictionary

Please note that this specifically happens only for **pegasus-arxiv** (Reopening Issue 6609). Before you close, please let me know if this works for pegasus-arxiv. Because I am getting the below error even after taking a fresh git fetch of transformers and removing all checkpoints from cache.

## Environment info
- `transformers` version: 3.0.2
- Platform: Linux-5.3.0-1034-azure-x86_64-with-debian-buster-sid
- Python version: 3.7.7
- PyTorch version (GPU?): 1.6.0 (True)
- Tensorflow version (GPU?): N/A
- Using GPU in script?:
- Using distributed or parallel set-up in script?:

### Who can help
@sshleifer
## Information

Model I am using (Bert, XLNet ...): **google/pegasus-arxiv**

## To reproduce
```ruby
mname = "google/pegasus-arxiv"
model = PegasusForConditionalGeneration.from_pretrained(mname)
```

throws error as below
File "/anaconda/envs/py37_default/lib/python3.7/site-packages/transformers/modeling_utils.py", line 894, in from_pretrained
model.class.name, "\n\t".join(error_msgs)
RuntimeError: Error(s) in loading state_dict for PegasusForConditionalGeneration:
size mismatch for model.encoder.embed_positions.weight: copying a param with shape torch.Size([512, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 1024]).
size mismatch for model.decoder.embed_positions.weight: copying a param with shape torch.Size([512, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 1024]).


Bart: make decoder_input_ids correctly if labels specified.

should call shift_tokens_right(labels), like T5.

cc @patil-suraj: does that make sense?


Fix confusing warnings during TF2 import from PyTorch

1. Swapped missing_keys and unexpected_keys.

2. Copy&paste error caused these warnings to say "from TF 2.0" when it's actually "from PyTorch".


Move threshold up for flaky test with Electra

As discussed on slack, this should take care of the Electra flaky test with PT/TF equivalence.

@LysandreJik just a ping for when you're back so you're aware.


[Tests] fix attention masks in Tests

This PR should fix the flaky test failures of `test_modeling_output_equivalence` and `test_feed_forward_chunking`.

I added a new random attention_mask generation function that makes sure that at least one token is attended to.


Pegasus: OSError: Unable to load weights from pytorch checkpoint file.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.0.2
- Platform: macOS-10.14.6-x86_64-i386-64bit
- Python version: 3.8.5
- PyTorch version (GPU?): 1.6.0 (False)
- Tensorflow version (GPU?): 2.2.0 (False)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
@sshleifer

## Information

Model I am using (Bert, XLNet ...): google/pegasus-cnn_dailymail

The problem arises when using:
```
import torch
from transformers import PegasusForConditionalGeneration, PegasusTokenizer

torch_device = 'cuda' if torch.cuda.is_available() else 'cpu'

model_name = 'google/pegasus-cnn_dailymail'
tokenizer = PegasusTokenizer.from_pretrained(model_name)
model = PegasusForConditionalGeneration.from_pretrained(model_name).to(torch_device)
```

Traceback:
```
RuntimeError                              Traceback (most recent call last)
~/projects/transformers/src/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    854             try:
--> 855                 state_dict = torch.load(resolved_archive_file, map_location="cpu")
    856             except Exception:

~/anaconda3/envs/abstractive_summarizer/lib/python3.8/site-packages/torch/serialization.py in load(f, map_location, pickle_module, **pickle_load_args)
    584                 return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
--> 585         return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
    586

~/anaconda3/envs/abstractive_summarizer/lib/python3.8/site-packages/torch/serialization.py in _legacy_load(f, map_location, pickle_module, **pickle_load_args)
    771         assert key in deserialized_objects
--> 772         deserialized_objects[key]._set_from_file(f, offset, f_should_read_directly)
    773         if offset is not None:

RuntimeError: unexpected EOF, expected 10498989 more bytes. The file might be corrupted.

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
<ipython-input-1-1ae6eb884edd> in <module>
      7 model_name = 'google/pegasus-cnn_dailymail'
      8 tokenizer = PegasusTokenizer.from_pretrained(model_name)
----> 9 model = PegasusForConditionalGeneration.from_pretrained(model_name).to(torch_device)

~/projects/transformers/src/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    855                 state_dict = torch.load(resolved_archive_file, map_location="cpu")
    856             except Exception:
--> 857                 raise OSError(
    858                     "Unable to load weights from pytorch checkpoint file. "
    859                     "If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True. "

OSError: Unable to load weights from pytorch checkpoint file. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.
```


[DistilBert] Flaky tests

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.0.2
- Platform: Linux-5.3.0-61-generic-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.5
- PyTorch version (GPU?): 1.6.0+cpu (False)
- Tensorflow version (GPU?): 2.1.1 (False)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Reproduce Error:

```python
#!/usr/bin/env python3
import torch
from transformers import DistilBertModel, DistilBertConfig

input_ids = torch.tensor([[55, 40, 88, 37, 12,  6, 20],
        [33, 87, 56,  6, 34, 92,  2],
        [ 4, 25, 95, 19,  9, 14, 80],
        [96, 45, 71, 10, 78, 33, 68],
        [72, 40, 59, 90,  5, 78, 44],
        [36, 15, 11, 18, 74, 40, 30],
        [84, 25,  5, 61, 18, 77, 35],
        [70, 87,  9, 42, 24, 65, 11],
        [28,  0, 28, 45, 92, 83, 96],
        [75, 41, 69, 61, 83, 31, 81],
        [94, 93, 79, 48, 24, 17,  9],
        [97,  5, 38, 94, 75,  8, 59],
        [31, 71, 87, 39, 97, 10, 22]])

attention_mask = torch.tensor([[1, 1, 1, 1, 0, 1, 0],
        [0, 0, 0, 0, 0, 0, 0],
        [1, 1, 1, 1, 1, 1, 1],
        [0, 1, 1, 1, 0, 0, 1],
        [1, 1, 1, 0, 1, 1, 0],
        [1, 1, 1, 1, 1, 1, 1],
        [0, 0, 1, 0, 0, 1, 1],
        [1, 0, 1, 0, 0, 0, 1],
        [0, 1, 0, 1, 1, 1, 0],
        [0, 1, 1, 0, 0, 0, 0],
        [0, 1, 0, 1, 1, 0, 1],
        [0, 1, 1, 1, 1, 1, 1],
        [0, 1, 0, 1, 0, 0, 0]])

distil_bert_config = {
  "activation": "gelu",
  "attention_dropout": 0.1,
  "dim": 32,
  "dropout": 0.1,
  "hidden_act": "gelu",
  "hidden_dim": 37,
  "initializer_range": 0.02,
  "max_position_embeddings": 512,
  "model_type": "distilbert",
  "n_heads": 4,
  "n_layers": 5,
  "pad_token_id": 0,
  "qa_dropout": 0.1,
  "return_dict": True,
  "seq_classif_dropout": 0.2,
  "sinusoidal_pos_embds": False,
  "vocab_size": 99
}

config = DistilBertConfig(**distil_bert_config)
torch.manual_seed(0)
model = DistilBertModel(config).eval()

last_hidden_state = model(input_ids, attention_mask=attention_mask)[0]

if torch.isnan(last_hidden_state).any().item():
    print("Error with DistilBert")
```

This code example allows yields nan values.


TFTrainer dataset doc & fix evaluation bug

discussed in #6551


unk handling in v3.0 different than v2.0?

Internals of the library; Tokenization.

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers
     -->

## Details
<!-- Description of your issue -->
I have this problem in runing my code in transformers3.0.2,
Trainable parameters: 102274668
248
250
252
252
248
250
252
252
Traceback (most recent call last):
  File "train.py", line 92, in <module>
    run('./configs/base_config.json')
  File "train.py", line 88, in run
    main(config)
  File "train.py", line 66, in main
    trainer.train()
  File "/DATA2/disk1/wangbingchen/project/ccks2020-task8-pytorch/base/base_trainer.py", line 67, in train
    result = self._train_epoch(epoch)
  File "/DATA2/disk1/wangbingchen/project/ccks2020-task8-pytorch/trainer/trainer.py", line 59, in _train_epoch
    for batch_idx, batch_data in enumerate(self.train_iter):
  File "/home/wangbingchen/wangbingchen/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 345, in __next__
    data = self._next_data()
  File "/home/wangbingchen/wangbingchen/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 385, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/home/wangbingchen/wangbingchen/anaconda3/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 47, in fetch
    return self.collate_fn(data)
  File "/DATA2/disk1/wangbingchen/project/ccks2020-task8-pytorch/data_process/military_data_process.py", line 179, in collate_fn
    text_token_ids = torch.LongTensor(np.array(text_token_ids))
TypeError: can't convert np.ndarray of type numpy.object_. The only supported types are: float64, float32, float16, int64, int32, int16, int8, uint8, and bool.
this is the problem of embedding length.But when I run this code in transformers2.11,everything is well.So,I want to ask the difference of transformers2.11 and transformers3.0.2.Thanks very much!
<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:


Fine tune masked language model on custom dataset 'index out of range in self'

# ❓ Questions & Help

Hi  I am training a mlm model with reference of this tutorial, https://huggingface.co/blog/how-to-train. However,I got the error of 'index out of range in self' but I already set max_length as well as block size in my code.  I am also not clear how to load and prepare data, should I mask certain words by myself? and pass the masked label while training by myself?  Because herehttps://huggingface.co/transformers/model_doc/bert.html#bertformaskedlm  I see '– Labels for computing the masked language modeling loss. Indices should be in [-100, 0, ..., config.vocab_size]' I am not sure using Trainer and DataCollatorForLanguageModeling could solve this.
## Details
<!-- Description of your issue -->
To reprpduce:

```
tokenizer = BertTokenizer.from_pretrained('./bert-large-cased',truncation = True, padding=True, max_length=100)
model = BertForMaskedLM.from_pretrained(
    "./bert-large-cased",
    output_attentions = False,
    output_hidden_states = True
)
device = 'cuda' if torch.cuda.is_available() else 'cpu' # Tell pytorch to run this model on the GPU.
model = model.to(device)
model.train()

from transformers import DataCollatorForLanguageModeling
data_collator = DataCollatorForLanguageModeling(
    tokenizer=tokenizer,mlm=True, mlm_probability=0.15  #I don't know if this is the only way to set up mask for mlm task....
)

from transformers import LineByLineTextDataset
train_dataset = LineByLineTextDataset(
    tokenizer=tokenizer,
    file_path="./na_en_train.txt", #I reorgnasized data to line by line form as the tutorial, which is stupid. but  I also tried TensorDataset, it got errors
    block_size=100,
)
eval_dataset = LineByLineTextDataset(
    tokenizer=tokenizer,
    file_path="./na_en_test.txt",
    block_size=100,
)
from transformers import Trainer, TrainingArguments
training_args = TrainingArguments(
    output_dir="./results",
    overwrite_output_dir=True,
    num_train_epochs=3,
    per_device_train_batch_size=16,  # batch size per device during training
    per_device_eval_batch_size=16,
    save_steps=10000,
    save_total_limit=2
)
trainer = Trainer(
    model=model,
    args=training_args,
    data_collator=data_collator,
    train_dataset=train_dataset,
    eval_dataset=eval_dataset,
    prediction_loss_only=True,
)
trainer.train()

```

#Error Message:
---------------------------------------------------------------------------

> IndexError                                Traceback (most recent call last)
> <ipython-input-116-3435b262f1ae> in <module>
> ----> 1 trainer.train()
>
> /opt/conda/envs/rapids/lib/python3.6/site-packages/transformers/trainer.py in train(self, model_path)
>     497                     continue
>     498
> --> 499                 tr_loss += self._training_step(model, inputs, optimizer)
>     500
>     501                 if (step + 1) % self.args.gradient_accumulation_steps == 0 or (
>
> /opt/conda/envs/rapids/lib/python3.6/site-packages/transformers/trainer.py in _training_step(self, model, inputs, optimizer)
>     620             inputs["mems"] = self._past
>     621
> --> 622         outputs = model(**inputs)
>     623         loss = outputs[0]  # model outputs are always tuple in transformers (see doc)
>     624
>
> /opt/conda/envs/rapids/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
>     548         .. note::
>     549             This method modifies the module in-place.
> --> 550
>     551         Args:
>     552             device (:class:`torch.device`): the desired device of the parameters
>
> /opt/conda/envs/rapids/lib/python3.6/site-packages/transformers/modeling_bert.py in forward(self, input_ids, attention_mask, token_type_ids, position_ids, head_mask, inputs_embeds, labels, encoder_hidden_states, encoder_attention_mask, output_attentions, output_hidden_states, **kwargs)
>    1081             encoder_attention_mask=encoder_attention_mask,
>    1082             output_attentions=output_attentions,
> -> 1083             output_hidden_states=output_hidden_states,
>    1084         )
>    1085
>
> /opt/conda/envs/rapids/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
>     548         .. note::
>     549             This method modifies the module in-place.
> --> 550
>     551         Args:
>     552             device (:class:`torch.device`): the desired device of the parameters
>
> /opt/conda/envs/rapids/lib/python3.6/site-packages/transformers/modeling_bert.py in forward(self, input_ids, attention_mask, token_type_ids, position_ids, head_mask, inputs_embeds, encoder_hidden_states, encoder_attention_mask, output_attentions, output_hidden_states)
>     751
>     752         embedding_output = self.embeddings(
> --> 753             input_ids=input_ids, position_ids=position_ids, token_type_ids=token_type_ids, inputs_embeds=inputs_embeds
>     754         )
>     755         encoder_outputs = self.encoder(
>
> /opt/conda/envs/rapids/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
>     548         .. note::
>     549             This method modifies the module in-place.
> --> 550
>     551         Args:
>     552             device (:class:`torch.device`): the desired device of the parameters
>
> /opt/conda/envs/rapids/lib/python3.6/site-packages/transformers/modeling_bert.py in forward(self, input_ids, token_type_ids, position_ids, inputs_embeds)
>     176
>     177         if inputs_embeds is None:
> --> 178             inputs_embeds = self.word_embeddings(input_ids)
>     179         position_embeddings = self.position_embeddings(position_ids)
>     180         token_type_embeddings = self.token_type_embeddings(token_type_ids)
>
> /opt/conda/envs/rapids/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
>     548         .. note::
>     549             This method modifies the module in-place.
> --> 550
>     551         Args:
>     552             device (:class:`torch.device`): the desired device of the parameters
>
> /opt/conda/envs/rapids/lib/python3.6/site-packages/torch/nn/modules/sparse.py in forward(self, input)
>     112             assert list(_weight.shape) == [num_embeddings, embedding_dim], \
>     113                 'Shape of weight does not match num_embeddings and embedding_dim'
> --> 114             self.weight = Parameter(_weight)
>     115         self.sparse = sparse
>     116
>
> /opt/conda/envs/rapids/lib/python3.6/site-packages/torch/nn/functional.py in embedding(input, weight, padding_idx, max_norm, norm_type, scale_grad_by_freq, sparse)
>    1722     if not torch.jit.is_scripting():
>    1723         if type(input) is not Tensor and has_torch_function((input,)):
> -> 1724             return handle_torch_function(hardswish, (input,), input, inplace=inplace)
>    1725     if inplace:
>    1726         return torch._C._nn.hardswish_(input)
>
> IndexError: index out of range in self
> **A link to original question on the forum/Stack Overflow**: https://discuss.huggingface.co/t/fine-tune-masked-language-model-on-custom-dataset/747


I can't reproduce the results of tf-xlm-r-ner-40-lang model

I tired to reproduce the results of tf-xlm-r-ner-40-lang model but there were compatibility issues in the token-classification/run_tf_ner.py
I fixed some of these issue but it's still not running as expected, @jplu could you please share with me the transformers/tf version used to produce the results of tf-xlm-r-ner-40-lang model.

i'm using:
transformers = 3.0.2
tensorflow = 2.3.0

Thanks in advance.


removed redundant arg in prepare_inputs

I am not sure why `model` was being passed in `_prepare_inputs`. It seemed redundant.


FillMaskPipeline return special tokens i.e. <mask> as prediction

# ❓ FillMaskPipeline return special tokens i.e. \<mask\> as prediction

## Details
Im training a new language model from scratch using ByteLevelBPETokenizer for tokenizer and RobertaForMaskedLM for self-supervised language model. The config is as follow:

Tokenizer:

```
tokenizer = ByteLevelBPETokenizer()

tokenizer.train(files=paths, vocab_size=100000, min_frequency=2, special_tokens=[
    "<s>",
    "<pad>",
    "</s>",
    "<unk>",
    "<mask>",
])
```

After the tokenizer was trained, it was loaded using

`tokenizer = RobertaTokenizerFast.from_pretrained("./BERTmese/Tokenizer", max_length=512)`

And the RobertaForMaskedLM  is configed as follow:

```
config = RobertaConfig(
    vocab_size=100000,
    max_position_embeddings=514,
    num_attention_heads=12,
    num_hidden_layers=6,
)
model = RobertaForMaskedLM(config=config)
```

Then I train the RobertaForMaskedLM  model using custom dataset and Adam optimizer as follow:
```
class CustomTextDataset(Dataset):
    def __init__(self, tokenizer, dataset, max_length):
        self.tokenizer = tokenizer
        self.dataset = dataset
        self.max_length = max_length

    def __len__(self):
        return len(self.dataset)

    def __getitem__(self, i) -> torch.Tensor:
        encoding = self.tokenizer(self.dataset[i]['text'],
                                add_special_tokens=True,
                                max_length=self.max_length,
                                padding='max_length',
                                truncation=True)
        return torch.tensor(encoding["input_ids"], dtype=torch.long)
```
```
for step in data_loader:
    input_ids = next(iter(train_loader))
    input_ids = input_ids.to(device)

    outputs = model(input_ids, labels=input_ids)
    loss, _ = outputs[:2]
    loss.backward()
    ...
```
For every 100 steps, I test the trained model using this code snippet:
```
model.eval()
# test on a sample text

fill_mask = FillMaskPipeline(
    model=model,
    tokenizer=tokenizer,
    topk=1
)

mask = fill_mask("ယခုလတွင်ပျားရည်နှင့်ပျားဖယောင်းများကိုစုဆောင်း<mask>သည်ဟုခန့်မှန်းနိုင်သည်။")[0]

print(mask['token'], decoder.decode([mask['token_str']]), mask['score'])

# set model back to train mode

model.train()
```
In the first hundred steps, the predict token is fined (it predict some Burmese word). However, after that, the model start to predict the '\<mask\>' token as the result. Im not sure if predicting \<mask\> token for masked language is normal?


wip/mbart: make batches that are identical to fairseq

This seems to + 0.2 BLEU


TextGenerationPipeline giving FutureWarning about AutoModelWithLMHead

@TevenLeScao
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.0.2
- Platform: Linux-4.19.112+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6
- PyTorch version (GPU?): 1.6.0 (False)
- Tensorflow version (GPU?): 2.3.0 (True)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

## Information

Model I am using (Bert, XLNet ...): GPT2LMHeadModel

The TextGenerationPipeline object causes FutureWarning about class `AutoModelWithLMHead` being deprecated
```
from transformers import pipeline
text_generator = pipeline("text-generation", model="pranavpsv/gpt2-genre-story-generator")

```

## To reproduce

Steps to reproduce the behavior:

Run the above script. This is the warning:

```
/usr/local/lib/python3.6/dist-packages/transformers/modeling_auto.py:798: FutureWarning: The class `AutoModelWithLMHead` is deprecated and will be removed in a future version. Please use `AutoModelForCausalLM` for causal language models, `AutoModelForMaskedLM` for masked language models and `AutoModelForSeq2SeqLM` for encoder-decoder models.
  FutureWarning,

```
## Expected behavior
I expected no warning since I thought the TextGenerationPipeline would use AutoModelForCausalLM object or GPT2LMHeadModel object for the model.


[seq2seq Example] Convert tensor to List[int] for decoding

Ran into an error today while using `finetune.py` where decoding kept failing because the output was a `torch.Tensor` instead of a `List[int]`. This fixes that.


PegasusForConditionalGeneration - Error in loading state dictionary

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.0.2
- Platform: Linux-5.3.0-1034-azure-x86_64-with-debian-buster-sid
- Python version: 3.7.7
- PyTorch version (GPU?): 1.6.0 (True)
- Tensorflow version (GPU?): not installed
- Using GPU in script?: Tried both
- Using distributed or parallel set-up in script?: No

### Who can help
@sshleifer

## Information

Model I am using (Bert, XLNet ...): google/pegasus-arxiv

The problem arises when using:
the official example scripts:

The tasks I am working on is:
generating summary using pegasus-arxiv

## To reproduce

Steps to reproduce the behavior:
run the below script
```ruby
mname = "google/pegasus-arxiv"
model = PegasusForConditionalGeneration.from_pretrained(mname)
```

This is throwing error
 File "abstractive_summarizer.py", line 21, in <module>
    model = PegasusForConditionalGeneration.from_pretrained(mname, force_download=True)
  File "/anaconda/envs/py37_default/lib/python3.7/site-packages/transformers/modeling_utils.py", line 894, in from_pretrained
    model.__class__.__name__, "\n\t".join(error_msgs)
RuntimeError: Error(s) in loading state_dict for PegasusForConditionalGeneration:
	size mismatch for model.encoder.embed_positions.weight: copying a param with shape torch.Size([512, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 1024]).
	size mismatch for model.decoder.embed_positions.weight: copying a param with shape torch.Size([512, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 1024]).

## Expected behavior
I tried running a sample in console this morning and it worked fine and I was able to generate summary using pegasus-arxiv. Once I transferred this to Jupyter notebook for some trial purpose, it downloaded pegasus-arxiv and after the same has been giving this error.

(If not able to simulate, please try with force_download=True)


How to use Huggingface model for continuous values directly?

Hi, I have a dataset which contains continuous values [ batch_size, features ]

Features look like this :

`[0.49221584, -0.021571456, -0.0920076, -0.14408934, -0.62306774]`

I want to apply transformer model on these values and pass it to the final layer, something like this

`batch_data ==> Transformer ==> output_layer ==> classification`

Currently, I am using hand-coded multi-head attention and norm with the feed-forward network to pass these values to the transformer block.

I Gone through hugging face models, but all the models accept tokens and sequences, Is there any way/hack How I can use hugging face transformer models on direct continuous values?


[Longformer] try if multi gpu works


Regression test for pegasus bugfix

The bug was that tokenizer.max_model_length (set through `tokenizer_config.json`) was sometimes 1024, but `max_position_embeddings` was only 512.  This means that the tokenizer can produce inputs of length 513, which will produce an IndexError when we try to get the from the embedding table.

Since the position embeddings are static for pegasus, the fix was on s3: set the max_position_embeddings to the correct value, and remove the saved, incorrectly sized position embeddings from the state dict.

The tradeoff here is that users can now pass max_position_embeddings=HUGE to the model without error, and pass huge inputs, and either OOM or get shitty performance. But if you are modifying config you sort of know what you are doing so I'm OK with it.

This PR:
- suppresses the warning created by the S3 change
- adds a regression test that `config.max_position_embeddings >= tokenizer.max_model_length`


Add tests to Trainer

This PR moves the tests of the various `data_collator` in `test_data_collator.py` and adds tests of the Trainer on a simple regression problem. While testing, a few problems were uncovered:
- The number of epochs is documented as a float but used as an int, fixed the documentation.
- There was one more step done than specified by the argument `max_steps`.
- The evaluation loss was wrong whenever the evaluation dataset length is not a round multiple of the batch size.

Those three things are also fixed in the PR. With the regression infrastructure, we can add more tests (for custom data collator, optimizers, schedulers etc...) since each training is fast. Will do in follow-up PRs as this one was starting to be of a decent size already.


1. Swapped missing_keys and unexpected_keys.
2. Copy&paste error caused these warnings to say "from TF 2.0" when it's actually "from PyTorch".


[cleanup] remove confusing newline


Create README.md

model card

Related to pretrained model cards


Fix GPT2DoubleHeadsModel to work with model.generate()

Related to pretrained model cards

`token_type_ids` were not passed through `model_specific_kwargs` nor cut properly with `use_cache`/`past` usage (maybe this should be corrected through bigger PR for all models available for caching).


[wip] Seq2SeqTrainer

Related to pretrained model cards

cc @sshleifer


Pegasus: IndexError: index out of range in self

@sshleifer

## Environment info
- `transformers` version: 3.0.2
- Platform: macOS-10.14.6-x86_64-i386-64bit
- Python version: 3.8.5
- PyTorch version (GPU?): 1.6.0 (False)
- Tensorflow version (GPU?): 2.2.0 (False)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

## Information
Model I am using: google/pegasus-cnn_dailymail

The problem arises when using:
```
import torch
from transformers import AutoModelWithLMHead, AutoTokenizer

torch_device = 'cuda' if torch.cuda.is_available() else 'cpu'

model = AutoModelWithLMHead.from_pretrained("google/pegasus-cnn_dailymail")
tokenizer = AutoTokenizer.from_pretrained("google/pegasus-cnn_dailymail")

src_text = [
    '(CNN)James Best, best known for his portrayal of bumbling sheriff Rosco P. Coltrane on TV\'s "The Dukes of Hazzard," died Monday after a brief illness. He was 88. Best died in hospice in Hickory, North Carolina, of complications from pneumonia, said Steve Latshaw, a longtime friend and Hollywood colleague. Although he\'d been a busy actor for decades in theater and in Hollywood, Best didn\'t become famous until 1979, when "The Dukes of Hazzard\'s" cornpone charms began beaming into millions of American homes almost every Friday night. For seven seasons, Best\'s Rosco P. Coltrane chased the moonshine-running Duke boys back and forth across the back roads of fictitious Hazzard County, Georgia, although his "hot pursuit" usually ended with him crashing his patrol car. Although Rosco was slow-witted and corrupt, Best gave him a childlike enthusiasm that got laughs and made him endearing. His character became known for his distinctive "kew-kew-kew" chuckle and for goofy catchphrases such as "cuff \'em and stuff \'em!" upon making an arrest. Among the most popular shows on TV in the early \'80s, "The Dukes of Hazzard" ran until 1985 and spawned TV movies, an animated series and video games. Several of Best\'s "Hazzard" co-stars paid tribute to the late actor on social media. "I laughed and learned more from Jimmie in one hour than from anyone else in a whole year," co-star John Schneider, who played Bo Duke, said on Twitter. "Give Uncle Jesse my love when you see him dear friend." "Jimmy Best was the most constantly creative person I have ever known," said Ben Jones, who played mechanic Cooter on the show, in a Facebook post. "Every minute of his long life was spent acting, writing, producing, painting, teaching, fishing, or involved in another of his life\'s many passions." Born Jewel Guy on July 26, 1926, in Powderly, Kentucky, Best was orphaned at 3 and adopted by Armen and Essa Best, who renamed him James and raised him in rural Indiana. Best served in the Army during World War II before launching his acting career. In the 1950s and 1960s, he accumulated scores of credits, playing a range of colorful supporting characters in such TV shows as "The Twilight Zone," "Bonanza," "The Andy Griffith Show" and "Gunsmoke." He later appeared in a handful of Burt Reynolds\' movies, including "Hooper" and "The End." But Best will always be best known for his "Hazzard" role, which lives on in reruns. "Jimmie was my teacher, mentor, close friend and collaborator for 26 years," Latshaw said. "I directed two of his feature films, including the recent \'Return of the Killer Shrews,\' a sequel he co-wrote and was quite proud of as he had made the first one more than 50 years earlier." People we\'ve lost in 2015 . CNN\'s Stella Chan contributed to this story.'
]
batch = tokenizer.prepare_seq2seq_batch(src_text, truncation=True, padding='longest').to(torch_device)
translated = model.generate(**batch)
tgt_text = tokenizer.batch_decode(translated, skip_special_tokens=True)
```

The tasks I am working on is:
* abstractive text summarization

## To reproduce
1. Run the script above. The traceback:
```
IndexError                                Traceback (most recent call last)
<ipython-input-1-ac154ffeb574> in <module>
     13 ]
     14 batch = tokenizer.prepare_seq2seq_batch(src_text, truncation=True, padding='longest').to(torch_device)
---> 15 translated = model.generate(**batch)
     16 tgt_text = tokenizer.batch_decode(translated, skip_special_tokens=True)

~/anaconda3/envs/abstractive_summarizer/lib/python3.8/site-packages/torch/autograd/grad_mode.py in decorate_context(*args, **kwargs)
     13         def decorate_context(*args, **kwargs):
     14             with self:
---> 15                 return func(*args, **kwargs)
     16         return decorate_context
     17

~/projects/transformers/src/transformers/generation_utils.py in generate(self, input_ids, max_length, min_length, do_sample, early_stopping, num_beams, temperature, top_k, top_p, repetition_penalty, bad_words_ids, bos_token_id, pad_token_id, eos_token_id, length_penalty, no_repeat_ngram_size, num_return_sequences, attention_mask, decoder_start_token_id, use_cache, **model_specific_kwargs)
    394             encoder = self.get_encoder()
    395
--> 396             encoder_outputs: tuple = encoder(input_ids, attention_mask=attention_mask)
    397
    398         # Expand input ids if num_beams > 1 or num_return_sequences > 1

~/anaconda3/envs/abstractive_summarizer/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    720             result = self._slow_forward(*input, **kwargs)
    721         else:
--> 722             result = self.forward(*input, **kwargs)
    723         for hook in itertools.chain(
    724                 _global_forward_hooks.values(),

~/projects/transformers/src/transformers/modeling_bart.py in forward(self, input_ids, attention_mask, output_attentions, output_hidden_states, return_dict)
    328
    329         inputs_embeds = self.embed_tokens(input_ids) * self.embed_scale
--> 330         embed_pos = self.embed_positions(input_ids)
    331         x = inputs_embeds + embed_pos
    332         x = self.layernorm_embedding(x)

~/anaconda3/envs/abstractive_summarizer/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    720             result = self._slow_forward(*input, **kwargs)
    721         else:
--> 722             result = self.forward(*input, **kwargs)
    723         for hook in itertools.chain(
    724                 _global_forward_hooks.values(),

~/anaconda3/envs/abstractive_summarizer/lib/python3.8/site-packages/torch/autograd/grad_mode.py in decorate_context(*args, **kwargs)
     13         def decorate_context(*args, **kwargs):
     14             with self:
---> 15                 return func(*args, **kwargs)
     16         return decorate_context
     17

~/projects/transformers/src/transformers/modeling_bart.py in forward(self, input_ids, use_cache)
   1337             # starts at 0, ends at 1-seq_len
   1338             positions = torch.arange(seq_len, dtype=torch.long, device=self.weight.device)
-> 1339         return super().forward(positions)

~/anaconda3/envs/abstractive_summarizer/lib/python3.8/site-packages/torch/nn/modules/sparse.py in forward(self, input)
    122
    123     def forward(self, input: Tensor) -> Tensor:
--> 124         return F.embedding(
    125             input, self.weight, self.padding_idx, self.max_norm,
    126             self.norm_type, self.scale_grad_by_freq, self.sparse)

~/anaconda3/envs/abstractive_summarizer/lib/python3.8/site-packages/torch/nn/functional.py in embedding(input, weight, padding_idx, max_norm, norm_type, scale_grad_by_freq, sparse)
   1812         # remove once script supports set_grad_enabled
   1813         _no_grad_embedding_renorm_(weight, input, max_norm, norm_type)
-> 1814     return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
   1815
   1816

IndexError: index out of range in self
```

## Expected behavior
IndexError: index out of range in self


tf2 transformers cache dir

what is the directory of tf2 transformers cache weights?

Thanks


Fix #6575

Make docs clearer as mentioned in #6575


Typo fix in 04-onnx-export


Add "Leveraging Pretrained Checkpoints for Generation" Seq2Seq models.

This PR adds the models from the following paper:

Paper: https://arxiv.org/pdf/1907.12461.pdf

The paper does a great job at showing how pretrained BERT & RoBERTa model can be leveraged for Seq2Seq tasks and yields good results on many seq2seq tasks. It's fits very well with the current implementation of the EncoderDecoder framework.

This PR adds code to port all pretrained encoder decoder models that can be found here: https://tfhub.dev/s?module-type=text-generation&subtype=module,placeholder,

which can be found here: https://huggingface.co/models?search=google%2Froberta
and here: https://huggingface.co/models?search=google%2Fbert2

An example of how a model can be used is here:
https://huggingface.co/google/roberta2roberta_L-24_bbc

Big thanks to @shashiongithub for providing me with the tokenizer files and giving valuable insights on setting the correct generation parameters!


[Tests common] Fix flaky test

The test `test_model_outputs_equivalence` fails quite often at the moment because of a problem with `nan - nan`. This should solve the issue.

Also added more explicit error message in case test error occurs.

Flaky error happens here for example: https://app.circleci.com/pipelines/github/huggingface/transformers/10798/workflows/44e689b2-f4b3-49be-88b3-a5b214eac6c5/jobs/75173


Unable to save and load RoBERTa model using tensorflow

I have trained my model with Roberta-base and tested, it works.
But when i try to save the model I get following error:

/usr/local/lib/python3.6/dist-packages/tensorflow/python/saved_model/save.py in list_dependencies(self, obj)
    126                   obj._object_identifier,  # pylint: disable=protected-access
    127                   name,
--> 128                   extra_dependencies.keys()))
    129         yield base.TrackableReference(name, extra_dependencies[name])
    130       else:

ValueError: Error when exporting object <tensorflow.python.keras.layers.core.Activation object at 0x7fef7d287128> of with identifier=_tf_keras_layer. The object has an attribute named regularization_losses, which is reserved. List of all reserved attributes: dict_keys(['regularization_losses', 'variables', 'trainable_variables', 'keras_api'])

Unable to solve it.

Is it some obvious mistake am doing?

# Code:

path = '/path/model_1'
tf.keras.models.save_model(model_roberta_base,path, overwrite=True,include_optimizer=False,save_format='tf')
new_model = tf.keras.models.load_model(path)
result =predct_func(new_model,"Technology driven by Medical imaging and Artificial Intellignence")
plot_result(result)


tf generation utils: remove unused kwargs


Delete Unused TFModelTesterMixin attributes

Help wanted

Extra attention is needed, help appreciated

Afaict, neither `test_torchscript` or `test_pruning` is used.

https://github.com/huggingface/transformers/blob/master/tests/test_modeling_tf_common.py#L76


[seq2seq] finetune.sh  OOMs in fp16 w torch 1.6 on colab

On trying to fine-tune either T5 or BART models for summarization I was encountering OOM repeatedly in the latest code whereas it used to work fine earlier for me, atleast on Google Colab.
On checking the startup scripts and latest commits I saw that optimizations have been added for native pytorch fp16 support recently. On removing the fp16 parameter from the script it started working as expected.
Please check if this could be a real issue or just a matter of a dangling parameter that needs to be removed?
Thanks

@sshleifer @patil-suraj


all_hidden_states indentation bug in modeling_bert.py

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- Platform:
- Python version:
- PyTorch version (GPU?):
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 blenderbot: @mariamabarham
 Bart: @sshleifer
 Marian: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->
@lavanyashukla
## Information

Model I am using (Bert, XLNet ...): Bert

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1.
2.
3.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->
https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_bert.py#L490
## Expected behavior
https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_bert.py#L490 There is a small bug here, I suppose Line491 and Line491 should be inside loop chunk.
<!-- A clear and concise description of what you would expect to happen. -->


Fix bart base test


wip: Code to add lang tags to marian model cards


Failing bart-base slow test

```
=================================== FAILURES ===================================
____________ BartModelIntegrationTests.test_bart_base_mask_filling _____________
[gw0] linux -- Python 3.7.6 /home/hf/actions-runner_transformers/_work/transformers/transformers/.env/bin/python

self = <tests.test_modeling_bart.BartModelIntegrationTests testMethod=test_bart_base_mask_filling>

    @slow
    def test_bart_base_mask_filling(self):
        pbase = pipeline(task="fill-mask", model="facebook/bart-base")
        src_text = [" I went to the <mask>."]
        results = [x["token_str"] for x in pbase(src_text)]
        expected_results = ["Ġbathroom", "Ġrestroom", "Ġhospital", "Ġkitchen", "Ġcar"]
>       self.assertListEqual(results, expected_results)
E       AssertionError: Lists differ: ['Ġlibrary', 'Ġhospital', 'Ġbathroom', 'Ġmovies', 'Ġpolice'] != ['Ġbathroom', 'Ġrestroom', 'Ġhospital', 'Ġkitchen', 'Ġcar']
E
E       First differing element 0:
```

https://github.com/huggingface/transformers/runs/996031882?check_suite_focus=true


Failing ONNX Slow Test

```
___________________ OnnxExportTestCase.test_quantize_pytorch ___________________
[gw0] linux -- Python 3.7.6 /home/hf/actions-runner_transformers/_work/transformers/transformers/.env/bin/python

self = <tests.test_onnx.OnnxExportTestCase testMethod=test_quantize_pytorch>

    @require_torch
    @slow
    def test_quantize_pytorch(self):
        for model in OnnxExportTestCase.MODEL_TO_TEST:
            path = self._test_export(model, "pt", 12)
            quantized_path = quantize(path)

            # Ensure the actual quantized model is not bigger than the original one
>           if quantized_path.stat().st_size >= Path(path).stat().st_size:
E           AttributeError: 'NoneType' object has no attribute 'stat'

tests/test_onnx.py:76: AttributeError
=============================== warnings summary ===============================

```

https://github.com/huggingface/transformers/runs/996031882?check_suite_focus=true


add intro to nlp lib & dataset links to custom datasets tutorial

Being a bit more explicit with the recommendation to use NLP where possible. Adds a link to the hub for each dataset used as well as a brief introduction to the NLP lib at the bottom.


BatchEncoding interacts poorly with apex.amp

## Environment info
- `transformers` version: 3.0.0
- Platform: Linux
- Python version: 3.6
- PyTorch version (GPU?): 1.6.0 GPU

### Who can help
tokenizers: @mfuntowicz

## Information

We are building a multi-task transformer on top of the transformers API, using Nvidia's Apex to get the best speed out of the GPUs. When using Apex with `O2` optimisations (which are necessary to get speedups on our model), it inserts casts around the inputs to `forward` which in our case is a `BatchEncoding`. This cast blindly calls `BatchEncoding.to` with a `dtype` argument, which in turn is passed through to the values `to` method because there is only a type hint `str` on the `BatchEncoding.to` input argument, not a `isinstance` check. This causes our tensor of BPE ids to be cast from a `LongTensor` into `HalfTensor` and then the embedding layer raises an error because it's not expecting floating point values (also we rounded away most of our vocabulary terms).

## To reproduce

Run amp.initialize with the `O2` opt level on a model which's forward method accepts BatchEncoding as the input.

This [line](https://github.com/NVIDIA/apex/blob/4ef930c1c884fdca5f472ab2ce7cb9b505d26c1a/apex/amp/_initialize.py#L46) in apex calls the `to` method blindly on all the input types, which in this case calls `BatchEncoding.to` which doesn't support the full scope of a pytorch `to` because it's designed for moving things between devices and not for casting them.

We're currently guarding it this way:
```python
def patched_to(self, device: Union[str,torch.device]):
    """Send all values to device by calling v.to(device)"""
    if isinstance(device, str) or isinstance(device, torch.device):
        self.data = {k: v.to(device=device) for k, v in self.data.items()}
    return self
BatchEncoding.to = patched_to
```

## Expected behavior

The `BatchEncoding.to` method to not to pass the cast through (as the documentation indicates).

Though it's not clear if this is because the `to` method is underspecified, or because Apex is being quite aggressive (and a little silly) in it's blind casting of everything. Either way, if the `BatchEncoding.to` method isn't supposed to be able to cast things (and it doesn't look like it) then it probably shouldn't pass casts down to the tensors it contains.


Error using pipeline with officiel docker image (transformers 3.0.2)

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.0.2
- Platform: Linux-4.19.76-linuxkit-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.5.1 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

## To reproduce

Steps to reproduce the behavior:

1. Start docker image:
`docker run -it huggingface/transformers-pytorch-cpu /bin/bash`
2. Start python 3:
`python3`
3. Import transfomers and create the pipeline:
```
>>> from transformers import pipeline
>>> nlp = pipeline('question-answering')
```
4. Define question and context:
```
>>> question='What is the GOC at Zinia-1 ?'
>>> context='s. However numerous types of data tend to prove presence of oil i.e. mud gas data show C3 and traces of C4, shows on cuttings and oil was seen by LFA during MDT pump out The middle unit U4 1844-1880.5 MD is oil bearing. No gas is seen from logs at the assumed GOC depth within this unit flat spot at 1849 MD or -1830 TVDSS The lower unit U2 1880.5-1909 mMD is also oil bearing. Pressure data indicate that the 3 units are on-trend although the GWD gas while drilling result suggest barriers between the U2, U4 and U5 in addition to the ultimate seal. The oil sampled in the UM4 has higher viscosity than the oil collected. s. However numerous types of data tend to prove presence of oil i.e. mud gas data show C3 and traces of C4, shows on cuttings and oil was seen by LFA during MDT pump out The middle unit U4 1844-1880.5 MD is oil bearing. No gas is seen from logs at the assumed GOC depth within this unit flat spot at 1849 MD or -1830 TVDSS The lower unit U2 1880.5-1909 mMD is also oil bearing. Pressure data indicate that the 3 units are on-trend although the GWD gas while drilling result suggest barriers between the U2, U4 and U5 in addition to the ultimate seal. The oil sampled in the UM4 has higher viscosity than the oil collected'
```
5. Call the prediction for question answering:
```
>>> nlp(question=question, context=context, verbose=False)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/dist-packages/transformers/pipelines.py", line 1316, in __call__
    for s, e, score in zip(starts, ends, scores)
  File "/usr/local/lib/python3.6/dist-packages/transformers/pipelines.py", line 1316, in <listcomp>
    for s, e, score in zip(starts, ends, scores)
KeyError: 0
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

Get the answer like this (this result comes from 2.11.0):
`{'score': 0.00025710496321816774, 'start': 300, 'end': 322, 'answer': '1849 MD or -1830 TVDSS'}`


[examples/text-classification] update xnli-mt url

This PR updates the XNLI-MT 1.0 file url, the zip at the old url is corrupted.

@sgugger


[Pegasus Doc] minor typo

Minor typo correction
@sshleifer


Latest source has bug in pipelines for short inputs (regardless of padding)

## Environment info
- `transformers` version: 3.0.2
- Platform: Darwin-19.2.0-x86_64-i386-64bit
- Python version: 3.6.8
- PyTorch version (GPU?): 1.4.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
@mfuntowicz @LysandreJik

## Information

Model I am using (Bert, XLNet ...): bert-large-uncased-whole-word-masking-finetuned-squad

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce
Run the latest source (do not get the error from pip)

Steps to reproduce the behavior:

1. install master from source
2. run question answering pipeline
3. Use a document shorter than max_model_length (with or without a padding_strategy)

Traceback:
```
Traceback (most recent call last):
....
    output = pipeline(question=question, context=doc, max_seq_len=512, max_answer_len=50)
  File "/Users/justingrace/Google Drive/healx_root/code/transformers/src/transformers/pipelines.py", line 1673, in __call__
    fw_args = {k: torch.tensor(v, device=self.device) for (k, v) in fw_args.items()}
  File "/Users/justingrace/Google Drive/healx_root/code/transformers/src/transformers/pipelines.py", line 1673, in <dictcomp>
    fw_args = {k: torch.tensor(v, device=self.device) for (k, v) in fw_args.items()}
ValueError: expected sequence of length 512 at dim 1 (got 481)
```

## Expected behavior

the module should not have an error for the non-padded tensors stored in the squad feature set. These are expected to not conform to the max_model_length and need to be handled alternatively -- for instance I am not sure these objects are required at all so they may be removed and masks can be used on the padded tensors.


CamembertForCausalLM

This PR adds `CamembertForCausalLM` by subclassing `RobertaForCausalLM`, so that it can be used with the `EncoderDecoderModel`.

@patrickvonplaten


Add hyperparameter search to Trainer

This PR introduces an access point to easily use optuna or Ray Tune hyperparameter search with `Trainer`.

Here is an example of use on MRPC (requires nlp installed from source on branch cloudpickle):
```
from nlp import load_dataset, load_metric
from transformers import AutoModelForSequenceClassification, AutoTokenizer, DataCollatorWithPadding, Trainer, TrainingArguments

tokenizer = AutoTokenizer.from_pretrained('bert-base-cased')
dataset = load_dataset('glue', 'mrpc')
metric = load_metric('glue', 'mrpc')

def encode(examples):
    outputs = tokenizer(examples['sentence1'], examples['sentence2'], truncation=True)
    return outputs

encoded_dataset = dataset.map(encode, batched=True)
# Won't be necessary when this PR is merged with master since the Trainer will do it automatically
encoded_dataset.set_format(columns=['attention_mask', 'input_ids', 'token_type_ids', 'label'])

def model_init():
    return AutoModelForSequenceClassification.from_pretrained('bert-base-cased', return_dict=True)

def compute_metrics(eval_pred):
    predictions, labels = eval_pred
    predictions = predictions.argmax(axis=-1)
    return metric.compute(predictions=predictions, references=labels)

# Evaluate during training and a bit more often than the default to be able to prune bad trials early.
# Disabling tqdm is a matter of preference.
training_args = TrainingArguments("test", evaluate_during_training=True, eval_steps=500, disable_tqdm=True)
trainer = Trainer(
    args=training_args,
    data_collator=DataCollatorWithPadding(tokenizer),
    train_dataset=encoded_dataset["train"],
    eval_dataset=encoded_dataset["validation"],
    model_init=model_init,
    compute_metrics=compute_metrics,
)

# Defaut objective is the sum of all metrics when metrics are provided, so we have to maximize it.
trainer.hyperparameter_search(direction="maximize")
```
This will use optuna or ray Tune depending on which platform is installed (default to optuna if both are there). You can force a backend by passing `backend="ray"` (or  `backend="optuna"`).

Here is another example on a simple regression problem (will convert to test in a follow-up PR):
```
from transformers import Trainer, TrainingArguments
import numpy as np

class RegressionDataset:
    def __init__(self, a=2, b=3, length=64, seed=42):
        np.random.seed(seed)
        self.length = length
        self.x = np.random.normal(size=(length,)).astype(np.float32)
        self.y = a * self.x + b + np.random.normal(scale=0.1, size=(length,)).astype(np.float32)

    def __len__(self):
        return self.length

    def __getitem__(self, i):
        return {'input_ids': self.x[i], 'label': self.y[i]}

class RegressionModel(torch.nn.Module):
    def __init__(self, a=0, b=0):
        super().__init__()
        self.a = torch.nn.Parameter(torch.tensor(a).float())
        self.b = torch.nn.Parameter(torch.tensor(b).float())

    def forward(self, input_ids=None, labels=None):
        y = input_ids * self.a + self.b
        if labels is None:
            return (y,)
        loss = torch.nn.functional.mse_loss(y, labels)
        return (loss, y)

train_set = RegressionDataset()
eval_set = RegressionDataset()

model = RegressionModel()
training_args = TrainingArguments("test", evaluate_during_training=True, eval_steps=8, disable_tqdm=True)

def compute_metrics(eval_pred):
    predictions, labels = eval_pred
    true = np.abs(predictions - labels) <= 0.25
    return {'accuracy': true.astype(np.float32).mean().item()}

trainer = Trainer(args=training_args, train_dataset=train_set, eval_dataset=eval_set, model_init=lambda: RegressionModel(), compute_metrics=compute_metrics)

best_trial = trainer.hyperparameter_search(direction="maximize")
```

To customize the hyperparameter searched, pass along a `hp_space` function like this (for optuna):
```
def my_hp_space(trial):
    return {
        "learning_rate": trial.suggest_float("learning_rate", 1e-2, 1, log=True),
        "num_train_epochs": trial.suggest_int("num_train_epochs", 1, 5),
        "seed": trial.suggest_int("seed", 1, 40),
        "per_device_train_batch_size": trial.suggest_categorical("per_device_train_batch_size", [4, 8, 16, 32, 64]),
    }
best_trial = trainer.hyperparameter_search(direction="maximize", hp_space=my_hp_space)
```

To customize the objective, pass along a `compute_objective` function.


Tokenizer further tokenizes pretokenized input

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: current master
- Platform: MacOS
- Python version: 3.7

### Who can help
@mfuntowicz

## Information
It seems that passing pretokenized input to the Tokenizer and setting `is_pretokenized=True` doesn't prevent the Tokenizer from further tokenizing the input. This issue already came up in #6046 and the reason for this seems to be #6573 . A workaround is to set `is_pretokenized=False`.
What hasn't been reported yet is that this issue also arises with FastTokenizers where we see the same behavior. However, there is no workaround for FastTokenizers (or at least I haven't found one...). Setting `is_pretokenized=False` will raise a ValueError.

## To reproduce

```python
from transformers.tokenization_auto import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("bert-base-german-cased")
fast_tokenizer = AutoTokenizer.from_pretrained("bert-base-german-cased", use_fast=True)

text = "Schartau sagte dem Tagesspiegel, dass Fischer ein Idiot ist"
pretokenized_text = ['Schar', '##tau', 'sagte', 'dem', 'Tages', '##spiegel', ',', 'dass', 'Fischer', 'ein', 'Id', '##iot', 'ist']

tokenized = tokenizer.encode(text)
# returns list of len 15 -> 13 tokens + 2 special tokens
pretokenized_tok = tokenizer.encode(pretokenized_text, is_pretokenized=True)
# returns list of len 23 -> too large
pretokenized_tok_2 = tokenizer.encode(pretokenized_text, is_pretokenized=False)
# returns list of len 15 -> 13 tokens + 2 special tokens

fast_tokenized = fast_tokenizer.encode(text)
# returns list of len 15 -> 13 tokens + 2 special tokens
fast_pretokenized_tok = fast_tokenizer.encode(pretokenized_text, is_pretokenized=True)
# returns list of len 23 -> too large
# fast_pretokenizer_tok2 = fast_tokenizer.encode(pretokenized_text, is_pretokenized=False)
# would raise: 'ValueError: TextInputSequence must be str'

tokenized_decoded = tokenizer.decode(tokenized)
# returns '[CLS] Schartau sagte dem Tagesspiegel, dass Fischer ein Idiot ist [SEP]'
pretokenized_tok_decoded = tokenizer.decode(pretokenized_tok)
# returns '[CLS] Schar # # tau sagte dem Tages # # spiegel, dass Fischer ein Id # # iot ist [SEP]'
pretokenized_tok_2_decoded = tokenizer.decode(pretokenized_tok_2)
# returns '[CLS] Schartau sagte dem Tagesspiegel, dass Fischer ein Idiot ist [SEP]'

fast_tokenized_decoded = fast_tokenizer.decode(fast_tokenized)
# returns '[CLS] Schartau sagte dem Tagesspiegel, dass Fischer ein Idiot ist [SEP]'
fast_pretokenized_tok_decoded = fast_tokenizer.decode(fast_pretokenized_tok)
# returns '[CLS] Schar # # tau sagte dem Tages # # spiegel, dass Fischer ein Id # # iot ist [SEP]'
```


[docs] Fix number of 'ug' occurrences in tokenizer_summary


[docs] Fix wrong newline in the middle of a paragraph


Dataset and DataCollator for BERT Next Sentence Prediction (NSP) task.

I am working with some pre-training tasks recently, and found that transformers have not yet supported NSP tasks.
So I write the code of **Dataset and DataCollator for BERT NSP task**, now I try to contribute.

This is my first contribution to an open source project. If there is something wrong, please point it out, thank you.


token-classification: update url of GermEval 2014 dataset

Hi,

the URL to the GermEval 2014 dataset was recently changed as the dataset is stored on a Google Drive now.

This PR updates the URL in main documentation and in the example shell scripts.


Can not use the convert_graph_to_onnx.py to convert the pytorch model to onnx model

When I convert the pytorch model to onnx as follow:

```
import os
import torch
from pytorch_pretrained_bert import BertModel

model = BertModel.from_pretrained('bert-base-uncased')
base_path = os.path.dirname(__file__)
pt_path = os.path.join(base_path, 'bert.pt')
torch.save(model, pt_path)
```

Then I use the convert_graph_to_onnx.py to convert the pytorch model to onnx model, as follow:
`python src/transformers/convert_graph_to_onnx.py --framework pt --model bert.pt bert.onnx`

**Error:**
`Error while converting the model: 'utf-8' codec can't decode byte 0x80 in position 64: invalid start byte`

I don't know what caused this

- `transformers` version:3.0.1
- Platform: pycharm
- Python version: 3.7.2
- PyTorch version (GPU?): 1.6.0
- Tensorflow version (GPU?): 2.0.0
- Using GPU in script?: NO
- Using distributed or parallel set-up in script?: NO

Thank you for the help :)


[Model card] Bert2GPT2 EncoderDecoder model

model card

Related to pretrained model cards


KeyError with DPR reader in Question Answering pipeline

## Environment info
- `transformers` version: 3.0.2 (I have also tried with current `master`)
- Platform: Linux-4.19.112+-x86_64-with-Ubuntu-18.04-bionic (with Google Colab)
- Python version: 3.6.9
- PyTorch version (GPU?): 1.6.0+cu101 (False)
- Tensorflow version (GPU?): 2.3.0 (False)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
@LysandreJik

## Information

Model I am using: DPR reader (i.e. https://huggingface.co/facebook/dpr-reader-single-nq-base) to perform a Question Answering task.
## To reproduce

Steps to reproduce the behavior:

```from transformers import pipeline
model_name="facebook/dpr-reader-single-nq-base"
tokenizer_name="facebook/dpr-reader-single-nq-base"
my_pipeline = pipeline("question-answering", model=model_name, tokenizer=tokenizer_name)
```

Error message:
``` KeyError                                  Traceback (most recent call last)
<ipython-input-2-b3a2784cffcf> in <module>()
      4 model_name="facebook/dpr-reader-single-nq-base"
      5 tokenizer_name="facebook/dpr-reader-single-nq-base"
----> 6 my_pipeline = pipeline("question-answering", model=model_name, tokenizer=tokenizer_name)

2 frames
/usr/local/lib/python3.6/dist-packages/transformers/configuration_auto.py in from_pretrained(cls, pretrained_model_name_or_path, **kwargs)
    208
    209         if "model_type" in config_dict:
--> 210             config_class = CONFIG_MAPPING[config_dict["model_type"]]
    211             return config_class.from_dict(config_dict, **kwargs)
    212         else:

KeyError: 'dpr'
```


XLNet Bug when training with apex 16-bit precision

XLNet training fail, while using 16-bit precision, because of tensor creation with explicit usage dtype=torch.float mode in relative_positional_encodings function.


Can not convert the pytorch pretrained bert model to onnx model

When I convert the pytorch pretrained bert model to onnx model  as follows:

```
import os
import torch
from pytorch_pretrained_bert import BertTokenizer, BertModel

model = BertModel.from_pretrained('bert-base-uncased')

input_1 = torch.LongTensor(1, 14)
input_2 = torch.LongTensor(1, 14)
input_3 = torch.LongTensor(1, 14)
base_path = os.path.dirname(__file__)
onnx_path = os.path.join(base_path, 'bert.onnx')
torch.onnx.export(model, (input_1, input_2, input_3), onnx_path)
```
**raise error**:
`IndexError: index out of range in self`

- `transformers` version: 3.0.2
- Platform: pycharm
- Python version: 3.8
- PyTorch version (GPU?): 1.6.0 no GPU
- onnx version: 1.7.0
- pytorch-pretrained-bert version: 0.6.2

Thank you for the help :)


New Feature: Best-First Beam Search

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

Best-First Beam Search is a faster beam search algorithm for decoding.
https://arxiv.org/abs/2007.03909 (By Clara Meister, Tim Vieira, Ryan Cotterell)

## Motivation

This technique can be essential for production-purpose inference and of great interest to our users. It would be a good idea to integrate Best-First Beam Search to Hugging Face transformers (for GPT, BART, T5, etc.).


T5 Gradient Checkpointing

Good Second Issue

Issues that are more difficult to do than "Good First" issues - give it a try if you want!

# 🚀 Feature request

Currently, only Bert supports gradient checkpointing which allow the model to be fine-tuned on GPUs with small memory.
It will be great to make T5 also support gradient checkpointing.

Code:
https://github.com/huggingface/transformers/blob/0735def8e1200ed45a2c33a075bc1595b12ef56a/src/transformers/modeling_bert.py#L461

## Motivation

T5 has very big models with 3B and 11B parameters which make it impossible to be fine-tuned on most GPUs. Gradient checkpointing will allow these huge models to be fine-tuned on GPUs. This will lead to much better results on downstream tasks using on house GPUs without the need to fine-tuned it on TPUs.

## Your contribution

If I am not mistaken all what need to be change is the following block:
https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_t5.py#L752

```
for i, (layer_module, past_key_value_state) in enumerate(zip(self.block, past_key_value_states)):
            if output_hidden_states:
                all_hidden_states = all_hidden_states + (hidden_states,)

            if getattr(self.config, "gradient_checkpointing", False):

                def create_custom_forward(module):
                    def custom_forward(*inputs):
                        return module(*inputs, output_attentions)

                    return custom_forward

                 layer_outputs = torch.utils.checkpoint.checkpoint(
                    create_custom_forward(layer_module),
                    hidden_states,
                    extended_attention_mask,
                    position_bias,
                    encoder_hidden_states,
                    encoder_extended_attention_mask,
                    encoder_decoder_position_bias,
                    head_mask[i],
                    past_key_value_state,
                    use_cache,
                    output_attentions,
                )

            else:
                layer_outputs = layer_module(
                    hidden_states,
                    attention_mask=extended_attention_mask,
                    position_bias=position_bias,
                    encoder_hidden_states=encoder_hidden_states,
                    encoder_attention_mask=encoder_extended_attention_mask,
                    encoder_decoder_position_bias=encoder_decoder_position_bias,
                    head_mask=head_mask[i],
                    past_key_value_state=past_key_value_state,
                    use_cache=use_cache,
                    output_attentions=output_attentions,
                )
                # layer_outputs is a tuple with:
                # hidden-states, key-value-states, (self-attention weights), (self-attention position bias), (cross-attention weights), (cross-attention position bias)
            hidden_states, present_key_value_state = layer_outputs[:2]

            if i == 0:
                # We share the position biases between the layers - the first layer store them
                # layer_outputs = hidden-states, key-value-states (self-attention weights), (self-attention position bias), (cross-attention weights), (cross-attention position bias)
                position_bias = layer_outputs[3 if output_attentions else 2]
                if self.is_decoder and encoder_hidden_states is not None:
                    encoder_decoder_position_bias = layer_outputs[5 if output_attentions else 3]
            # append next layer key value states
            if use_cache:
                present_key_value_states = present_key_value_states + (present_key_value_state,)

            if output_attentions:
                all_attentions = all_attentions + (layer_outputs[2],)  # We keep only self-attention weights for now
```

@patrickvonplaten  thanks in advance for looking into it.