TFBertModel not working at all with any type of model

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- Platform: macOS
- Python version: 3.7
- PyTorch version (GPU?): NO
- Tensorflow version (GPU?): 2.3.1
- Using GPU in script?: NO
- Using distributed or parallel set-up in script?: NO

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

## Information

Model I am using (Bert, XLNet ...): Bert "dbmdz/bert-base-italian-xxl-cased"

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [X] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [X] my own task or dataset: (give details below)

<---------------------------------------------->

Hi @LysandreJik , @stefan-it , @sgugger , I'm trying to use `dbmdz/bert-base-italian-xxl-cased`for creating a keras model for a classification task.
I've followed the documentation but I continue to receive the following error:

```
tensorflow.python.framework.errors_impl.InvalidArgumentError:  indices[5,0] = 102 is not in [0, 2)
	 [[node functional_1/bert/embeddings/token_type_embeddings/embedding_lookup (defined at /anaconda3/envs/profanity-detector/lib/python3.7/site-packages/transformers/modeling_tf_bert.py:186) ]] [Op:__inference_train_function_29179]
```

This is the model:

```
from transformers import TFBertModel, BertTokenizer

bert_model = TFBertModel.from_pretrained("dbmdz/bert-base-italian-xxl-cased")
tokenizer = BertTokenizer.from_pretrained("dbmdz/bert-base-italian-xxl-cased")

input_ids = tf.keras.layers.Input(shape=(constants.MAX_SEQ_LENGTH,), dtype=tf.int32)
token_type_ids = tf.keras.layers.Input(shape=(constants.MAX_SEQ_LENGTH,), dtype=tf.int32)
attention_mask = tf.keras.layers.Input(shape=(constants.MAX_SEQ_LENGTH,), dtype=tf.int32)

seq_output, _ = bert_model({
        "input_ids": input_ids,
        "token_type_ids": token_type_ids,
        "attention_mask": attention_mask
})

pooling = tf.keras.layers.GlobalAveragePooling1D()(seq_output)
dropout = tf.keras.layers.Dropout(0.2)(pooling)
output = tf.keras.layers.Dense(constants.CLASSES, activation="softmax")(dropout)

model = tf.keras.Model(
        inputs=[input_ids, token_type_ids, attention_mask],
        outputs=[output]
)

model.compile(optimizer=tf.optimizers.Adam(lr=0.00001), loss='sparse_categorical_crossentropy', metrics=['accuracy'])
```

My dataset is tokenized by this method:
```
    def map_to_dict(self, input_ids, attention_masks, token_type_ids, labels):
        return {
            "input_ids": input_ids,
            "token_type_ids": token_type_ids,
            "attention_mask": attention_masks,
        }, labels

    def tokenize_sequences(self, tokenizer, max_sequence_length, data, labels):
        try:
            token_ids = []
            token_type_ids = []
            attention_mask = []

            for sentence in data:
                bert_input = tokenizer.encode_plus(
                    sentence,
                    add_special_tokens=True,  # add [CLS], [SEP]
                    max_length=max_sequence_length,  # max length of the text that can go to BERT
                    truncation=True,
                    pad_to_max_length=True,  # add [PAD] tokens
                    return_attention_mask=True  # add attention mask to not focus on pad tokens
                )

                token_ids.append(bert_input["input_ids"])
                token_type_ids.append(bert_input["token_type_ids"])
                attention_mask.append(bert_input["attention_mask"])

            return tf.data.Dataset.from_tensor_slices((token_ids, token_type_ids, attention_mask, labels)).map(self.map_to_dict)
        except Exception as e:
            stacktrace = traceback.format_exc()

            logger.error("{}".format(stacktrace))

            raise e

ds_train_encoded = tokenize_sequences(tokenizer, 512, X_train, y_train).shuffle(10000).batch(6)
```
X_train examples:
```
["Questo video è davvero bellissimo", "La qualità del video non è proprio il massimo"......]
```

y_train examples:
```
[[1], [0]...]
```

I continue to receive the error described before.

```
tensorflow.python.framework.errors_impl.InvalidArgumentError:  indices[5,0] = 102 is not in [0, 2)
	 [[node functional_1/bert/embeddings/token_type_embeddings/embedding_lookup (defined at /anaconda3/envs/profanity-detector/lib/python3.7/site-packages/transformers/modeling_tf_bert.py:186) ]] [Op:__inference_train_function_29179]
```

If I try to use TFBertForSequenceClassification everything works fine (for this reason I'm excluding tokenization problems).

Can you please provide a solution or a well documented guide for using TFBertModel class with Keras model (I cannot find it)?

Thank you


Add a new model ConvBert

# 🌟 New model addition

Pre-trained language models like BERT and its variants have recently achieved impressive performance in various natural language understanding tasks. However, BERT heavily relies on the global self-attention block and thus suffers large memory footprint and computation cost. Although all its attention heads query on the whole input sequence for generating the attention map from a global perspective, we observe some heads only need to learn local dependencies, which means the existence of computation redundancy. We therefore propose a novel span-based dynamic convolution to replace these self-attention heads to directly model local dependencies. The novel convolution heads, together with the rest self-attention heads, form a new mixed attention block that is more efficient at both global and local context learning. We equip BERT with this mixed attention design and build a ConvBERT model. Experiments have shown that ConvBERT significantly outperforms BERT and its variants in various downstream tasks, with lower training cost and fewer model parameters. Remarkably, ConvBERTbase model achieves 86.4 GLUE score, 0.7 higher than ELECTRAbase, while using less than 1/4 training cost.

<!-- Important information -->

## Open source status

* [x] the model implementation is available: (https://github.com/yitu-opensource/ConvBert)
* [x] the model weights are available: (https://drive.google.com/drive/folders/1pSsPcQrGXyt1FB45clALUQf-WTNAbUQa)
* [x] who are the authors: (@zihangJiang @zhoudaquan)


Pretrain PEGASUS from scratch

I want to pre-train PEGASUS model from scratch on a language other than English. Is there any way to do this using huggingace API's? The source code released by the authors is complicated in use to pre-train. Also little documentation available to do this.


[doc] typo fix

s/e.g./i.e./ as what follows is not an example, but a "that is" statement + fix language

@sgugger


mBart prefix and suffix for language id

## Environment info
- `transformers` version: 3.5.1

### Who can help
 mBART: @patrickvonplaten
 documentation: @sgugger

## Information

It seems that there is error or inconsistency between [mbart tokenizer](https://github.com/huggingface/transformers/blob/55e8d0cea25be18b044523b30f4bef58fec63289/src/transformers/tokenization_mbart.py) and comments/docs inline the code.

comments on code explain
```
def build_inputs_with_special_tokens(
        self, token_ids_0: List[int], token_ids_1: Optional[List[int]] = None
    ) -> List[int]:
        """
        Build model inputs from a sequence or a pair of sequence for sequence classification tasks by concatenating and
        adding special tokens. An MBART sequence has the following format, where ``X`` represents the sequence:
        - ``input_ids`` (for encoder) ``X [eos, src_lang_code]``
        - ``decoder_input_ids``: (for decoder) ``[tgt_lang_code] X [eos]``
```

but `set_tgt_lang_special_tokens`  considers that `decoder_input_ids` is **X [eos][tgt_lang_code]** instead of **[tgt_lang_code] X [eos]**

```
def set_tgt_lang_special_tokens(self, lang: str) -> None:
        """Reset the special tokens to the target language setting. Prefix [tgt_lang_code], suffix =[eos]."""
        self.cur_lang_code = self.lang_code_to_id[lang]
        self.prefix_tokens = []
        self.suffix_tokens = [self.eos_token_id, self.cur_lang_code]
```

Shouldn't it be:
```
def set_tgt_lang_special_tokens(self, lang: str) -> None:
        """Reset the special tokens to the target language setting. Prefix [tgt_lang_code], suffix =[eos]."""
        self.cur_lang_code = self.lang_code_to_id[lang]
        self.prefix_tokens = [self.cur_lang_code]
        self.suffix_tokens = [self.eos_token_id]
```


Bertabs example: index_select(): Expected dtype int64 for index

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.3.1
- Platform: Ubuntu 18.04
- Python version: 3.8.0
- PyTorch version (GPU?): torch==1.7.0
- Tensorflow version (GPU?):
- Using GPU in script?:yes
- Using distributed or parallel set-up in script?: no

### Who can help
@patil-suraj

## Information

Following the example in the seq2seq/bertabs readme.md.
I am getting this error:
```
File "/code/tools/transformers/examples/seq2seq/bertabs/modeling_bertabs.py", line 919, in _fast_translate_batch
    alive_seq = torch.cat([alive_seq.index_select(0, select_indices), topk_ids.view(-1, 1)], -1)
RuntimeError: index_select(): Expected dtype int64 for index
```
In a debugger, I see that the 'select_indices' parameter is a tensor of floats.

I don't understand the beam mechanism, so I don't know where to start troubleshooting this.

Any help would be great!

-Tim


converting tensorflow checkpoint to pytorch

@LysandreJik , I am trying to convert REALM checkpoints in google-research/language/ to a pytorch checkpoint.

One of the arguments to convert_bert_original_tf_checkpoint_to_pytorch.py is bert config.json file. I don't see this file in the model directory. Just wanted to confirm that I can use any bert_config.json ?
From the code, I see :
" help="The config json file corresponding to the pre-trained BERT model. \n"
        "This specifies the model architecture.",


[models website: files section] various issues/suggestions for a better UI

Feature request

Request for a new feature

Models website - files section: suggestions/issues:

1. [ ] `git lfs install` not working

I tried to download a model as instructed here:
https://huggingface.co/facebook/bart-large/tree/main#how-to-use
```
$ git lfs install
git: 'lfs' is not a git command. See 'git --help'.

The most similar commands are
        last
        ls
```

I had to do:
```
apt install git-lfs
```
and then it worked:

```
$ git lfs install
Updated git hooks.
Git LFS initialized.
```

Perhaps it needs a link to:
https://docs.github.com/en/free-pro-team@latest/github/managing-large-files/installing-git-large-file-storage
if apt doesn't have it or a user is on a different setup.

2. [ ] would it be possible to rename "Use in transformers" to "Use/Download" - it wasn't obvious to look there for download instructions. I was trying to click on the files.
3. [x] the fact that the files in the [listing](https://huggingface.co/facebook/bart-large/tree/main#) are clickable and some of them are containing the final data and others contain just a reference and no way to get to the final data is kind of inconsistent and confusing. at the very least the ones with a reference instead of data should have a link to how to get to the data.
4. [ ] ` .gitattributes` is irrelevant to the user in that listing, it's a circumstantial file and not part of the model files, IMHO ;)
5. [ ] <title> is missing (uses generic title) - not optimal for bookmarking/email forwarding

@julien-c


Switch `return_dict` to `True` by default.

model card

Related to pretrained model cards

# What does this PR do?

As part of the main changes we want to bring to Transformers v4, this PR switches the `return_dict` argument default from `False` to `True`. Most of the examples and documentation were already using this value (the PR removes `return_dict=True` in all those instances since this is now the default).

**New model outputs**

The new model outputs are dictionaries (instead of tuples) with a bit of added functionality: you can access elements by their keys, as attributes or even by index (to keep most of the backward compatibility). Here is an example:

```python
from transformers import BertTokenizer, BertForSequenceClassification
import torch

tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertForSequenceClassification.from_pretrained('bert-base-uncased')

inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
labels = torch.tensor([1]).unsqueeze(0)  # Batch size 1
outputs = model(**inputs, labels=labels)
```

The `outputs` object contains loss and logits and you can access them as keys:
```python
loss = outputs["loss"]
logits = outputs["logits"]
```
or as attributes.
```python
loss = outputs.loss
logits = outputs.logits
```
In an IDE you can thus use autocomplete to help.

You can still index with integers or a slice:
```python
loss = outputs[0]
logits = outputs[1]
loss, logits = outputs[:2]
```
but you can't unpack the tuple directly as was done before:
```python
loss, logits = outputs
```
will return "loss" and "logits" in `loss` and `logits` (like dictionaries).

**Known caveats**

Torchscript is incompatible with dict outputs (until 1.7) so doesn't work with those new outputs. The flag `return_dict` will default to `False` if you use `torchscript=True` when building your model.

TF Saved models deal with dictionaries but not their subclasses, so if you save and load your model using `tf.keras.models.load_model`, it will output regular dictionaries. This means that you won't be able to index in the outputs with integers or attributes, only their string keys.

```python
tf.saved_model.save(model, tmpdirname)
model = tf.keras.models.load_model(tmpdirname)
outputs = model(inputs)
loss = outputs["loss"]
logits = outputs["logits"]
```

Apex used in optimization mode "O2" will also lose the output types of the model and return regular dictionaries. This means that you won't be able to index in the outputs with integers or attributes, only their string keys.

**Breaking change**

All code of the form
```python
loss, output = model(**inputs)
```
needs to be switched to
```python
loss, output = model(**inputs).to_tuple()
```
or use the key/attributes of the model output returned.

Alternatively, you can pass `return_dict=False` when creating your model to get regular tuples as outputs.


Adding PrefixConstrainedLogitsProcessor

# What does this PR do?
This pull request adds a new decoding strategy that constrains the next token to generate based on a callable function. It is a new PR that fixes https://github.com/huggingface/transformers/pull/7784 since the generate function went trough refactoring. @patrickvonplaten


[T5] Fix load weights function

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #7791

The problem in 7791 was that the code that was used to convert mtf t5 to hf t5 was outdated and had a couple of bugs:

1) weight embedding were not tied -> random weight matrices were used instead
2) the tokenizer didn't add EOS which meant that the input_ids were wrong.

@agemagician I will merge this into master now so that you don't have to work on the hacky branch I did a while back. Your models should be convertible with the new code added here and then everything should work as expected.

The following code now produces the correct results:

```python
#!/usr/bin/env python3
from transformers import T5Tokenizer  # noqa: E402
from transformers.convert_t5_original_tf_checkpoint_to_pytorch import (  # noqa: E402
    convert_tf_checkpoint_to_pytorch,
)
from transformers.modeling_t5 import T5Config, T5ForConditionalGeneration  # noqa: E402
import torch

path_to_tf_checkpoint = "t5_mesh_checkpoints"

config = T5Config.from_pretrained("t5-base")
config.d_ff = 2048
config.d_kv = 64
config.d_model = 512
config.num_decoder_layers = 6
config.num_layers = 6
config.num_heads = 8
config.vocab_size = 32128
config.tie_word_embeddings = True
config.save_pretrained(path_to_tf_checkpoint)

convert_tf_checkpoint_to_pytorch(path_to_tf_checkpoint, path_to_tf_checkpoint + "/config.json", path_to_tf_checkpoint)

input_txt = ["javascript documentation generation: function isStandardBrowserEnv ( ) { if ( typeof navigator !== 'undefined' && ( navigator . product === 'ReactNative' || navigator . product === 'NativeScript' || navigator . product === 'NS' ) ) { return false ; } return ( typeof window !== 'undefined' && typeof document !== 'undefined' ) ; }"]

tok = T5Tokenizer.from_pretrained(path_to_tf_checkpoint)

model = T5ForConditionalGeneration.from_pretrained(path_to_tf_checkpoint, return_dict=True)
model.to("cuda")

input_ids = tok(input_txt, return_tensors="pt").input_ids
outputs = model.generate(input_ids.to("cuda"), num_beams=4)
print(tok.decode(outputs[0]))
```
=> gives "<pad> Returns true if the browser is a native element.</s> "

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSTM: @stas00
 -->

I checked that the new `convert_weights_function` works with previous mtf (t5==0.7.1) models.


Add bart-large-mnli model card

Related to pretrained model cards

# What does this PR do?

Adds a model card for facebook/bart-large-mnli. Since this model is currently the default for the zero-shot pipeline/widget, it adds an introduction to zero-shot text classification with references & example snippets.


Problem while pretraining MLM from scratch using Transformers

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.1.0
- Platform: Linux-3.10.0-1127.19.1.el7.x86_64-x86_64-with-centos-7.8.2003-Core
- Python version: 3.6.9
- PyTorch version (GPU?): 1.6.0 (True)
- Tensorflow version (GPU?): 2.3.0 (True)
- Using GPU in script?: Yes(Two GPUs, Nvidia P100)
- Using distributed or parallel set-up in script?: No

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 Trainer: **@sgugger**

 -->
 Trainer: @sgugger
## Information

Model I am using (RoBERTa):

The problem arises when using:
* [* ] my own modified scripts: (give details below)
'''Python
config=RobertaConfig(vocab_size=30_000, max_position_embeddings=inputlen, num_attention_heads=12,
                     num_hidden_layers=6,hidden_dropout_prob=0.1,attention_probs_dropout_prob=0.1,
                     initializer_range=0.2, intermediate_size=3072, type_vocab_size=1)

training_args=TrainingArguments(output_dir="/LAB_SHARED/Projects/011-Language_Model/Data/LanguageModel/BadgerBERT",overwrite_output_dir=True,
                                num_train_epochs=10, do_train=True, do_eval=True, evaluate_during_training=True,
                                per_gpu_train_batch_size=64, learning_rate=0.0004,
                                gradient_accumulation_steps=32,
                                logging_steps=128,
                                warmup_steps=30000,
                                weight_decay=0.01,
                                eval_steps=128,
                                save_steps=128,
                                save_total_limit=2, prediction_loss_only=True)

trainer=Trainer(model=model,args=training_args,data_collator=data_collator,train_dataset=dataset,eval_dataset=dataset1,prediction_loss_only=True)

'''

The tasks I am working on is:
* [ *] my own task or dataset: (give details below)
Training masked language model

## To reproduce

Steps to reproduce the behavior:

1. Please use the following configuration
2. I used my own training dataset, but I guess there should be no difference. My input text file size is around 40GB

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

Traceback (most recent call last):█████████████████████████████████████████████████████████████████▉| 279691/279692 [5:22:01<00:00, 15.48it/s]
  File "CreateLanguageModel.py", line 84, in <module>
    trainer.train()
  File "**/lib64/python3.6/site-packages/transformers/trainer.py", line 785, in train
    torch.save(self.optimizer.state_dict(), os.path.join(output_dir, "optimizer.pt"))
  File "**/lib64/python3.6/site-packages/torch/serialization.py", line 361, in save
    with _open_file_like(f, 'wb') as opened_file:
  File "**/lib64/python3.6/site-packages/torch/serialization.py", line 229, in _open_file_like
    return _open_file(name_or_buffer, mode)
  File "**/lib64/python3.6/site-packages/torch/serialization.py", line 210, in __init__
    super(_open_file, self).__init__(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: '**/checkpoint-128/optimizer.pt'

## Expected behavior
Instead of continue training, it stops. It seems that the problem is related to the check-points. I don't know why it tries to open (save) check-point-128 while it already saved 512!
Besides that it already saved check-point-160 while I did not set any parameter as 160, so I don't know where the 160 came from.
I expected the code to just report the evaluation loss periodically, save checkpoints and finish the training.

<!-- A clear and concise description of what you would expect to happen. -->


`TypeError: unhashable type: 'list'` when using DataCollatorForWholeWordMask

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.5.0
- Platform: Linux-5.9.1-arch1-1-x86_64-with-arch
- Python version: 3.7.3
- PyTorch version (GPU?): 1.7.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.-->
 @sgugger

## Information

Model I am using (Bert, XLNet ...): Bert

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

This is the code I am running

```python
from transformers import BertTokenizer, BertForMaskedLM, AdamW, BertConfig, get_linear_schedule_with_warmup, pipeline, DataCollatorForWholeWordMask, DataCollatorForLanguageModeling

tokenizer = BertTokenizer.from_pretrained('bert-base-multilingual-cased', do_lower_case=True)

sent = "The capital of France is Paris."

data_collator = DataCollatorForWholeWordMask(tokenizer=tokenizer, mlm=True, mlm_probability=0.15)

encoded = tokenizer(
  sent,
  truncation = True,
  add_special_tokens = True,
  max_length = 64,
  return_tensors = 'pt',
  return_special_tokens_mask = True,
)

masked = data_collator([encoded])

print(masked)
```

It gives me the following error

```python
Traceback (most recent call last):
  File "sadness.py", line 19, in <module>
    masked = data_collator([encoded])
  File "/home/csikfeng/.local/lib/python3.7/site-packages/transformers/data/data_collator.py", line 328, in __call__
    token = self.tokenizer._convert_id_to_token(id)
  File "/home/csikfeng/.local/lib/python3.7/site-packages/transformers/tokenization_bert.py", line 241, in _convert_id_to_token
    return self.ids_to_tokens.get(index, self.unk_token)
TypeError: unhashable type: 'list'
```

But if instead I use `data_collator = DataCollatorForLanguageModeling(tokenizer=tokenizer, mlm=True, mlm_probability=0.15)
` like this
```python
from transformers import BertTokenizer, BertForMaskedLM, AdamW, BertConfig, get_linear_schedule_with_warmup, pipeline, DataCollatorForWholeWordMask, DataCollatorForLanguageModeling

tokenizer = BertTokenizer.from_pretrained('bert-base-multilingual-cased', do_lower_case=True)

sent = "The capital of France is Paris."

data_collator = DataCollatorForLanguageModeling(tokenizer=tokenizer, mlm=True, mlm_probability=0.15)

encoded = tokenizer(
  sent,
  truncation = True,
  add_special_tokens = True,
  max_length = 64,
  return_tensors = 'pt',
  return_special_tokens_mask = True,
)

masked = data_collator([encoded])

print(masked)
```
I do not get any errors

```
{'input_ids': tensor([[[  101, 10105, 12185, 10108, 63184, 10112, 10124, 24289, 10107,   119,
            102]]]), 'token_type_ids': tensor([[[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]]]), 'attention_mask': tensor([[1]]), 'labels': tensor([[[-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100]]])}
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

It should just perform whole word masking and not have errors.


LayoutLM Token Classification not learning

## Environment info
- `transformers` version: 3.4.0
- Platform: in docker based on image: nvidia/cuda:10.1-cudnn7-devel-ubuntu18.04
- Python version: 3.7.9
- PyTorch version (GPU?): 1.5.1+cu92 (True)
- Tensorflow version (GPU?): 2.2.0-rc0 (True)
- Using GPU in script?: True
- Using distributed or parallel set-up in script?: <fill in>

## Information

Model I am using (Bert, XLNet ...): LayoutLMForTokenClassification

The problem arises when using: my own scripts

The tasks I am working on is: my own task
NER task. I've reproduced the implementation of Dataset, compute metrics (and other helper functions) as in the original repo [microsoft/layoutlm repo](https://github.com/microsoft/unilm/blob/master/layoutlm/examples/seq_labeling/run_seq_labeling.py)

When initially trying with the original repo and training script the model managed to learn and provided reasonable results after very few epochs. After implementing with Huggingface the model doesn't learn at all even after a much higher number of epochs.

## To reproduce
Model loading and trainer configuration:
```
config = LayoutLMConfig.from_pretrained(
    <path_layoutlm_base_uncased>,
    num_labels=<num_labels>,
    cache_dir=None
)
model = LayoutLMForTokenClassification.from_pretrained(
    <path_layoutlm_base_uncased>
    from_tf=bool(".ckpt" in <path_layoutlm_base_uncased>),
    config=config,
    cache_dir=None,
)

device = torch.device("cuda")
model.train().to(device)

TrainingArguments(
    output_dir=<pytorch_model_dir>,  # output directory
    do_train=True,
    do_eval=True,
    do_predict=False,
    evaluation_strategy=EvaluationStrategy.EPOCH,
    num_train_epochs=<epochs>,  # total # of training epochs
    per_device_train_batch_size=<batch_size>,  # batch size per device during training
    per_device_eval_batch_size=<batch_size>,  # batch size for evaluation
    weight_decay=<weight_decay>,  # strength of weight decay
    learning_rate=<learning_rate>,
    adam_epsilon=<adam_epsilon>,
    logging_dir=<profile_logs>,  # Tensorboard log directory
    logging_steps=0,  # it logs when running evaluation so no need to log on step interval
    save_steps=0,
    seed=seed,
    overwrite_output_dir=True,
    disable_tqdm=False,
    load_best_model_at_end=True,
    save_total_limit=10,
    fp16=True,
)

trainer = MetaMazeTrainer(
    model=model,  # the instantiated 🤗 Transformers model to be trained
    args=training_args,  # training arguments, defined above
    train_dataset=train_dataset,  # training dataset
    eval_dataset=test_dataset,  # evaluation dataset
    compute_metrics=compute_metrics,
)

```

## Expected behavior
Similar results to the original repo as the given the same parameters to the trainer and the Dataset being the same after processing the data.

Is this due to the ongoing integration of this model? Is the setup wrong?


Reformer model crashes during casual LM evaluation

## Environment info
- `transformers` version: 3.4.0
- Platform: Linux-5.4.0-47-generic-x86_64-with-debian-bullseye-sid
- Python version: 3.6.12
- PyTorch version (GPU?): 1.7.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: -

### Who can help

I tried to dig into the code but could not find out why this is happening, so I am tagging @sgugger since this might be a `Trainer` related issue as well as @patrickvonplaten as I am using `ReformerWithLMHead`.

## Information

I am using `ReformerWithLMHead` with a custom dataset and already set up the masked language modeling task so I moved on to casual LM but something odd happened. My setup is based on the official notebook from @patrickvonplaten and it works fine for masked LM.

```python
data_collator = DataCollatorForLanguageModeling(
    tokenizer=tokenizer, mlm=False
)

def compute_metrics(pred):
    """
        pred.label_ids = (prediction_set_size, sequence_length)
        pred.predictions = (prediction_set_size, sequence_length, vocab_size)
            prob. dist. along vocab size
        Since we do masked language modelling, most of the sequence is MASKED with -100
        and only the non masked should be checked. :)
    """
    non_masked_indices = (pred.label_ids != -100)
    predictions = np.argmax(pred.predictions, axis=-1)
    labels = pred.label_ids[non_masked_indices]
    predictions = predictions[non_masked_indices]
    return {"accuracy": np.mean(np.asarray(predictions == labels), dtype=np.float)}

trainer = Trainer(
    model=model,
    args=training_args,
    compute_metrics=compute_metrics,
    data_collator=data_collator,
    train_dataset=dataset,
    eval_dataset=eval_dataset,
    prediction_loss_only=False)

trainer.train()
```

I set up the collator for the non-mlm task but left the custom metric (also based on the official notebook) to calculate accuracy since it should be the same as before (IMO). The tricky part is if I explicitly set `prediction_loss_only=False` I get an error indicating that the `logits` could not have been nested_detached:

```bash
  File "src/lm/reformer_casual_lm.py", line 146, in <module>
    trainer.train()
  File "/home/qbeer/miniconda3/envs/nlp/lib/python3.6/site-packages/transformers/trainer.py", line 786, in train
    self._maybe_log_save_evalute(tr_loss, model, trial, epoch)
  File "/home/qbeer/miniconda3/envs/nlp/lib/python3.6/site-packages/transformers/trainer.py", line 843, in _maybe_log_save_evalute
    metrics = self.evaluate()
  File "/home/qbeer/miniconda3/envs/nlp/lib/python3.6/site-packages/transformers/trainer.py", line 1251, in evaluate
    output = self.prediction_loop(eval_dataloader, description="Evaluation")
  File "/home/qbeer/miniconda3/envs/nlp/lib/python3.6/site-packages/transformers/trainer.py", line 1348, in prediction_loop
    loss, logits, labels = self.prediction_step(model, inputs, prediction_loss_only)
  File "/home/qbeer/miniconda3/envs/nlp/lib/python3.6/site-packages/transformers/trainer.py", line 1452, in prediction_step
    logits = nested_detach(logits)
  File "/home/qbeer/miniconda3/envs/nlp/lib/python3.6/site-packages/transformers/trainer_pt_utils.py", line 66, in nested_detach
    return type(tensors)(nested_detach(t) for t in tensors)
  File "/home/qbeer/miniconda3/envs/nlp/lib/python3.6/site-packages/transformers/trainer_pt_utils.py", line 66, in <genexpr>
    return type(tensors)(nested_detach(t) for t in tensors)
  File "/home/qbeer/miniconda3/envs/nlp/lib/python3.6/site-packages/transformers/trainer_pt_utils.py", line 66, in nested_detach
    return type(tensors)(nested_detach(t) for t in tensors)
  File "/home/qbeer/miniconda3/envs/nlp/lib/python3.6/site-packages/transformers/trainer_pt_utils.py", line 66, in <genexpr>
    return type(tensors)(nested_detach(t) for t in tensors)
  File "/home/qbeer/miniconda3/envs/nlp/lib/python3.6/site-packages/transformers/trainer_pt_utils.py", line 66, in nested_detach
    return type(tensors)(nested_detach(t) for t in tensors)
  File "/home/qbeer/miniconda3/envs/nlp/lib/python3.6/site-packages/transformers/trainer_pt_utils.py", line 66, in <genexpr>
    return type(tensors)(nested_detach(t) for t in tensors)
  File "/home/qbeer/miniconda3/envs/nlp/lib/python3.6/site-packages/transformers/trainer_pt_utils.py", line 67, in nested_detach
    return tensors.detach()
AttributeError: 'NoneType' object has no attribute 'detach'
```

If I just delete the `prediction_loss_only=False` line the training runs but my custom metric is not evaluated since in the training class, the gathered labels and predictions are only not `None` when this value is set to `False`:

```python
eval_loss = eval_losses_gatherer.finalize()
preds = preds_gatherer.finalize() if not prediction_loss_only else None
label_ids = labels_gatherer.finalize() if not prediction_loss_only else None

if self.compute_metrics is not None and preds is not None and label_ids is not None:
    metrics = self.compute_metrics(EvalPrediction(predictions=preds, label_ids=label_ids))
```
## Expected behavior

I expect that my custom metric is evaluated and the training not crashing randomly.

Thanks in advance.


Update deepset/roberta-base-squad2 model card

Related to pretrained model cards

Update model card since our v1 and v2 of the model are in this repo.

Note that accessing models doesn't seem to be working when referencing tag name #8521


Tagged versions of model in new model hub don't work

Our model, deepset/roberta-base-squad2 was originally uploaded under the old style model hub.

I have committed a new version of the deepset/roberta-base-squad2 model onto the model hub using the new git based system introduced in transformers 3.5.0. I have 2 tags (v1.0 and v2.0) that I have also pushed to the repo. The tags show up in the model hub drop down but when I click on either of the tags, it says "Not Found: Error: Invalid rev id".

It seems I cannot load the models when I specify `revision=v1.0` or `revision=v2.0`. If I don't specify a revision, it seems to load a model though I'm not sure which. This is the code I used:
```
tokenizer = AutoTokenizer.from_pretrained(
  "deepset/roberta-base-squad2",
  revision="v2.0" # tag name, or branch name, or commit hash
)
```

What steps can I take so that I can access both versions through the model hub website, and by specifying name and revision?

Thanks,
Branden

 @julien-c


Model sharing doc: more tweaks

cc @Pierrci


MLflowCallback to log run_name argument

# 🚀 Feature request

When using the MLflowCallback (set as default for Trainer), I would like to log the `run_name` argument passed to TrainingArguments as the Run Name on the MLflow dashboard. Currently runs are being logged as nameless.
E.g. see below.
![Screenshot 2020-11-13 at 10 29 32](https://user-images.githubusercontent.com/2844109/99062592-21abd500-259b-11eb-9caf-c43d2bcb31dc.png)

## Motivation

Trainer makes training 🤗  models so easy and MLflow is great for organising experiments/caching artifacts. I would like to make it easier to organise experimental runs and make research easier, particularly for larger teams. This feature would be a very simple patch on the original PR #8016

example usage
```
training_args = TrainingArguments(
    label_names=['labels_t1', 'labels_t2'],
    output_dir='./runs,          # output directory
    run_name='multitask_clf_<run_name>',
)
trainer = Trainer(
    model=model,        # the instantiated 🤗 Transformers model to be trained
    args=training_args,        # training arguments, defined above
    train_dataset=train_dataset,        # training dataset
)
```

## Your contribution

I can submit a PR


[T5] Bug correction & Refactor

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

**!!!BUG DETECTION!!!**

While integrating T5v1.1/mT5, a bug in T5 was detected. T5 actually never uses a `relative_position_bias` in the `EncDecSelfAttention` layer. Previously we used a bi-directional `relative_position_bias` in the `EncDecSelfAttention`, which is wrong IMO (see https://github.com/huggingface/transformers/issues/6285#issuecomment-702371111 for reference). An integration test against original T5 model was added to make sure removing `relative_posiiton_bias` is the correct behavior (in case @craffel reads this - maybe you could confirm :-)).

Luckily, the bug did not significantly influence the results as can be seen by the very minor changes in the slow tests. This  is also why it wasn't noticed earlier. => Oo all pre-trained & fine-tuned T5Models still work!

In addition this PR:
- Refactor: clean the code, and remove unnecessarily complicated code
- Remove `n_positions` / `max_position_embeddings` from the config, since T5 is not limited by a  fixed learned position embedding matrix, see: https://github.com/huggingface/transformers/issues/8047

Fixes #8047

Also cc @agemagician for information (I highly doubt though that this will fix the problem we have in your case)

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSTM: @stas00
 -->


XLM-RoBERTa tokenizer changes characters during tokenization

I'm using `transformers` 3.5.0.

Whenever the XLM-RoBERTa tokenizer encounters some characters such as '²' (super-script 2, `ord('²')` is 178), it converts them to other characters (in this example, to the plain '2', with `ord('2')` being 50).

That is, with `tokenizer = AutoTokenizer.from_pretrained('xlm-roberta-large')`, `tokenizer.tokenize('²')` or, altnernatively, `tokenizer.tokenize(chr(178))` returns `['▁2']`.


SWA

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSTM: @stas00
 -->


Adding the prepare_seq2seq_batch function to ProphetNet

# What does this PR do?

I tried to use ProphetNet with Seq2SeqTrainer, but it failed.
The error message told me: this is because the collator uses `prepare_seq2seq_batch()` in `_encode()`, but `prepare_seq2seq_batch()` is not implemented in ProphetNet Tokenizer.

I've gotten kind advices in the HuggingFace forum, and implemented the function.

https://discuss.huggingface.co/t/the-reason-prepare-seq2seq-batch-for-prophetnet-is-not-existed/1758

The modifications are as below:
- Add `prepare_seq2seq_batch()` in `/src/transformers/tokenization_prophetnet.py`.
- To use .view in loss computation in Seq2SeqTrainer, I add a part where it is confirmed that logits is contiguous in `/src/transformers/modeling_prophetnet.py`.

I've checked it works on CPU and GPU as below:

```
!python finetune_trainer.py \
    --learning_rate=3e-5 \
    --do_train --do_eval --evaluate_during_training \
    --max_source_length 511 \
    --per_device_train_batch_size 2 \
    --predict_with_generate \
    --n_train 300 \
    --n_val 100 \
    --model_name_or_path microsoft/prophetnet-large-uncased \
    --data_dir $XSUM_DIR \
    --output_dir tmp_gpu \
    --overwrite_output_dir
```

Although the `PRETRAINED_POSITIONAL_EMBEDDINGS_SIZES` is 512,  If `--max_source_lenght` is set to 512, `CUDA error` occurs.
I'm sorry, but I have not been able to identify the cause of this.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
https://discuss.huggingface.co/t/the-reason-prepare-seq2seq-batch-for-prophetnet-is-not-existed/1758
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?
https://github.com/forest1988/colaboratory/blob/main/prophetnet_seq2seqtrainer.ipynb
I'm sorry, I misunderstood what is being asked here.  Now I understood that the `./tests/`  code is needed.
~~I'm working on this, but I'm getting errors in formatting etc. and using `black` won't fix it.~~
I added related content to the `test_tokenization_prophetnet.py`. I changed the environment and made it work again according to the instructions, and it seems that formatting with `black` works appropriately.

## Who can review?

@patrickvonplaten
@sshleifer

Thank you for kindly answering my questions in the forum!


How to pretrain the model (like Roberta) again?

I don't want to pretrain the model from scratch. I have some dataset related to my task. I want to pretrain the model from transformers the second time. Could someone give me some advice on how to do it or which document to read?  Thank you!


Using Pretrained BERT model to add additional words that are not recognized by the model

Hello,

I want some help regarding adding additional words in the existing BERT model. I have two quires kindly guide me:

I am working on NER task for a domain:

There are few words (not sure the exact numbers) that BERT recognized as [UNK], but those entities are required for the model to recognize. The pretrained model learns well (up to 80%) accuracy on "bert-base-cased" while providing labeled data and fine-tune the model but intuitively the model will learn better if it recognize all the entities.

1. Do i need to add those unknown entities in vocabs.txt and train the model again?

2. Do i need to train the BERT model on my data from Scratch?

Thanks...


Issue while model sharing and uploading on huggingface

The model I am using (Bert, XLNet ...): Roberta for questionanswering

I am trying to follow the tutorial given in https://huggingface.co/transformers/model_sharing.html and while I am able to load my model in the local repository, I am unable to save my model and tokenizer using "model.save_pretrained("https://huggingface.co/saburbutt/testing")
tokenizer.save_pretrained("https://huggingface.co/saburbutt/testing")"

If I try to open the link it says "Cannot GET /saburbutt/testing/tokenizer_config.json"

While I try to "echo "hello" >> README.md" or use git functions, It gives me the error "fatal: not a git repository (or any of the parent directories): .git" The task I am working on is SQuAD

<!-- A clear and concise description of what you would expect to happen. -->
I expect the model to be saved in the huggingface/saburbutt/testing repository


Adding Confusion matrix support in Trainer

I want to add confusion matrix support in Trainer. It would be a useful addition. The only dependency would be `sklearn` which this library already uses for metrics. It can allow users to better understand about predictions coming from model. @sgugger Let me know if this is something you want to add.


Finetune TFBertForMaskedLM model.fit() ValueError

## The Problem
I have been trying to train TFBertForMaskedLM model with tensorflow. But when i use model.fit() always encounter some question.Hope someone can help and propose some solution.

## Reference Paper and sample output
The Paper title is "Conditional Bert for Contextual Augmentation". In short, just change type_token_ids to label_ids. if the label of sentence is 5, length is 10 and max_sequence_length = 16. It will process output as follows:
```
input_ids = [101, 523, 791, 3189, 677, 5221, 524, 1920, 686, 102, 0, 0, 0, 0, 0, 0]
attention_mask = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0]
token_type_ids = [5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 0, 0, 0, 0, 0, 0]
labels = [-100, -100, 791, -100, 677, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100]
```

## Environment
- tensorflow == 2.2.0
- huggingface == 3.5.0
- datasets == 1.1.2
- dataset total label is 5. (1~5)
- GPU : GCP P100 * 1

## Dataset output (max_sequence_length=128, batch_size=1)
```python
{'attention_mask': <tf.Tensor: shape=(128,), dtype=int32, numpy=
 array([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], dtype=int32)>,
 'input_ids': <tf.Tensor: shape=(128,), dtype=int32, numpy=
 array([  101,   523,   791,  3189,   677,  5221,   524,  1920,   686,
         4518,  6240,   103,  2466,  2204,  2695,   100,   519,  5064,
         1918,   736,  2336,   520,   103,  2695,  1564,  4923,  8013,
          678,  6734,  8038,  8532,   131,   120,   120,  8373,   119,
          103,  9989,   103,  8450,   120,   103,   120, 12990,  8921,
         8165,   102,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0], dtype=int32)>,
 'labels': <tf.Tensor: shape=(128,), dtype=int32, numpy=
 array([-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        4634, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        4158, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, 8429, -100,  119, -100, -100,  100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100], dtype=int32)>,
 'token_type_ids': <tf.Tensor: shape=(128,), dtype=int32, numpy=
 array([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], dtype=int32)>}
```

## Model code
```python
from transformers import AdamWeightDecay, TFBertForMaskedLM, BertConfig

def create_model():
    configuration = BertConfig.from_pretrained('bert-base-chinese')
    model = TFBertForMaskedLM.from_pretrained('bert-base-chinese',
                                              config=configuration)
    model.bert.embeddings.token_type_embeddings = tf.keras.layers.Embedding(5, 768,
                                                                            embeddings_initializer=tf.keras.initializers.TruncatedNormal(stddev=0.02))
    return model
model = create_model()

optimizer = tf.keras.optimizers.Adam(learning_rate=3e-5)
loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
metrics = [tf.keras.metrics.Mean(), tf.keras.metrics.SparseCategoricalAccuracy('accuracy')]

model.compile(optimizer = optimizer,
              loss = loss,
              metrics = metrics)

model.fit(tf_sms_dataset,
          epochs=1,
          verbose=1)
```

## Warning Message when use TFBertForMaskedLM
```
Some layers from the model checkpoint at bert-base-chinese were not used when initializing TFBertForMaskedLM: ['nsp___cls']
- This IS expected if you are initializing TFBertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing TFBertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
All the layers of TFBertForMaskedLM were initialized from the model checkpoint at bert-base-chinese.
If your task is similar to the task the model of the checkpoint was trained on, you can already use TFBertForMaskedLM for predictions without further training.
```

## Error Message
```
ValueError                                Traceback (most recent call last)
<ipython-input-42-99b78906fef7> in <module>()
      5 model.fit(tf_sms_dataset,
      6           epochs=1,
----> 7           verbose=1)

10 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py in _method_wrapper(self, *args, **kwargs)
     64   def _method_wrapper(self, *args, **kwargs):
     65     if not self._in_multi_worker_mode():  # pylint: disable=protected-access
---> 66       return method(self, *args, **kwargs)
     67
     68     # Running inside `run_distribute_coordinator` already.

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq, max_queue_size, workers, use_multiprocessing)
    846                 batch_size=batch_size):
    847               callbacks.on_train_batch_begin(step)
--> 848               tmp_logs = train_function(iterator)
    849               # Catch OutOfRangeError for Datasets of unknown size.
    850               # This blocks until the batch has finished executing.

/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py in __call__(self, *args, **kwds)
    578         xla_context.Exit()
    579     else:
--> 580       result = self._call(*args, **kwds)
    581
    582     if tracing_count == self._get_tracing_count():

/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py in _call(self, *args, **kwds)
    625       # This is the first call of __call__, so we have to initialize.
    626       initializers = []
--> 627       self._initialize(args, kwds, add_initializers_to=initializers)
    628     finally:
    629       # At this point we know that the initialization is complete (or less

/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py in _initialize(self, args, kwds, add_initializers_to)
    504     self._concrete_stateful_fn = (
    505         self._stateful_fn._get_concrete_function_internal_garbage_collected(  # pylint: disable=protected-access
--> 506             *args, **kwds))
    507
    508     def invalid_creator_scope(*unused_args, **unused_kwds):

/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py in _get_concrete_function_internal_garbage_collected(self, *args, **kwargs)
   2444       args, kwargs = None, None
   2445     with self._lock:
-> 2446       graph_function, _, _ = self._maybe_define_function(args, kwargs)
   2447     return graph_function
   2448

/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py in _maybe_define_function(self, args, kwargs)
   2775
   2776       self._function_cache.missed.add(call_context_key)
-> 2777       graph_function = self._create_graph_function(args, kwargs)
   2778       self._function_cache.primary[cache_key] = graph_function
   2779       return graph_function, args, kwargs

/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py in _create_graph_function(self, args, kwargs, override_flat_arg_shapes)
   2665             arg_names=arg_names,
   2666             override_flat_arg_shapes=override_flat_arg_shapes,
-> 2667             capture_by_value=self._capture_by_value),
   2668         self._function_attributes,
   2669         # Tell the ConcreteFunction to clean up its graph once it goes out of

/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/func_graph.py in func_graph_from_py_func(name, python_func, args, kwargs, signature, func_graph, autograph, autograph_options, add_control_dependencies, arg_names, op_return_value, collections, capture_by_value, override_flat_arg_shapes)
    979         _, original_func = tf_decorator.unwrap(python_func)
    980
--> 981       func_outputs = python_func(*func_args, **func_kwargs)
    982
    983       # invariant: `func_outputs` contains only Tensors, CompositeTensors,

/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py in wrapped_fn(*args, **kwds)
    439         # __wrapped__ allows AutoGraph to swap in a converted function. We give
    440         # the function a weak reference to itself to avoid a reference cycle.
--> 441         return weak_wrapped_fn().__wrapped__(*args, **kwds)
    442     weak_wrapped_fn = weakref.ref(wrapped_fn)
    443
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/func_graph.py in wrapper(*args, **kwargs)
    966           except Exception as e:  # pylint:disable=broad-except
    967             if hasattr(e, "ag_error_metadata"):
--> 968               raise e.ag_error_metadata.to_exception(e)
    969             else:
    970               raise

ValueError: in user code:

    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:571 train_function  *
        outputs = self.distribute_strategy.run(
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:951 run  **
        return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:2290 call_for_each_replica
        return self._call_for_each_replica(fn, args, kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:2649 _call_for_each_replica
        return fn(*args, **kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:541 train_step  **
        self.trainable_variables)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:1804 _minimize
        trainable_variables))
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py:521 _aggregate_gradients
        filtered_grads_and_vars = _filter_grads(grads_and_vars)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py:1219 _filter_grads
        ([v.name for _, v in grads_and_vars],))

    ValueError: No gradients provided for any variable: ['tf_bert_for_masked_lm_2/bert/embeddings/word_embeddings/weight:0', 'tf_bert_for_masked_lm_2/bert/embeddings/position_embeddings/embeddings:0', 'tf_bert_for_masked_lm_2/bert/embeddings/LayerNorm/gamma:0', 'tf_bert_for_masked_lm_2/bert/embeddings/LayerNorm/beta:0', 'tf_bert_for_masked_lm_2/bert/embeddings/embedding_1/embeddings:0', 'tf_bert_for_masked_lm_2/bert/encoder/layer_._0/attention/self/query/kernel:0', 'tf_bert_for_masked_lm_2/bert/encoder/layer_._0/attention/self/query/bias:0', 'tf_bert_for_masked_lm_2/bert/encoder/layer_._0/attention/self/key/kernel:0', 'tf_bert_for_masked_lm_2/bert/encoder/layer_._0/attention/self/key/bias:0', 'tf_bert_for_masked_lm_2/bert/encoder/layer_._0/attention/self/value/kernel:0', 'tf_bert_for_masked_lm_2/bert/encoder/layer_._0/attention/self/value/bias:0', 'tf_bert_for_masked_lm_2/bert/encoder/layer_._0/attention/output/dense/kernel:0', 'tf_bert_for_masked_lm_2/bert/encoder/layer_._0/attention/output/dense/bias:0', 'tf_bert_for_masked_lm_2/bert/encoder/layer_._0/attention/output/LayerNorm/gamma:0', 'tf_bert_for_masked_lm_2/bert/encoder/layer_._0/attention/output/LayerNorm/beta:0', 'tf_bert_for_masked_lm_2/bert/encoder/layer_._0/intermediate/dense/kernel:0', 'tf_bert_for_masked_lm_2/bert/encoder/layer_._0/intermediate/dense/bias:0', 'tf_bert_for_masked_lm_2/bert/encoder/layer_._0/output/dense/kernel:0', 'tf_bert_for_masked_lm_2/bert/encoder/layer_._0/output/dense/bias:0', 'tf_bert_f...
```

Have Someone can help. I will thanks a lot.

## Other Test
I used english sentence to test. example as follows:
```python
from transformers import TFBertForMaskedLM, BertConfig

def create_model():
    configuration = BertConfig.from_pretrained('bert-base-uncased')
    model = TFBertForMaskedLM.from_pretrained('bert-base-uncased',
                                              config=configuration)
    return model

model = create_model()
eng_tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
token_info = eng_tokenizer(text="We are very happy to show you the 🤗 Transformers library.", padding='max_length', max_length=20)

optimizer = tf.keras.optimizers.Adam(learning_rate=3e-5)
loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
metrics = [tf.keras.metrics.Mean(), tf.keras.metrics.SparseCategoricalAccuracy("acc")]

dataset = tf.data.Dataset.from_tensor_slices(dict(token_info))
dataset = dataset.batch(1).prefetch(tf.data.experimental.AUTOTUNE)

model.compile(optimizer = optimizer,
              loss = model.compute_loss,
              metrics = metrics)

model.fit(dataset)
```

token_info output dataset
```
{
  'input_ids': [101, 2057, 2024, 2200, 103, 2000, 2265, 2017, 103, 100, 19081, 3075, 1012, 102, 0, 0, 0, 0, 0, 0]
  'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0]
  'token_type_ids': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
  'labels': [-100, -100, -100, -100, 3407, -100, -100, -100, 1996, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100]
}
```

Get same error.....
```
ValueError: in user code:

    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:571 train_function  *
        outputs = self.distribute_strategy.run(
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:951 run  **
        return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:2290 call_for_each_replica
        return self._call_for_each_replica(fn, args, kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:2649 _call_for_each_replica
        return fn(*args, **kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:541 train_step  **
        self.trainable_variables)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:1804 _minimize
        trainable_variables))
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py:521 _aggregate_gradients
        filtered_grads_and_vars = _filter_grads(grads_and_vars)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py:1219 _filter_grads
        ([v.name for _, v in grads_and_vars],))

    ValueError: No gradients provided for any variable: ['tf_bert_for_masked_lm_2/bert/embeddings/word_embeddings/weight:0', 'tf_bert_for_masked_lm_2/bert/embeddings/position_embeddings/embeddings:0', 'tf_bert_for_masked_lm_2/bert/embeddings/token_type_embeddings/embeddings:0', 'tf_bert_for_masked_lm_2/bert/embeddings/LayerNorm/gamma:0', 'tf_bert_for_masked_lm_2/bert/embeddings/LayerNorm/beta:0',
```

I'm not sure if there is a problem with the integration of fit() into the model?


Model templates encoder only

Only merge the encoder part, not the encoder-decoder part of #7636.

Will work on a decoder in the future.

Applied your comments @sgugger @patrickvonplaten, but opened a new PR on a new branch so that we can keep the old one for reference when integrating the encoder-decoder model.


TPU issue: possible memory leak in eval loop

I am running into a HBM OOM during the eval loop of xlnet (`--model_name_or_path xlnet-large-cased`) when running on TPUs. No matter which batch size I use, the behavior is the same:
1. training loop succeeds
2. eval loop starts, makes it about halfway, then the TPU runs out of HBM memory and eval loop dies

All the other models that we test are OK. The `xlnet-large-cased` test last passed on 2020-09-14.

Since this is unrelated to batch size, I thought maybe there is a memory leak on the TPU. I think the eval loop is the more likely culprit than the training loop since the only OOM happens during eval.

Here are the last few lines of output before oom:
```
E 2020-11-12T04:51:27.984001264Z Saving model checkpoint to MNLI
E 2020-11-12T04:51:27.989368910Z Configuration saved in MNLI/config.json
E 2020-11-12T04:51:40.438957029Z Model weights saved in MNLI/pytorch_model.bin
E 2020-11-12T04:51:40.535782031Z 11/12/2020 04:51:40 - INFO - run_glue -   *** Evaluate ***
E 2020-11-12T04:51:40.536480018Z The following columns in the evaluation set don't have a corresponding argument in `XLNetForSequenceClassification.forward` and have been ignored: idx, hypothesis, premise.
E 2020-11-12T04:51:40.540513400Z ***** Running Evaluation *****
E 2020-11-12T04:51:40.540566285Z   Num examples = 9815
E 2020-11-12T04:51:40.540575559Z   Batch size = 8
E 2020-11-12T05:11:26.995136217Z
  0%|          | 0/154 [00:00<?, ?it/s]
  1%|1         | 2/154 [00:11<14:01,  5.53s/it]
  2%|1         | 3/154 [00:22<18:15,  7.25s/it]
...
 49%|####9     | 76/154 [14:34<15:49, 12.17s/it]
 50%|#####     | 77/154 [14:48<16:25, 12.80s/it]2020-11-12 05:11:26.994477: E     511 tensorflow/compiler/xla/xla_client/xla_util.cc:76] >>> Dumping Computation 0
```

I'm not sure what the issue could be. It seems like both the [training loop](https://github.com/huggingface/transformers/blob/master/src/transformers/trainer.py#L743) and the [eval loop](https://github.com/huggingface/transformers/blob/master/src/transformers/trainer.py#L1409) are using `ParallelLoader`, which should call `xm.mark_step` for [every call to `next`](https://github.com/pytorch/xla/blob/master/torch_xla/distributed/parallel_loader.py#L37).

Does anyone else have any ideas what could be happening?

## Environment info

- `transformers` version: 3.5.0
- Platform: Linux-4.9.0-13-amd64-x86_64-with-debian-9.13
- Python version: 3.6.10
- PyTorch version (GPU?): 1.8.0a0+d0df29a (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: Yes
- Using TPU in script?: Yes

### Who can help
@sgugger @LysandreJik

## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: MNLI
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. git clone https://github.com/huggingface/transformers.git
2. cd transformers && pip install .
3. pip install datasets
4. Training command:

```
              python examples/xla_spawn.py \
                --num_cores 8 \
                examples/text-classification/run_glue.py \
                --logging_dir=./tensorboard-metrics \
                --task_name MNLI \
                --cache_dir ./cache_dir \
                --do_train \
                --do_eval \
                --num_train_epochs 3 \
                --max_seq_length 128 \
                --learning_rate 3e-5 \
                --output_dir MNLI \
                --overwrite_output_dir \
                --logging_steps 100 \
                --save_steps 3000 \
                --overwrite_cache \
                --tpu_metrics_debug \
               --model_name_or_path xlnet-large-cased \
              --per_device_train_batch_size 32 \
              --per_device_eval_batch_size 8
```

## Expected behavior

Eval loop finishes without TPU OOM.


Fill-mask pipeline removes space after token prediction when loading pre-training model based on roberta-base

## Environment info
- `transformers` version: 3.5.0
- Platform: Linux-4.15.0-122-generic-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.5
- PyTorch version (GPU?): 1.7.0+cu101 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: False
- Using distributed or parallel set-up in script?: False

### Who can help @mfuntowicz

## Information

Model I am using (Bert, XLNet ...): **roberta-base**

The problem arises when using:
* [x] the official example scripts:
-> continued **pre-training roberta-base** using the v3.5.0 examples/language-modeling/**run_mlm_wwm.py**
* [x] my own modified scripts:
To test the model during training I simply instantiate a pipeline class with target pretraining checkpoint folder and input masked strings to check the probabilities:
```
unmasker = pipeline('fill-mask', model=model_checkpoint_path)
results = unmasker(masked_text)
print(json.dumps(results, indent=4))
```

## Expected behavior

The expected behavior for input string "The goal of MASK is happiness." if loading model "roberta-base" would be:
[
    {
        "sequence": "The goal of life is happiness.",
        "score": 0.07787031680345535,
        "token": 301,
        "token_str": "\u0120life"
    },
    {
        "sequence": "The goal of meditation is happiness.",
        "score": 0.040741581469774246,
        "token": 20183,
        "token_str": "\u0120meditation"
    }
]

## Observed behavior

For the same input string I obtain no space following the predicted token when loading the further pre-trained model from a checkpoint folder, example result:
 [
    {
        "sequence": "The goal of Kiwis happiness.",
        "score": 0.11430764198303223,
        "token": 21472,
        "token_str": "\u0120Kiw"
    },
    {
        "sequence": "The goal of anis happiness.",
        "score": 0.04334629327058792,
        "token": 41,
        "token_str": "\u0120an"
    },
    {
        "sequence": "The goal of buis happiness.",
        "score": 0.03720756620168686,
        "token": 10306,
        "token_str": "\u0120bu"
    }
]

As I thought it might be a tokenizer issue from the checkpoint, I tried to specify one from the roberta-base model used to continue the pre-training from. It solves the issue, so it seems like the pre-training steps have corrupted the tokenizer as loaded from the checkpoint. The results I get after 200,000 steps of additional pretraining from roberta-base:
[
    {
        "sequence": "The goal of this is happiness.",
        "score": 0.28572556376457214,
        "token": 42,
        "token_str": "\u0120this"
    },
    {
        "sequence": "The goal of it is happiness.",
        "score": 0.10664933174848557,
        "token": 24,
        "token_str": "\u0120it"
    },
    {
        "sequence": "The goal of all is happiness.",
        "score": 0.07055338472127914,
        "token": 70,
        "token_str": "\u0120all"
    },
    {
        "sequence": "The goal of life is happiness.",
        "score": 0.056005414575338364,
        "token": 301,
        "token_str": "\u0120life"
    }
]

-> so regardless of the quality of the result token prediciton, loading the roberta-base original tokenizer solves the issue.


DPR model: FileNotFoundError: Couldn't find file

I am using DPR model:
```python
from transformers import DPRQuestionEncoderTokenizer, DPRQuestionEncoder
from datasets import load_dataset

question_tokenizer = DPRQuestionEncoderTokenizer.from_pretrained('facebook/dpr-question_encoder-single-nq-base')
question_encoder = DPRQuestionEncoder.from_pretrained('facebook/dpr-question_encoder-single-nq-base')
wiki = load_dataset("wiki_dpr", with_embeddings=False, with_index=True, split="train")

def get_top(question, topk=5):
    question_emb = question_encoder(**question_tokenizer(question, return_tensors="pt"))[0].detach().numpy()
    passages_scores, passages = wiki.get_nearest_examples("embeddings", question_emb, k=topk)
    all_passgae = ""
    for score, title, text in zip(passages_scores, passages['title'], passages['text']):
        if len(all_passgae.split(" ")) < 450:
            all_passgae += f" ({title}) {text}"
    return all_passgae

get_top("who was the first US president?")
```

This was working until last week. However, now when running it I am getting the following error:
```
Downloading: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 232k/232k [00:00<00:00, 1.65MB/s]
Downloading: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 493/493 [00:00<00:00, 443kB/s]
Downloading: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 438M/438M [00:05<00:00, 73.8MB/s]
Downloading: 7.91kB [00:00, 7.04MB/s]
Downloading: 21.9kB [00:00, 19.5MB/s]
Using custom data configuration psgs_w100.no_embeddings.compressed
Downloading and preparing dataset wiki_dpr/psgs_w100.no_embeddings.compressed (download: Unknown size, generated: Unknown size, post-processed: Unknown size, total: Unknown size) to /home/danielk/.cache/huggingface/datasets/wiki_dpr/psgs_w100.no_embeddings.compressed/0.0.0/14b973bf2a456087ff69c0fd34526684eed22e48e0dfce4338f9a22b965ce7c2...
Downloading: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.54k/1.54k [00:00<00:00, 1.66MB/s]
Downloading: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 13.8G/13.8G [03:41<00:00, 62.3MB/s]
Traceback (most recent call last):
  File "2.create_tasks.py", line 9, in <module>
    wiki = load_dataset("wiki_dpr", with_embeddings=False, with_index=True, split="train")
  File "/home/danielk/qoogle-experiments/env37/lib/python3.7/site-packages/datasets/load.py", line 611, in load_dataset
    ignore_verifications=ignore_verifications,
  File "/home/danielk/qoogle-experiments/env37/lib/python3.7/site-packages/datasets/builder.py", line 468, in download_and_prepare
    self._download_prepared_from_hf_gcs()
  File "/home/danielk/qoogle-experiments/env37/lib/python3.7/site-packages/datasets/builder.py", line 507, in _download_prepared_from_hf_gcs
    resource_path = utils.cached_path(remote_cache_dir + "/" + resource_file_name)
  File "/home/danielk/qoogle-experiments/env37/lib/python3.7/site-packages/datasets/utils/file_utils.py", line 308, in cached_path
    use_etag=download_config.use_etag,
  File "/home/danielk/qoogle-experiments/env37/lib/python3.7/site-packages/datasets/utils/file_utils.py", line 474, in get_from_cache
    raise FileNotFoundError("Couldn't find file at {}".format(url))
FileNotFoundError: Couldn't find file at https://storage.googleapis.com/huggingface-nlp/cache/datasets/wiki_dpr/psgs_w100.no_embeddings.compressed/0.0.0/psgs_w100.nq.IVFPQ4096_HNSW32_PQ64-IP-train.faiss
```

I wonder if the changes made to the model-hub has anything to do with this.
@LysandreJik @lhoestq @julien-c

Here is my environment, for completeness:

```
Python 3.7.5 (default, Nov  7 2019, 10:50:52)
[GCC 8.3.0] on linux
```
and
```
tensorboard              2.4.0
tensorboard-plugin-wit   1.7.0
tensorboardX             2.1
tensorflow               2.3.1
tensorflow-datasets      4.1.0
tensorflow-estimator     2.3.0
tensorflow-metadata      0.25.0
tensorflow-text          2.3.0
termcolor                1.1.0
tfds-nightly             4.1.0.dev202011120108
threadpoolctl            2.1.0
tokenizers               0.9.3
torch                    1.6.0
tqdm                     4.49.0
transformers             3.5.0
typing-extensions        3.7.4.3
urllib3                  1.26.1
```


Unexpected behavior when using PubMedBERT with AutoModelForMaskedLM

## Information

I'm getting some strange behavior when using `AutoModelForMaskedLM` with PubMedBERT to impute masked tokens. The screenshot below shows a simple example where I would expect PubMedBERT to give reasonable values, but the suggested tokens are really strange. As shown just below this, `bert-base-uncased` seems to behave reasonably.

<img width="1013" alt="Screen Shot 2020-11-12 at 10 28 13 AM" src="https://user-images.githubusercontent.com/3958904/98984623-4e96b400-24d7-11eb-9a5f-faac70b1b399.png">

Same code as above, in text:

```python
import torch
from transformers import AutoTokenizer, AutoModelForMaskedLM

model_name = 'microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract'
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForMaskedLM.from_pretrained(model_name).to('cuda')

text = f'Heart disease is {tokenizer.mask_token} leading cause of death in the United States.'
tokenized = tokenizer(text, return_tensors='pt').to('cuda')
print(tokenizer.convert_ids_to_tokens(tokenized.input_ids.squeeze()))
output = model(**tokenized, return_dict=True)
output.logits.size()
print(tokenizer.convert_ids_to_tokens(torch.topk(output.logits[0, 4, :], 10).indices))

model_name = 'bert-base-uncased'
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForMaskedLM.from_pretrained(model_name).to('cuda')

tokenized = tokenizer(text, return_tensors='pt').to('cuda')
print(tokenizer.convert_ids_to_tokens(tokenized.input_ids.squeeze()))
output = model(**tokenized, return_dict=True)
print(tokenizer.convert_ids_to_tokens(torch.topk(output.logits[0, 4, :], 10).indices))
```

## Environment info

- `transformers` version: 3.3.1
- Platform: Linux-3.10.0-957.5.1.el7.x86_64-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyTorch version (GPU?): 1.6.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No


Failed to push model repo

Hi, when I upload my model to hub as the [new document](https://huggingface.co/transformers/model_sharing.html) say, I got this error:
```
Delta compression using up to 16 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (4/4), 1.07 GiB | 3.08 MiB/s, done.
Total 4 (delta 0), reused 1 (delta 0)
remote:
remote: -------------------------------------------------------------------------
remote: Your push was rejected because it contains files larger than 10M.
remote: Please use https://git-lfs.github.com/ to store larger files.
remote: -------------------------------------------------------------------------
remote:
remote: Offending files:
remote:  - tf_model.h5 (ref: refs/heads/main)
To https://huggingface.co/mymusise/gpt2-medium-chinese
 ! [remote rejected] main -> main (pre-receive hook declined)
error: failed to push some refs to 'https://huggingface.co/mymusise/gpt2-medium-chinese'
```

## Environment info

- `transformers` version: 3.5.0
- Platform: Ubuntu 18.04


Training the TFGPT2LMHeadModel with model.fit produces error

MWE:
```python
from transformers.modeling_tf_gpt2 import TFGPT2LMHeadModel
from transformers.configuration_gpt2 import GPT2Config
from tensorflow.data import Dataset
import tensorflow as tf

data = tf.random.uniform(shape=[10000], dtype = tf.int32, maxval = 100)
src = tf.constant(data)
def split_input_target(chunk):
    return chunk[:-1], chunk[1:]
ds = Dataset.from_tensor_slices(src) \
            .batch(256 + 1, drop_remainder = True) \
            .map(split_input_target)
model = TFGPT2LMHeadModel(GPT2Config())
loss = ['sparse_categorical_crossentropy'] + [None] * 12
model.compile(loss = loss, metrics = ['sparse_categorical_accuracy'])
model.fit(ds)
```
This generates the error `ValueError: Dimensions must be equal, but are 256 and 12 for '{{node Equal_1}} = Equal[T=DT_FLOAT, incompatible_shape_error=true](Cast_6, Cast_7)' with input shapes: [256,1], [2,256,12,1].` Is there any explanation somewhere on how to train like this?


TF T5-small with output hidden state and attention not owrking

- `transformers` version: 2.11
- Platform: Multiple
- Python version: multiple

### Who can help
 T5: @patrickvonplaten
 tensorflow: @jplu

## Information

Model I am using (Bert, XLNet ...): T5

The problem arises when t5-small is loaded from pretrain with output_hidden_states=True, output_attentions=True

sample script https://colab.research.google.com/drive/1oF8hMaQg1yl2fE6QPUYKSTZcer4Mlk6S?usp=sharing

if these parameter is removed script works

I am getting following error.

```python
/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_utils.py in generate(self, input_ids, max_length, min_length, do_sample, early_stopping, num_beams, temperature, top_k, top_p, repetition_penalty, bad_words_ids, bos_token_id, pad_token_id, eos_token_id, length_penalty, no_repeat_ngram_size, num_return_sequences, attention_mask, decoder_start_token_id, use_cache)
    780                 encoder_outputs=encoder_outputs,
    781                 attention_mask=attention_mask,
--> 782                 use_cache=use_cache,
    783             )
    784         else:

/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_utils.py in _generate_beam_search(self, input_ids, cur_len, max_length, min_length, do_sample, early_stopping, temperature, top_k, top_p, repetition_penalty, no_repeat_ngram_size, bad_words_ids, bos_token_id, pad_token_id, decoder_start_token_id, eos_token_id, batch_size, num_return_sequences, length_penalty, num_beams, vocab_size, encoder_outputs, attention_mask, use_cache)
   1027                 input_ids, past=past, attention_mask=attention_mask, use_cache=use_cache
   1028             )
-> 1029             outputs = self(**model_inputs)  # (batch_size * num_beams, cur_len, vocab_size)
   1030             next_token_logits = outputs[0][:, -1, :]  # (batch_size * num_beams, vocab_size)
   1031

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py in __call__(self, *args, **kwargs)
    983
    984         with ops.enable_auto_cast_variables(self._compute_dtype_object):
--> 985           outputs = call_fn(inputs, *args, **kwargs)
    986
    987         if self._activity_regularizer:

/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_t5.py in call(self, inputs, **kwargs)
   1061             encoder_attention_mask=attention_mask,
   1062             head_mask=head_mask,
-> 1063             use_cache=use_cache,
   1064         )
   1065

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py in __call__(self, *args, **kwargs)
    983
    984         with ops.enable_auto_cast_variables(self._compute_dtype_object):
--> 985           outputs = call_fn(inputs, *args, **kwargs)
    986
    987         if self._activity_regularizer:

/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_t5.py in call(self, inputs, attention_mask, encoder_hidden_states, encoder_attention_mask, inputs_embeds, head_mask, past_key_value_states, use_cache, training)
    572             # required mask seq length can be calculated via length of past
    573             # key value states and seq_length = 1 for the last token
--> 574             mask_seq_length = shape_list(past_key_value_states[0][0])[2] + seq_length
    575         else:
    576             mask_seq_length = seq_length

IndexError: list index out of range
```

## Expected behavior

how to get the attention and hidden states as output?
Even if you can share a sample for pytorch I would be able to make it work for TF.


Why is the XLM-RoBERTa sometimes producing a standalone start of the word character (the special underscore with ord = 9601)

I'm using the `transformers` library 3.4.0.

The XLM-RoBERTa tokenizer produces in certain cases standalone start of the word characters. Is that intended?

For example:
```
tokenizer.tokenize('amerikanische')
['▁', 'amerikanische']
```
while
```
tokenizer.tokenize('englische')
['▁englische']
```


Fix doc bug

Fix the example of Trainer, hope it help.

@sgugger


Unable to install Transformers

Hi all - I'm unable to install transformers from source. I need this for a project, it's really annoying not be able to use your amazing work. Could you please help me? :) Thank you so much.

**Issue**
pip install is blocked at **sentencepiece-0.1.91** install and crashes

**What I tried**
- I tried to find a workaround by installing the latest version of sentencepiece 0.1.94 but it doesn't solve the issue
- I tried to download the repository locally and change the version requirement in setup.py and requirement.txt it doesn't solve neither
- My system: MacOS 10.15.7 / Python 3.9.0 / Pip 20.2.4 / Anaconda3 with PyTorch

**The error messages and pip list to show you I installed latest sentencepiece**

```
(env) (base) Cecilias-MacBook-Air:transformers mymacos$ pip3 install -e .
Obtaining file:///Users/mymacos/Documents/OpenAI/transformers
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting filelock
  Using cached filelock-3.0.12-py3-none-any.whl (7.6 kB)
Collecting sentencepiece==0.1.91
  Using cached sentencepiece-0.1.91.tar.gz (500 kB)
    ERROR: Command errored out with exit status 1:
     command: /Users/mymacos/Documents/OpenAI/env/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/s7/73dlmpfj3253cpbpl6v96rbm0000gn/T/pip-install-5ceji0j1/sentencepiece/setup.py'"'"'; __file__='"'"'/private/var/folders/s7/73dlmpfj3253cpbpl6v96rbm0000gn/T/pip-install-5ceji0j1/sentencepiece/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/s7/73dlmpfj3253cpbpl6v96rbm0000gn/T/pip-pip-egg-info-svt86xy8
         cwd: /private/var/folders/s7/73dlmpfj3253cpbpl6v96rbm0000gn/T/pip-install-5ceji0j1/sentencepiece/
    Complete output (5 lines):
    Package sentencepiece was not found in the pkg-config search path.
    Perhaps you should add the directory containing `sentencepiece.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'sentencepiece' found
    Failed to find sentencepiece pkgconfig
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

(env) (base) Cecilias-MacBook-Air:transformers mymacos$ pip list
Package           Version
----------------- -------
astroid           2.4.2
isort             5.6.4
lazy-object-proxy 1.4.3
mccabe            0.6.1
numpy             1.19.4
pip               20.2.4
pylint            2.6.0
PyYAML            5.3.1
**sentencepiece     0.1.94**
setuptools        50.3.2
six               1.15.0
toml              0.10.2
wheel             0.35.1
wrapt             1.12.1
```


Model sharing doc

# What does this PR do?

This PR expands the model sharing doc with some instructions specific to colab.

Unrelated: some fixes in marian.rst that I thought I had pushed directly to master but had not.


Error when loading a model cloned without git-lfs is quite cryptic

Feature request

Request for a new feature

# 🚀 Error message request

If you forget to install git-LFS (e.g. on Google Colab) and you just do:

```python
!git clone https://huggingface.co/facebook/bart-base
from transformers import AutoModel
model = AutoModel.from_pretrained('./bart-base')
```

The cloning seems to work well but the model weights are not downloaded. The error message is then quite cryptic and could probably be tailored to this (probably) common failure case:

```
loading weights file ./bart-large-cnn/pytorch_model.bin
---------------------------------------------------------------------------
UnpicklingError                           Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    950             try:
--> 951                 state_dict = torch.load(resolved_archive_file, map_location="cpu")
    952             except Exception:

4 frames
UnpicklingError: invalid load key, 'v'.

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    952             except Exception:
    953                 raise OSError(
--> 954                     f"Unable to load weights from pytorch checkpoint file for '{pretrained_model_name_or_path}' "
    955                     f"at '{resolved_archive_file}'"
    956                     "If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True. "

OSError: Unable to load weights from pytorch checkpoint file for './bart-large-cnn' at './bart-large-cnn/pytorch_model.bin'If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.
```

git-LFS specification files are pretty simple to parse and typically look like this:
```
version https://git-lfs.github.com/spec/v1
oid sha256:097417381d6c7230bd9e3557456d726de6e83245ec8b24f529f60198a67b203a
size 440473133
```

The first *key* is always `version`: https://github.com/git-lfs/git-lfs/blob/master/docs/spec.md


Created ModelCard for Hel-ach-en MT model

model card

Related to pretrained model cards

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSTM: @stas00
 -->