BlenderBot RuntimeError: CUDA error: device-side assert triggered

## Environment info

- `transformers` version: 4.0.0
- Platform: Linux-5.4.0-56-generic-x86_64-with-glibc2.29
- Python version: 3.8.5
- PyTorch version (GPU?): 1.7.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: yes  (GTX 1060 6GB)
- Using distributed or parallel set-up in script?: no

### Who can help
@patrickvonplaten

## Information

Model I am using (Bert, XLNet ...): I am using the BlenderbotForConditionalGeneration ('facebook/blenderbot-90M') along with the relevant small tokenizer.

The problem arises when using:

I am using my own trainer implementation. I think that the problem has to do with the indexes of the labels. More specifically when I am using:

```outputs = self.model(input_ids=inputs, attention_mask=inputs_att, labels=pad_targets, return_dict=True)```

everything works fine as the "pad_targets" are the targets using 0  as the index for masked (padded) tokens.
However when I am using:

```outputs = self.model(input_ids=inputs, attention_mask=inputs_att,  labels=repl_targets, return_dict=True)```
and then printing the outputs['loss'] the following error is occurred:

`RuntimeError: CUDA error: device-side assert triggered`

as the "repl_targets" are the targets using the -100  as the index for masked (padded) tokens.

The aforementioned error also occurs when using the argument:
`decoder_input_ads=repl_targets`

The tasks I am working on is:
 Dialogue generation in Empathetic Dialogues dataset.

## Expected behavior

I think that there is a problem with the -100 padding token. But I am not sure :)


🐛 [TF_BART] "<internal expr>" has dtype float32 in the TRUE branch, but dtype=int32 in the FALSE branch

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.1.0.dev0
- Platform: Linux-4.19.0-13-cloud-amd64-x86_64-with-debian-10.7
- Python version: 3.7.3
- PyTorch version (GPU?): 1.7.0 (False)
- Tensorflow version (GPU?): 2.2.0 (False)
- Using GPU in script?: No (TPU)
- Using distributed or parallel set-up in script?: Yes

### Who can help

@patrickvonplaten

## Information

Model I am using (Bert, XLNet ...): TFBart

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

## To reproduce

When I try to run TF_Bart on TPU, I'm getting the following error :

> TypeError: "<internal expr>" has dtype float32 in the TRUE branch, but dtype=int32 in the FALSE branch. TensorFlow control flow requires that they are the same.

It seems to come from the dropout operation :

https://github.com/huggingface/transformers/blob/b01ddc9577b87f057e163d49563ee3f74f4810cf/src/transformers/models/bart/modeling_tf_bart.py#L373

<details>
 <summary> Full stack trace (click to expand...)</summary>

>2020/12/11 00:00:55 - INFO - transformers_addons.trainer_tf -   ***** Running Evaluation *****
2020/12/11 00:00:55 - INFO - transformers_addons.trainer_tf -     Batch size = 8
The parameters `output_attentions`, `output_hidden_states` and `use_cache` cannot be updated when calling a model.They have to be set to True/False in the config object (i.e.: `config=XConfig.from_pretrained('name', output_attentions=True)`).
The parameter `return_dict` cannot be set in graph mode and will always be set to `True`.
The parameters `output_attentions`, `output_hidden_states` and `use_cache` cannot be updated when calling a model.They have to be set to True/False in the config object (i.e.: `config=XConfig.from_pretrained('name', output_attentions=True)`).
The parameter `return_dict` cannot be set in graph mode and will always be set to `True`.
Traceback (most recent call last):
  File "train.py", line 203, in <module>
    main()
  File "train.py", line 194, in main
    result = trainer.evaluate()
  File "/home/remondnicola/text-summarization/transformers_addons/trainer_tf.py", line 281, in evaluate
    output = self._prediction_loop(eval_dataset, description="Evaluation", prediction_loss_only=prediction_loss_only)
  File "/home/remondnicola/text-summarization/transformers_addons/trainer_tf.py", line 207, in _prediction_loop
    loss, logits = self._evaluate_steps(features, labels)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/def_function.py", line 580, in __call__
    result = self._call(*args, **kwds)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/def_function.py", line 627, in _call
    self._initialize(args, kwds, add_initializers_to=initializers)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/def_function.py", line 506, in _initialize
    *args, **kwds))
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/function.py", line 2446, in _get_concrete_function_internal_garbage_collected
    graph_function, _, _ = self._maybe_define_function(args, kwargs)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/function.py", line 2777, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/function.py", line 2667, in _create_graph_function
    capture_by_value=self._capture_by_value),
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/func_graph.py", line 981, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/def_function.py", line 441, in wrapped_fn
    return weak_wrapped_fn().__wrapped__(*args, **kwds)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/function.py", line 3299, in bound_method_wrapper
    return wrapped_fn(*args, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/func_graph.py", line 968, in wrapper
    raise e.ag_error_metadata.to_exception(e)
TypeError: in user code:
>
>    /home/remondnicola/text-summarization/transformers_addons/trainer_tf.py:169 _evaluate_steps  *
        per_replica_loss, per_replica_logits = self.args.strategy.experimental_run_v2(
    train.py:29 _run_model  *
        out = self.model(features, training=training, **labels)
    /home/remondnicola/text-summarization/transformers_addons/models/bart/modeling_tf_bart.py:88 call  *
        outputs = super().call(inputs["input_ids"],
    /home/remondnicola/.venv/summarization/lib/python3.7/site-packages/transformers/models/bart/modeling_tf_bart.py:1110 call  *
        outputs = self.model(
    /home/remondnicola/.venv/summarization/lib/python3.7/site-packages/transformers/models/bart/modeling_tf_bart.py:977 call  *
        inputs["encoder_outputs"] = self.encoder(
    /home/remondnicola/.venv/summarization/lib/python3.7/site-packages/transformers/models/bart/modeling_tf_bart.py:373 call  *
        x = tf.nn.dropout(x, rate=self.dropout if training else 0)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/autograph/operators/control_flow.py:924 if_stmt
        basic_symbol_names, composite_symbol_names)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/autograph/operators/control_flow.py:962 tf_if_stmt
        error_checking_orelse)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/util/deprecation.py:507 new_func
        return func(*args, **kwargs)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/control_flow_ops.py:1177 cond
        return cond_v2.cond_v2(pred, true_fn, false_fn, name)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/cond_v2.py:91 cond_v2
        op_return_value=pred)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/func_graph.py:981 func_graph_from_py_func
        func_outputs = python_func(*func_args, **func_kwargs)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/autograph/operators/control_flow.py:958 error_checking_orelse
        basic_symbol_names + composite_symbol_names)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/autograph/operators/control_flow.py:298 _verify_tf_cond_vars
        functools.partial(_verify_single_cond_var, name), body_var, orelse_var)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/util/nest.py:617 map_structure
        structure[0], [func(*x) for x in entries],
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/util/nest.py:617 <listcomp>
        structure[0], [func(*x) for x in entries],
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/autograph/operators/control_flow.py:267 _verify_single_cond_var
        orelse_var.dtype.name))
>
>    TypeError: "<internal expr>" has dtype float32 in the TRUE branch, but dtype=int32 in the FALSE branch. TensorFlow control flow requires that they are the same.

</details>


XLNet ONNX model giving error: "Attempting to broadcast an axis by a dimension other than 1"

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.5.1
- Platform: Linux-4.14.193-113.317.amzn1.x86_64-x86_64-with-glibc2.10
- Python version: 3.7.6
- PyTorch version (GPU?): 1.7.0 (True)
- Tensorflow version (GPU?): 2.3.1 (True)
- Using GPU in script?: False
- Using distributed or parallel set-up in script?: False

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten @TevenLeScao
 Blenderbot: @patrickvonplaten
 Bart: @patrickvonplaten
 Marian: @patrickvonplaten
 Pegasus: @patrickvonplaten
 mBART: @patrickvonplaten
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 RAG: @patrickvonplaten, @lhoestq
 FSMT: @stas00
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

@TevenLeScao @mfuntowicz @patil-suraj

## Information

Model I am using (Bert, XLNet ...): XLNet

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Trained HuggingFace Transformers model XLNetForSequenceClassification on custom dataset with PyTorch backend.
2. Used provided `convert_graph_to_onnx.py` script to convert model (from saved checkpoint) to ONNX format.
3. Loaded the model with ONNXRuntime
4. When feeding in int64 numpy arrays `input_ids` and `attention_masks`, the model returns the following error except when both inputs have shape (x, 1) or (x, 6). There is nothing in the configuration of the model or the structure of the training data from my end that would require shape (x, 1) or (x, 6).

```
RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Non-zero status code returned while running Add node. Name:'Add_26' Status Message: /onnxruntime_src/onnxruntime/core/providers/cpu/math/element_wise_ops.h:361 void onnxruntime::BroadcastIterator::Init(int64_t, int64_t) axis == 1 || axis == largest was false. Attempting to broadcast an axis by a dimension other than 1. 5 by 6
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
The expected behavior is for the model to return predictions successfully (i.e. probabilities for all classes).


The example code does not work

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.3.1
- Platform: Linux-4.4.0-154-generic-x86_64-with-Ubuntu-16.04-xenial
- Python version: 3.6.12
- PyTorch version (GPU?): 1.6.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 -->
 @LysandreJik  @sgugger
## Information

When I run the code about Question Answering from the documentation https://huggingface.co/transformers/task_summary.html, there is an error reported.

## To reproduce

```
from transformers import AutoTokenizer, AutoModelForQuestionAnswering
import torch
tokenizer = AutoTokenizer.from_pretrained("bert-large-uncased-whole-word-masking-finetuned-squad")
model = AutoModelForQuestionAnswering.from_pretrained("bert-large-uncased-whole-word-masking-finetuned-squad")
text = r"""
🤗 Transformers (formerly known as pytorch-transformers and pytorch-pretrained-bert) provides general-purpose
architectures (BERT, GPT-2, RoBERTa, XLM, DistilBert, XLNet…) for Natural Language Understanding (NLU) and Natural
Language Generation (NLG) with over 32+ pretrained models in 100+ languages and deep interoperability between
TensorFlow 2.0 and PyTorch.
"""
if __name__ == "__main__":
    questions = [
        "How many pretrained models are available in 🤗 Transformers?",
        "What does 🤗 Transformers provide?",
        "🤗 Transformers provides interoperability between which frameworks?",
    ]
    for question in questions:
        inputs = tokenizer(question, text, add_special_tokens=True, return_tensors="pt")
        input_ids = inputs["input_ids"].tolist()[0]
        text_tokens = tokenizer.convert_ids_to_tokens(input_ids)
        outputs = model(**inputs)
        answer_start_scores = outputs.start_logits
        answer_end_scores = outputs.end_logits
        answer_start = torch.argmax(
            answer_start_scores
        )  # Get the most likely beginning of answer with the argmax of the score
        answer_end = torch.argmax(answer_end_scores) + 1  # Get the most likely end of answer with the argmax of the score
        answer = tokenizer.convert_tokens_to_string(tokenizer.convert_ids_to_tokens(input_ids[answer_start:answer_end]))
        print(f"Question: {question}")
        print(f"Answer: {answer}")
```
## Error message
  File "/home/pxf109/LegalContractModel/example.py", line 22, in <module>
    answer_start_scores = outputs.start_logits
AttributeError: 'tuple' object has no attribute 'start_logits


[finetune_trainer] enhancements and fixes

The main need was to add speed metrics to perform speed performance regressions. But on the way a bunch of other things got worked on. Hopefully you will find the proposed changes useful.

This PR change `trainer`

* [x] adds an optional `metric_key_prefix` for `evaluate` and `predict` functions to return metrics with a prefix key set by the user rather than the default `eval_`.

This PR change `finetune_trainer`

* [x] utils: sort json keys when dumping to filesystem
* [x] renames s/eval/val/ for the validation dataset results
* [x] adds speed metrics for all: train/eval/test (samples_per_second/runtime/n_objs)
* [x] refactors logging/saving code for each mode
* [x] renames internal vars to tell which is metrics and which is output that is more than just metrics
* [x] fixes a bug where all_results.json wasn't getting saved in the right place
* [x] rounds up loss values to 4 decimals - before it was `"eval_loss": 368.2950744628906,` - not sure if it's better done upstream in the trainer?

Here is a sample of `all_results.json` after this change:

```
{
    "epoch": 1.0,
    "test_bleu": 22.8548,
    "test_gen_len": 35.9,
    "test_loss": 734.8612,
    "test_n_ojbs": 10,
    "test_runtime": 2.5185,
    "test_samples_per_second": 3.971,
    "train_n_ojbs": 200,
    "train_runtime": 24.9101,
    "train_samples_per_second": 8.029
    "val_bleu": 26.581,
    "val_gen_len": 31.4,
    "val_loss": 738.3438,
    "val_n_ojbs": 200,
    "val_runtime": 33.9329,
    "val_samples_per_second": 5.894,
}
```

@sgugger, @patil-suraj, @patrickvonplaten


google/bert2bert_L-24_wmt_de_en doesn't match official implementation

## Environment info
- `transformers` version: 4.0.0
- Platform: Linux-5.4.0-1030-aws-x86_64-with-debian-buster-sid
- Python version: 3.7.8
- PyTorch version (GPU?): 1.7.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

### Who can help
@patrickvonplaten ; maybe @patil-suraj

## Information

I'm trying to running the `transformers` implementation of WMT14 DE->EN translation, using the `google/bert2bert_L-24_wmt_de_en` checkpoint and [instructions](https://huggingface.co/google/bert2bert_L-24_wmt_de_en).

The BLEU score I get using translations from `transformers` implementation are substantially lower than those I get from [the official Tensorflow model](https://github.com/google-research/google-research/tree/master/bertseq2seq) -- 24.7 w/ HF vs 34.0 w/ the official implementation.

## To reproduce

The following snippet shows qualitative differences in the output of the models:

```python
import datasets
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

# --
# Load dataset

dataset  = datasets.load_dataset("wmt14", "de-en", split="test")
sentence = dataset[20]['translation']['de']
target   = dataset[20]['translation']['en']

print(target)
# If the street is clear, the pedestrian obtains a green light immediately, if not, there is a delay of around 15 seconds.

# --
# HF model

tokenizer  = AutoTokenizer.from_pretrained("google/bert2bert_L-24_wmt_de_en", pad_token="<pad>", eos_token="</s>", bos_token="<s>")
model      = AutoModelForSeq2SeqLM.from_pretrained("google/bert2bert_L-24_wmt_de_en")

input_ids  = tokenizer(sentence, return_tensors="pt", add_special_tokens=False).input_ids
output_ids = model.generate(input_ids)[0]

output_str = tokenizer.decode(output_ids, skip_special_tokens=True)

print(output_str)
# the road is free, it takes about 15 seconds if not directly for the footganger.

# --
# TF model

import tensorflow.compat.v1 as tf
import tensorflow_hub as hub
import tensorflow_text as tf_text

tf.disable_eager_execution()

# Load model
model = hub.Module('https://tfhub.dev/google/bertseq2seq/bert24_de_en/1')

# Setup session
sess = tf.InteractiveSession()
sess.run(tf.tables_initializer())
sess.run(tf.global_variables_initializer())

# Define graph

src       = tf.placeholder(tf.string, shape=[None])
translate = model(src)

# Translate
output_str = sess.run(translate, feed_dict = {
    src : [sentence]
})

print(output_str[0])
# "If the road is clear, there is a green area for the pedestrian, if not it takes about 15 seconds."
```

I can also share the (custom) scripts I'm using to run inference on the entire dataset and compute BLEU scores.  Note I am using the same BLEU code for both implementations.

## Expected behavior

I would expect the BLEU scores and the quality of the translations to be comparable.

Thanks!


Zero Shot Classification Pipeline fails when running in CPU-only Docker container

## Environment info
- `transformers` version: 4.0.0
- Platform: MacOS 10.15.7 (2018 MacBook Pro 15")
- Python version: 3.6.9
- PyTorch version (GPU?): 1.7.0 CPU Only
- Tensorflow version (GPU?): N/A
- Using GPU in script?: N/A
- Using distributed or parallel set-up in script?: N/A

### Who can help
Maybe @LysandreJik ?

## Information

Model I am using (Bert, XLNet ...): facebook/bart-large-mnli

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [X] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [X] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Pull and start the official [transformers-pytorch-cpu](https://hub.docker.com/r/huggingface/transformers-pytorch-cpu/dockerfile) container.
2. `docker exec -it huggingface bash`
3. `python3`
4. `from transformers import pipeline`
5. `classifier = pipeline("zero-shot-classification", model='facebook/bart-large-mnli', tokenizer='facebook/bart-large-mnli', device=-1)`

Step 4 above results in the following warning being printed:
```
/usr/local/lib/python3.6/dist-packages/torch/cuda/__init__.py:52: UserWarning: CUDA initialization: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx (Triggered internally at  /pytorch/c10/cuda/CUDAFunctions.cpp:100.)
  return torch._C._cuda_getDeviceCount() > 0
```

Step 5 above results in the models being downloaded, then `Killed` is printed and the Python interpreter exits.

## Expected behavior
When running locally in a Jupyter Notebook or directly in the terminal (not in a container), the following works correctly and the warning about the CUDA initialization isn't printed:
```
from transformers import pipeline
classifier = pipeline("zero-shot-classification", model='facebook/bart-large-mnli', tokenizer='facebook/bart-large-mnli', device=-1)
```

The problem seems to be limited to either the zero shot classification pipeline, or the facebook/bart-large-mnli model, since the following works correctly in the container (though the warning from Step 4 about the CUDA initialization is still printed):
```
from transformers import pipeline
classifier = pipeline('sentiment-analysis')
```


BERT outputs are different with the same input in training mode

When the training mode is enabled, BERT model returns different outputs even for the same input, is there any idea on why this happens?

```python
from transformers import BertModel, BertTokenizer
model = BertModel.from_pretrained("bert-base-uncased")
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
a = tokenizer.encode("Hello how are you?", return_tebsors='pt')
model.train()

torch.mean(model(a)[0])
# tensor(-0.0167, grad_fn=<MeanBackward0>)
torch.mean(model(a)[0])
# tensor(-0.0162, grad_fn=<MeanBackward0>)
torch.mean(model(a)[0])
# tensor(-0.0156, grad_fn=<MeanBackward0>)

```


Fix typo #9012 (#1)

There is a tiny typo in the code "transformers/examples/language-modeling/run_mlm_wwm.py" at line 284. [Details.](https://github.com/huggingface/transformers/issues/9012)

# What does this PR do?

Fixes #9012

## Before submitting
- [Y] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [Y] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

 albert, bert, XLM: @LysandreJik
 examples/distillation: @VictorSanh
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


fix the typo 9012

# What does this PR do?

Fixes #9012

## Before submitting
- [Y] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [Y] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @
 albert, bert, XLM: @LysandreJik
 examples/distillation: @VictorSanh
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


[docs] missing info on call back registry

https://huggingface.co/transformers/main_classes/callback.html is missing instructions/examples on how to register a callback. Thanks.

@sgugger


Improve coverage of the documentation

Currently, some public classes are not documented anywhere because we didn't create the corresponding doc pages. Those missing pages are:

- Benchmark classes
- Bert Japanese
- Data collators

If someone feels like working on one of those, please tag yourself with a comment on this issue. Once the objects are properly documented, they can be removed from the `SHOULD_BE_DOCUMENTED` constant in [this file](https://github.com/huggingface/transformers/blob/1310e1a758edc8e89ec363db76863c771fbeb1de/utils/check_repo.py#L374).


Refactor FLAX tests

# What does this PR do?

This PR refactors the FLAX models tests in a `test_modeling_flax_common` file and speeds them up by using small random models instead of pretrained ones. It will hopefully speed up the CI and make it less flaky!


Make ProphetNetModel really compatible with EncoderDecoder

The interesting part of ProphetNet is its decoder which can do n-gram causal language modeling. So it could be very interesting to load a pre-trained prophetnet decoder model into an encoder-decoder design with - let's say - a longformer encoder for long-range sequence modeling.

Due to some narrow-minded thinking on my part, this didn't work previously.

```python
from transformers import EncoderDecoderModel

EncoderDecoderModel.from_encoder_decoder_pretrained("allenai/longformer-large-4096", "microsoft/prophetnet-large-uncased")
```

As one can see none of pre-trained **decoder** weights are loaded into the model. The reason is because `ProphetNetForCausalLM` was badly modularized in `ProphetNetForCausalLM`.

Merging this PR would make it possible to load any prophetnet decoder into an encoder-decoder model and fine-tuning an "build-it-yourself" encoder decoder would become much easier, *e.g.*:

```python
from transformers import EncoderDecoderModel
import torch

model = EncoderDecoderModel.from_encoder_decoder_pretrained("allenai/longformer-large-4096", "microsoft/prophetnet-large-uncased")

input_ids = torch.tensor([10 * [1]])
labels = torch.tensor([10 * [0]])
loss = model(input_ids, decoder_input_ids=labels, labels=labels).loss
loss.backward()
```

The above use-case might also be interesting for @ibeltagy actually.

## Breaking changes

This does introduce a pretty heavy breaking change to `ProphetNetForCausalLM`. However, the only reason this class was created was to make it useable with `EncoderDecoderModel` and this arguably failed a bit the first time since it made it way too difficult to load pretrained ProphetNet models into the `EncoderDecoderModel`. I guess I see this more of solving a bug then "new design". Also there are no pre-trained `ProphetNetForCausalLM` models on the model hub and I highly doubt anybody has really used this class.

I want to use the same pattern for BartForCausalLM and T5ForCausalLM, so it'd be great to get this merged even though there are some breaking changes.


Hi, I am trying to run below code, it can be found at this [link](https://huggingface.co/transformers/model_doc/dpr.html#dprreader)
```
from transformers import DPRReader, DPRReaderTokenizer
tokenizer = DPRReaderTokenizer.from_pretrained('facebook/dpr-reader-single-nq-base')
model = DPRReader.from_pretrained('facebook/dpr-reader-single-nq-base')
encoded_inputs = tokenizer(
        questions=["What is love ?"],
        titles=["Haddaway"],
        texts=["'What Is Love' is a song recorded by the artist Haddaway"],
        return_tensors='pt'
    )
outputs = model(**encoded_inputs)
start_logits = outputs.stat_logits
end_logits = outputs.end_logits
relevance_logits = outputs.relevance_logits
```
But got error

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.0.2
- Platform: Windows (GCP Instance)
- Python version: 3.8.6
- PyTorch version (GPU?): '1.7.0+cpu'
- Tensorflow version (GPU?): Not installed
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten @TevenLeScao
 Blenderbot: @patrickvonplaten
 Bart: @patrickvonplaten
 Marian: @patrickvonplaten
 Pegasus: @patrickvonplaten
 mBART: @patrickvonplaten
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 RAG: @patrickvonplaten, @lhoestq
 FSMT: @stas00
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)
I am just trying to run DPR model.

## To reproduce

Steps to reproduce the behavior:

1. Execute this in your python
```
from transformers import DPRReader, DPRReaderTokenizer
tokenizer = DPRReaderTokenizer.from_pretrained('facebook/dpr-reader-single-nq-base')
model = DPRReader.from_pretrained('facebook/dpr-reader-single-nq-base')
encoded_inputs = tokenizer(
        questions=["What is love ?"],
        titles=["Haddaway"],
        texts=["'What Is Love' is a song recorded by the artist Haddaway"],
        return_tensors='pt'
    )
outputs = model(**encoded_inputs)
start_logits = outputs.stat_logits
end_logits = outputs.end_logits
relevance_logits = outputs.relevance_logits
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->
The error traceback is below:
```
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-2f646c4dd41f> in <module>
----> 1 from transformers import DPRReader, DPRReaderTokenizer
      2 tokenizer = DPRReaderTokenizer.from_pretrained('facebook/dpr-reader-single-nq-base')
      3 model = DPRReader.from_pretrained('facebook/dpr-reader-single-nq-base')
      4 encoded_inputs = tokenizer(
      5         questions=["What is love ?"],

ImportError: cannot import name 'DPRReader' from 'transformers' (C:\<some_path>\env\lib\site-packages\transformers\__init__.py)

```

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
I haven't executed this but I would hope nothing in output as all the results are stored in variables.


GPT2 attention mask

I want to use gpt2 to generate a list of options

> an option is a sentence starting with a special token '<option>'.

As I don't want the following options to rely on the previous options, I think I should mask all the previous options.

I could simply implement that during generation by generating one option per time, but I don't know how to do that during training.


Initial README for `t5-small-indonesian-summarization-cased` model

model card

Related to pretrained model cards

Initial README for Indonesian T5 Summarization Small Model


[TF Bart] Refactor TFBart

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Mirror of #8900 for TFBart.

The same improvements are done for Bart except adding torchscript functionality (as it does not exist in tf bart).

- [x] Keep dims consistent within the model -> no switching around between time x batch_size and batch_size x time. We can just stick to batch_size x time throughout the whole forward pass just like other models do too.
- [x] Clean the Attention layer: Replace dict cache by past_key_values tuple (consistency with other models and stateless which is better IMO). Break up complicated if-else cascade and remove unnecessary parameters.
- [x] Correct error with past_key_values/decoder_input_ids/use_cache
- [x] Add input_embeds to Bart
- [x] (very subjectively) better naming
- [x] Check that all slow tests are passing
- [x] Update docstring and final design change check
- [x] Refactor Bart tests
- [x] should solve https://github.com/huggingface/transformers/issues/9048
- [x] Check no speed regression


Initial README for `t5-base-indonesian-summarization-cased` model

Related to pretrained model cards

Initial README for Indonesian T5 Summarization Base Model


Uber AI plug and play language model (PPLM)

Hi Team,

Thanks for the hugging face repo and appreciate  your great efforts towards adding datasets and models, I was trying to find PPLM model in model page but it was shown as 404 error, could you please check the model  if it's available and let me know

Thanks

Ajay


Compatibility scripts

# 🚀 Feature request

Scripts that translate code in older transformers versions into equivalent code that is compatible with newer versions.

## Motivation

After talking with several other people in my research groups, compatibility issues and getting stuck with old versions have turned out to be pretty common problems. Seeing as many of the things preventing backward compatibility are syntactic (slightly different interfaces for tokenizers, different file paths) I thought it might be possible to add scripts to the package, which translate code to fit, say, one major version higher up (then if a user wanted to step up multiple versions, they could just run several scripts in sequence).

Some rudimentary usage example:
`python transformers-2.11-3.0.py my_project/*`

## Your contribution

I could try to implement such an example scripts, but it would probably take me months and result in a sub-optimal output due to my knowledge about python parsing and the interface changes between major and minor versions of transformers.
I did find out a [refactoring](https://github.com/python-rope/rope) library for python, or some snippet showing how to [unparse a python AST](https://svn.python.org/projects/python/trunk/Demo/parser/unparse.py).


Untranslation of some words from an external dictionary

I use some pre-trained translator models from your library (for example, Helsinki-NLP). During the translation process, I would like to leave some words untranslatable (for example, acronyms, toponyms or names) due to the presence of errors in their translation.  I tried adding extra tokens and replacing these words with conditional tokens (<extra_id_0>), but this translation requires raising the num_beams parameter, which significantly slows down the translation. Unfortunately, I can't find any additional mechanisms to perform this task. Is it provided or creating it is a custom task?


Use Softmax classifiering for run_glue.py example

Hi,
I want to do binary text classification and I'm adapting [run_glue.py](https://github.com/huggingface/transformers/blob/master/examples/text-classification/run_glue.py)  script to my task. The current model uses a linear classifier and the predictions are not in the range of [0,1]. Could you please guide me on how I could use softmax classifier instead of linear classifier?

I added following code after the model is loaded but I get an error related related to loss function that I pasted below. Any suggestion on how to fix this?
   `model.classifier = torch.nn.Softmax(dim=1)`

```
File "run_glue.py", line 300, in main
    model_path=model_args.model_name_or_path if os.path.isdir(model_args.model_name_or_path) else None
  File "/Users/royaho/PycharmProjects/download_pretrained_model/venv/lib/python3.6/site-packages/transformers/trainer.py", line 775, in train
    tr_loss += self.training_step(model, inputs)
  File "/Users/royaho/PycharmProjects/download_pretrained_model/venv/lib/python3.6/site-packages/transformers/trainer.py", line 1112, in training_step
    loss = self.compute_loss(model, inputs)
  File "/Users/royaho/PycharmProjects/download_pretrained_model/venv/lib/python3.6/site-packages/transformers/trainer.py", line 1136, in compute_loss
    outputs = model(**inputs)
  File "/Users/royaho/PycharmProjects/download_pretrained_model/venv/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/Users/royaho/PycharmProjects/download_pretrained_model/venv/lib/python3.6/site-packages/transformers/modeling_bert.py", line 1377, in forward
    loss = loss_fct(logits.view(-1, self.num_labels), labels.view(-1))
  File "/Users/royaho/PycharmProjects/download_pretrained_model/venv/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/Users/royaho/PycharmProjects/download_pretrained_model/venv/lib/python3.6/site-packages/torch/nn/modules/loss.py", line 962, in forward
    ignore_index=self.ignore_index, reduction=self.reduction)
  File "/Users/royaho/PycharmProjects/download_pretrained_model/venv/lib/python3.6/site-packages/torch/nn/functional.py", line 2468, in cross_entropy
    return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
  File "/Users/royaho/PycharmProjects/download_pretrained_model/venv/lib/python3.6/site-packages/torch/nn/functional.py", line 2262, in nll_loss
    .format(input.size(0), target.size(0)))
ValueError: Expected input batch_size (768) to match target batch_size (3).

```


run_clm.py Issue | MODEL_FOR_CAUSAL_LM_MAPPING is None

When I use latest code of [run_clm.py](https://github.com/huggingface/transformers/blob/master/examples/language-modeling/run_clm.py), it raises the following issue:
```
python run_clm.py \
    --model_name_or_path gpt2 \
    --train_file path_to_train_file \
    --validation_file path_to_validation_file \
    --do_train \
    --do_eval \
    --output_dir /tmp/test-clm
```

```
Traceback (most recent call last):
  File "run_clm.py", line 51, in <module>
    MODEL_CONFIG_CLASSES = list(MODEL_FOR_CAUSAL_LM_MAPPING.keys())
AttributeError: 'NoneType' object has no attribute 'keys'
```
I checked and noticed that MODEL_FOR_CAUSAL_LM_MAPPING is None. Any suggestion?


About the input of BERT

Hello, if I want to maintain two different dictionaries, one is BERT's original dictionary and the other is a custom dictionary, and then the input is `[CLS]  BERT dictionary corpus [SEP] custom dictionary corpus [SEP]`, how do I handle the input of the model and what part of the source code do I need to change? Thanks!


Error tokenizer = AutoTokenizer.from_pretrained("vinai/phobert-base")

I run pretrain PhoBERT error .
ValueError                                Traceback (most recent call last)
<ipython-input-75-d17717702336> in <module>()
      3
      4 phobert = AutoModel.from_pretrained("vinai/phobert-base")
----> 5 tokenizer = AutoTokenizer.from_pretrained("vinai/phobert-base")
      6
      7 # INPUT TEXT MUST BE ALREADY WORD-SEGMENTED!

/usr/local/lib/python3.6/dist-packages/transformers/models/auto/tokenization_auto.py in from_pretrained(cls, pretrained_model_name_or_path, *inputs, **kwargs)
    323             if tokenizer_class is None:
    324                 raise ValueError(
--> 325                     "Tokenizer class {} does not exist or is not currently imported.".format(tokenizer_class_candidate)
    326                 )
    327             return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)

ValueError: **Tokenizer class PhobertTokenizerFast does not exist or is not currently imported.**


Fix typo in modeling_tf_bart

# What does this PR do?

Fix typo in `modeling_tf_bart`

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?

@patrickvonplaten @sshleifer


getattr introduces bug when setting booleans with config file

Hi,

In finetune_trainer.py, line 162: https://github.com/huggingface/transformers/blob/5e637e6c690e45d13ebf7296e1ea9dcc188d0f07/examples/seq2seq/finetune_trainer.py#L162

If the user is calling this script with a json config file and setting one of the attributes to false, then on line 162, the result of if `getattr(training_args, p, None)` would be false and if condition would not be called, this results in bug for setting booleans,  could you change this line to following to resolve it:

`if hasattr(training_args, p): `

thank you.


Fix PreTrainedTokenizer.pad when first inputs are empty

# What does this PR do?

Currently, `PreTrainedTokenizer.pad` errors when the first `input_ids` are empty (because it tries to guess the type of the tokens by looking at the first element). This PR slightly changes the behavior to loop until we find a non empty list.

Fixes #8674 (not the initial issue but the one mentioned at the end)


Fix documention of book in LayoutLM

# What does this PR do?

The documentation of the `bbox` argument in the LayoutLM models has some bad copy-paste errors, this PR fixes that.

Fixes #9016


LayoutLM wrong shape for bbox in docs

## Environment info
(Colab, 09 december 2020, CPU runtime)
- `transformers` version: 4.0.1
- Platform: Linux-4.19.112+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.7.0+cu101 (False)
- Tensorflow version (GPU?): 2.3.0 (False)
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help
 documentation: @sgugger

## Information

LayoutLM documentation indicates that the shape for the bbox input is [B, seq_len]:
![image](https://user-images.githubusercontent.com/10614968/101692115-76355980-3a4e-11eb-9aa2-a0218c26fead.png)

However, from the code (https://github.com/huggingface/transformers/blob/master/src/transformers/models/layoutlm/modeling_layoutlm.py#L103) bounding boxes are encoded in the form [tl_col, tl_row, br_col, br_row]. Therefore the accepted shape is [B, seq_len, 4].

## To reproduce

Reproduced on this colab: https://colab.research.google.com/drive/1ZRPKlX8-C41nYq3o6QVS68h1JAj1nQMI?usp=sharing

## Expected behavior

Better explanation in docs


MPNet copyright files

# What does this PR do?

MPnet and the copyright PRs were merged around the same time, so MPNet does not have copyright in every files it introduced. This PR fixes that.


Enforce all objects in the main init are documented

# What does this PR do?

Some objects added by contributors or the team are regularly forgotten. This PR changes the script that inspects whether or not models are documented to encompass all objects in the main init (and adds documentation for multiple forgotten objects).


[model_cards] Migrate cards from this repo to model repos on huggingface.co

Related to pretrained model cards

Fellow reviewers/contributors, please take a look at the documentation part and let me know your thoughts.

---
#### ⚠ Still to-do before merging ⚠
- [x] Post a message on the Forum: https://discuss.huggingface.co/t/announcement-all-model-cards-will-be-migrated-to-hf-co-model-repos/2755
- [x] Update the buttons on the model pages
- [x] merge all out-standing model card PRs on the transformers repo
- [x] the actual migration into the hf.co model repos

ETA: I plan on doing this Thursday (Dec 10) or Friday (Dec 11)!


"run_mlm_wwm.py", line 284 AttributeError: 'DataTrainingArguments' object has no attribute 'valid_ref_file'

Hi! There is a tiny typo in the code "transformers/examples/language-modeling/run_mlm_wwm.py" at line 284. It should be:
`if data_args.validation_ref_file is not None:` since at line 103 in `DataTrainingArguments` it defined as `validation_ref_file:`


Create README.md

Related to pretrained model cards

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSMT: @stas00
 -->


Reorganize examples

# What does this PR do?

This PR reorganizes the examples folder by splitting it in two:
- `examples` that stay in this folder are the base example scripts maintained with the state of the library, expected to work on master. We accept PRs on them and will try our best to fix issues.
- `research-projects` are (often) more complex examples that we don't really maintain. They work on a specific version of the library  (sometimes even a specific commit). We don't accept PRs on them except minor typo fixes *or* PRs from the original authors that want to bring an update to those scripts. Issues opened for those are probably less efficient than directly contacting the authors.

Each example/research project lives in a folder of its own, with its particular requirements in a `requirements.txt` file (instead of a global requirements file as before).

The seq2seq subfolder is less organized than the others, so I did my best to split its research project part from its example part. I made sure all tests are passing and duplicated the needed files, but @stas00 and @patil-suraj please tell me if you something obvious that I missed. We will leave the research-project part as is and clean a bit more the part in the examples in other PRs.


fixes #8968

**This is the same PR as: [link](https://github.com/huggingface/transformers/pull/8991#issue-534604029). I was asked to create a new one due to a merge mistake.**

# What does this PR do? (Text of the previous PR)

One of the 3.X releases introduced output objects that replaced the previously returned tuples. This PR updates the transformers notebook to reflect that update.

Fixes #8968

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

@patrickvonplaten
@LysandreJik


[Docs] Fix some typos for group beam search


Fix link to stable version in the doc navbar

# What does this PR do?

Currently the link to the stable version in the navigation bar of the docs does not work properly, this PR fixes that.


Diverse beam search 2

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Copy of #8627 because branch got messed up.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSMT: @stas00
 -->


Add the code_search_net datasets tag to CodeBERTa model cards

Related to pretrained model cards

# What does this PR do?

TL;DR

Related to this PR on `huggingface/datasets`: https://github.com/huggingface/datasets/pull/1288

## Who can review?

@julien-c


Add MP Net 2

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Copy of #8971 that had to close because of problems with git history.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSMT: @stas00
 -->


Turn attentions and hidden-states into a tensor

# What does this PR do?

This PR turns the `all_attentions` and `all_hidden_states` values into tensors instead of a tuple. This update is to properly allow the dict outputs in TF serving, because the value of each key cannot be something else than a TF tensor.

Here a simple piece of code to produce the issue:
```
from transformers import TFBertModel, BertConfig
import tensorflow as tf
config = BertConfig.from_pretrained("bert-base-cased", output_attentions=True)
model = TFBertModel.from_pretrained("bert-base-cased", config=config)
tf.saved_model.save(model, "my_model")
```

Gets the error:
```
ValueError: Got a dictionary containing non-Tensor value (<tf.Tensor 'StatefulPartitionedCall:0' shape=(None, 12, None, None) dtype=float32>, <tf.Tensor 'StatefulPartitionedCall:1' shape=(None, 12, None, None) dtype=float32>, <tf.Tensor 'StatefulPartitionedCall:2' shape=(None, 12, None, None) dtype=float32>, <tf.Tensor 'StatefulPartitionedCall:3' shape=(None, 12, None, None) dtype=float32>, <tf.Tensor 'StatefulPartitionedCall:4' shape=(None, 12, None, None) dtype=float32>, <tf.Tensor 'StatefulPartitionedCall:5' shape=(None, 12, None, None) dtype=float32>, <tf.Tensor 'StatefulPartitionedCall:6' shape=(None, 12, None, None) dtype=float32>, <tf.Tensor 'StatefulPartitionedCall:7' shape=(None, 12, None, None) dtype=float32>, <tf.Tensor 'StatefulPartitionedCall:8' shape=(None, 12, None, None) dtype=float32>, <tf.Tensor 'StatefulPartitionedCall:9' shape=(None, 12, None, None) dtype=float32>, <tf.Tensor 'StatefulPartitionedCall:10' shape=(None, 12, None, None) dtype=float32>, <tf.Tensor 'StatefulPartitionedCall:11' shape=(None, 12, None, None) dtype=float32>) for key attentions in the output of the function __inference_serving_15889 used to generate a SavedModel signature. Dictionaries outputs for functions used as signatures should have one Tensor output per string key.
```


Add TFRag

# What does this PR do?
This is a reopen PR of TFRag Draft version  (https://github.com/huggingface/transformers/pull/8892 )
which somehow seems broken and not accessible at the moment .

## Things done
- `TFRagModel`
- `TFRagSequenceForGeneration`
- `TFRagTokenForGeneration`
- beam_search generation
- "Work-around" example in graph mode training (The full graph-mode training need of `.numpy()` for retriever calling, and this doesn't work on graph mode) --> using `context_input_ids` in place of `input_ids`
- Complete test on `TFRag`

## Things not yet done ...
- Integrate with T5 as generator <-- In the next PR

## Who can review?
@jplu @patrickvonplaten


🌟 CTRLsum

# 🌟 New model addition

## Model description

>Current summarization systems yield generic summaries that are disconnected from users’ preferences and expectations. To address this limitation, we present **CTRLsum**, a novel framework for controllable summarization.
>
> Our approach enables users to control multiple aspects of generated summaries by interacting with the summarization system through textual input in the form of a set of keywords or descriptive prompts.
Using a single unified model, CTRLsum is able to achieve a broad scope of summary manipulation at inference time without requiring additional human annotations or pre-defining a set of control aspects during training.
We quantitatively demonstrate the effectiveness of our approach on three domains of summarization datasets and five control aspects:
> 1) entity-centric
> 2) length-controllable summarization
> 3) contribution summarization on scientific papers
> 4) invention purpose summarization on patent filings
> 5) question-guided summarization on news articles in a reading comprehension setting
>
> Moreover, when used in a standard, uncontrolled summarization setting, CTRLsum achieves state-of-the-art results on the CNN/DailyMail dataset.

## Open source status

* [x] the model implementation is available: https://github.com/salesforce/ctrl-sum
* [x] the model weights are available: _Download link available in the README of the repo_
* [x] who are the authors: @jxhe @muggin


## Environment info
- `transformers` version:4.0.0
- Platform:Google Colab
- Python version:3
- Tensorflow version (GPU?):2.3.0
- Using GPU in script?:No

### Who can help
@patrickvonplaten
@patil-suraj
@jplu

## Information

I refer to the URL below and want to run the fine-tuning on mT5.
https://huggingface.co/transformers/training.html
https://huggingface.co/transformers/model_doc/mt5.html

Model I am using (mT5):

```
from transformers import MT5Model, T5Tokenizer, TFMT5Model
model = TFMT5Model.from_pretrained("google/mt5-base")
tokenizer = T5Tokenizer.from_pretrained("google/mt5-base")
```

```
from transformers import BertTokenizer, glue_convert_examples_to_features
import tensorflow as tf
import tensorflow_datasets as tfds

data = tfds.load('glue/mrpc')
train_dataset = glue_convert_examples_to_features(data['train'], tokenizer, max_length=128, task='mrpc')
train_dataset = train_dataset.shuffle(100).batch(32).repeat(2)
```

```
optimizer = tf.keras.optimizers.Adam(learning_rate=3e-5)
loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
model.compile(optimizer=optimizer, loss=loss)
model.fit(train_dataset, epochs=2, steps_per_epoch=115)
```

the output produced :
```
ValueError                                Traceback (most recent call last)
<ipython-input-9-650f77977ac3> in <module>()
      3 loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
      4 model.compile(optimizer=optimizer, loss=loss)
----> 5 model.fit(train_dataset, epochs=2, steps_per_epoch=115)
      6 # model.fit({"inputs": train_dataset},epochs=2, steps_per_epoch=115)
      7 # model.fit(train_dataset)

10 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/func_graph.py in wrapper(*args, **kwargs)
    971           except Exception as e:  # pylint:disable=broad-except
    972             if hasattr(e, "ag_error_metadata"):
--> 973               raise e.ag_error_metadata.to_exception(e)
    974             else:
    975               raise

ValueError: in user code:

    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:806 train_function  *
        return step_function(self, iterator)
    /usr/local/lib/python3.6/dist-packages/transformers/models/t5/modeling_tf_t5.py:1094 call  *
        decoder_outputs = self.decoder(
    /usr/local/lib/python3.6/dist-packages/transformers/models/t5/modeling_tf_t5.py:642 call  *
        raise ValueError(f"You have to specify either {err_msg_prefix}inputs or {err_msg_prefix}inputs_embeds")

    ValueError: You have to specify either decoder_inputs or decoder_inputs_embeds

```


AlbertTokenizer handles special tokens incorrectly

If I download the pretrained vocab `https://huggingface.co/albert-base-v1/resolve/main/spiece.model` to the local file system and use the following snippet, the tokenizer does not handle the special tokens properly:
```
tokenizer = AlbertTokenizer('spiece.model')
tokenizer.tokenize('[CLS] Hello World ! [SEP]')
['▁[', 'cl', 's', ']', '▁hello', '▁world', '▁', '!', '▁[', 's', 'ep', ']']
```
If I use `from_pretrained` to load the vocab, it works well:
```
tokenizer = AlbertTokenizer.from_pretrained('albert-base-v1')
tokenizer.tokenize('[CLS] Hello World ! [SEP]')
['[CLS]', '▁hello', '▁world', '▁', '!', '[SEP]']
```


Marge - Pre-training via Paraphrasing

# 🌟 New model addition

## Model description


[wip] [ci] doc-job-skip take #4.5 dry-run via github direct edit

This is take 4.5 on attempting to find a reliable way to get a list of modified files of this PR. It's identical to https://github.com/huggingface/transformers/pull/8980 but this PR was created from github UI direct file edit, so as we can see, it doesn't provide `CIRCLE_PR_NUMBER` - Nothing bad happens, but the check can't be done  since we have no information to work with :(

It also happens with PR's made from a non-personal branch, https://github.com/huggingface/transformers/pull/9015
And the result is that the check is completely skipped as it has no data to work with:
https://app.circleci.com/pipelines/github/huggingface/transformers/17118/workflows/48285d78-cb04-4feb-87f8-77cb02ac2593/jobs/134493

Hoping that circlePR will fix that bug.

This PR:

* [x] tests a PR submission from non-personal forked repo
* [x] switches to `head.user.login` for the username to checkout the branch with - using PR username as it's in the master will not work if the branch is coming from a non-forked repo (original that is). (could also use `.head.repo.full_name` for the whole thing at once.)

For now I will let this PR sit for a while and add other fixes if we find more edge cases.


Remove use of deprecated method in Trainer HP search

# What does this PR do?

Somehow this one slip in the cracks and was forgotten when we removed old deprecated method. This might warrant a release patch if we don't do a new release soon.

Fixes #8995


AttributeError: 'Trainer' object has no attribute 'is_world_master'

## Environment info

- `transformers` version: 4.0.0
- Platform: Linux-4.15.0-96-generic-x86_64-with-debian-buster-sid
- Python version: 3.7.8
- PyTorch version (GPU?): 1.7.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No
- Ray version: 1.0.1.post1

### Who can help
@sgugger — Would you be able to offer any insight?

## Information

Model I am using (Bert, XLNet ...): BertForSequenceClassification

The problem arises when using:
* [ ] the official example scripts:
* [ x] my own modified scripts:

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task:
* [ x] my own task or dataset:

## To reproduce
The error occurs when I try to run a hyperparameter search for my finetuning step using Ray Tune. I am able to successfully finetune the BertForSequenceClassification model normally — the error only arises when running hyperparameter search.

```
config = BertConfig.from_pretrained(pretrained_model_path, num_labels=num_labels, finetuning_task ='text-classification')

def model_init():
    return BertForSequenceClassification.from_pretrained(pretrained_model_path, config=config)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=train_dataset,
    eval_dataset=eval_dataset,
    model_init=model_init
)

trainer.hyperparameter_search(
    direction="minimize",
    backend="ray",
    n_trials=20,
    keep_checkpoints_num=1,
    resources_per_trial = {'gpu':1, 'cpu':1}
)
```

## Expected behavior

I am trying to run Ray Tune from Huggingface as per these instructions: https://huggingface.co/blog/ray-tune

If anyone has any insight as to what could be causing this error, it would be greatly appreciated, thank you!


DistilBert PyTorch to TensorFlow conversion - input sequence length is max 5 tokens for tensorflow

## Environment info

- `transformers` version: 3.3.1
- Platform: Linux-5.4.0-1028-aws-x86_64-with-debian-buster-sid
- Python version: 3.7.7
- PyTorch version (GPU?): 1.6.0 (True)
- Tensorflow version (GPU?): 2.3.1 (True)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

### Who can help
@LysandreJik
@jplu

## Information

Model I am using (Bert, XLNet ...): DistilBert

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

I load the PyTorch model as a TensorFlow model and then save it to the TensorFlow SavedModel format:
`tf_model = TFDistilBertModel.from_pretrained(model_name, from_pt=True)
tf.saved_model.save(tf_model, path)`

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

I am attempting to take a Sentence Transformer (ST) model I trained in PyTorch and use it in TensorFlow.js. The code above is specifically converting the DistilBert model ST uses to a TensorFlow SavedModel format. Then, I load the SavedModel format into a TFJS Graph Model format and write the pooling layer. When implementing a forward pass in JS, I noticed that the input sequence length must be 5 tokens (instead of 128). I checked the SavedModel format (.pb file) to rule out an issue from TF to TFJS and noticed that the shapes all have 5 where 128 should be.

## To reproduce

Steps to reproduce the behavior:

Run the above code on the DistilBert model we trained (not very reproducible). These are the contents of the folder:
* config.json
* pytorch_model.bin
* sentence_bert_config.json (this file has the max_seq_length=128 parameter - I tried adding it to config.json, but it doesn't work)
* special_tokens_map.json
* tokenizer_config.json
* vocab.txt

This is the output of the script.
2020-12-08 23:32:19.317202: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
2020-12-08 23:32:20.677265: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcuda.so.1
2020-12-08 23:32:20.677408: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-12-08 23:32:20.678158: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties:
pciBusID: 0000:00:1e.0 name: Tesla T4 computeCapability: 7.5
coreClock: 1.59GHz coreCount: 40 deviceMemorySize: 14.75GiB deviceMemoryBandwidth: 298.08GiB/s
2020-12-08 23:32:20.678182: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
2020-12-08 23:32:20.680072: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcublas.so.10
2020-12-08 23:32:20.681791: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcufft.so.10
2020-12-08 23:32:20.682130: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcurand.so.10
2020-12-08 23:32:20.684002: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusolver.so.10
2020-12-08 23:32:20.685065: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusparse.so.10
2020-12-08 23:32:20.688448: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudnn.so.7
2020-12-08 23:32:20.688575: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-12-08 23:32:20.689406: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-12-08 23:32:20.690132: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0
2020-12-08 23:32:20.690351: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN)to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2020-12-08 23:32:20.713215: I tensorflow/core/platform/profile_utils/cpu_utils.cc:104] CPU Frequency: 2499995000 Hz
2020-12-08 23:32:20.713451: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5572187ea750 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-12-08 23:32:20.713477: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-12-08 23:32:20.878776: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-12-08 23:32:20.879646: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x557218b0f200 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2020-12-08 23:32:20.879675: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Tesla T4, Compute Capability 7.5
2020-12-08 23:32:20.879895: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-12-08 23:32:20.880628: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties:
pciBusID: 0000:00:1e.0 name: Tesla T4 computeCapability: 7.5
coreClock: 1.59GHz coreCount: 40 deviceMemorySize: 14.75GiB deviceMemoryBandwidth: 298.08GiB/s
2020-12-08 23:32:20.880665: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
2020-12-08 23:32:20.880697: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcublas.so.10
2020-12-08 23:32:20.880712: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcufft.so.10
2020-12-08 23:32:20.880726: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcurand.so.10
2020-12-08 23:32:20.880744: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusolver.so.10
2020-12-08 23:32:20.880761: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusparse.so.10
2020-12-08 23:32:20.880780: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudnn.so.7
2020-12-08 23:32:20.880853: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-12-08 23:32:20.881644: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-12-08 23:32:20.882347: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0
2020-12-08 23:32:20.882390: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
2020-12-08 23:32:21.437267: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1257] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-12-08 23:32:21.437314: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1263]      0
2020-12-08 23:32:21.437328: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1276] 0:   N
2020-12-08 23:32:21.437554: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-12-08 23:32:21.438350: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-12-08 23:32:21.439124: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1402] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 12367 MB memory) -> physical GPU (device: 0, name: Tesla T4, pci bus id: 0000:00:1e.0, compute capability: 7.5)
2020-12-08 23:32:21.678399: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
2020-12-08 23:32:21.874045: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcublas.so.10
All PyTorch model weights were used when initializing TFDistilBertModel.

All the weights of TFDistilBertModel were initialized from the PyTorch model.
If your task is similar to the task the model of the ckeckpoint was trained on, you can already use TFDistilBertModel for predictions without further training.
WARNING:tensorflow:Skipping full serialization of Keras layer <tensorflow.python.keras.layers.core.Dropout object at 0x7fafd1a97fd0>, because it is not built.
WARNING:tensorflow:Skipping full serialization of Keras layer <tensorflow.python.keras.layers.core.Dropout object at 0x7fafd0056e10>, because it is not built.
WARNING:tensorflow:Skipping full serialization of Keras layer <tensorflow.python.keras.layers.core.Dropout object at 0x7fafc051ea10>, because it is not built.
WARNING:tensorflow:Skipping full serialization of Keras layer <tensorflow.python.keras.layers.core.Dropout object at 0x7fafc0535690>, because it is not built.
WARNING:tensorflow:Skipping full serialization of Keras layer <tensorflow.python.keras.layers.core.Dropout object at 0x7fafc04cb3d0>, because it is not built.
WARNING:tensorflow:Skipping full serialization of Keras layer <tensorflow.python.keras.layers.core.Dropout object at 0x7fafc04d9f50>, because it is not built.
WARNING:tensorflow:From /home/ubuntu/anaconda3/lib/python3.7/site-packages/tensorflow/python/training/tracking/tracking.py:111: Model.state_updates (from tensorflow.python.keras.engine.training) is deprecated and will be removed in a future version.
Instructions for updating:
This property should not be used in TensorFlow 2.0, as updates are applied automatically.
WARNING:tensorflow:From /home/ubuntu/anaconda3/lib/python3.7/site-packages/tensorflow/python/training/tracking/tracking.py:111: Layer.updates (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
This property should not be used in TensorFlow 2.0, as updates are applied automatically.

## Expected behavior
The TensorFlow model should have a maximum input sequence length of 128, not 5.


Templates overhaul 1

Re-opening of https://github.com/huggingface/transformers/pull/8981 after history was messed up.


New squad example

Reopening from #8924 since the rebase gave too big a diff.


fixes #8968

# What does this PR do?

One of the 3.X releases introduce output objects that replaced the previously returned tuples. This PR updates the transformers notebook to reflect that update.

Fixes #8968

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

@patrickvonplaten


[Flax] Serialization, Design changes

This PR is a proposition for some changes of the flax design. @mfuntowicz - I'm trying to get a better understanding of how to best use the flax library with Transformers' philosophy. Would be super happy about some comments from your side :-)

1) I think Flax's `from_pretrained()` should default to the flax serialization and not the PyTorch one. Flax's serialization didn't work previously and the model was loaded from PyTorch by default. This PR changes this to Flax and makes `from_pretrained()` and `save_pretrained()` work. I uploaded Bert's and Roberta's flax model weights to the model hub (I noticed that I accidently overwrote an existing Flax `bert-base-cased` - hope that was fine @mfuntowicz - it's doesn't break anything on master since PT was loaded by default)

2) Not sure why we have the `model_class` class attribute in Flax - I don't think we need it no? @mfuntowicz - It would be nice to avoid it for simplicity IMO.

3) I added a `FlaxBertPretrainedModel` class, just as it's done for PyTorch. IMO, ideally we should stay as close as possible to the design of PyTorch. Not sure at all if something like this could work:

```python
class FlaxBertForMaskedLM(FlaxBertPretrainedModel):

         def __init__(self, config, state, seed, **kwargs):
               self.bert = FlaxBertModel(config, state[self.base_model_prefix], seed, **kwargs)  # pass bert relevant

        @nn.compact
        def __call__(....):
               last_hidden_states = self.bert(hidden_states)[0]
               logits = FlaxBertLMPredictionHead(vocab_size=self.vocab_size, name="mlm", dtype=self.dtype)(last_hidden_states)
```
=> What do you think @mfuntowicz ?

Would be awesome if we could do some flax library design discussions here @mfuntowicz @LysandreJik @sgugger @thomwolf


Make `ModelOutput` pickle-able

# What does this PR do?

To be pickle-able or deep-copyable, `ModelOutput`s need to have all fields with a default. This was already the case on the TF side, just doing it on the PT side as well.

Fixes #8978


[WIP] Add Tapas (bis)

Related to pretrained model cards

This is a clean branch based on #8113 which is up-to-date with master. To do:

- [x] Make sure all tests pass (currently 44 passed, 4 skipped for `test_modeling_tapas.py`) cc @LysandreJik
- [x] `make style` & `make quality`
- [x] Investigating forward/backward pass => weird issue with fine-tuning already-finetuned WTQ checkpoint, I guess people should just not do it
- [x]  Add notebooks to show how to use:
- `tapas-base-finetuned-sqa`: https://colab.research.google.com/drive/1zMW-D2kYrpDA-cvpNJ-ctGD-tDXWebZa?usp=sharing
- `tapas-base-finetuned-tabfact`: https://colab.research.google.com/drive/1Ug6gzPFgf3J0dR-0f4spt0eyPS10dD1l?usp=sharing

Once they all pass, I'll start uploading more checkpoints to the model hub.


Tensor arrays

# What does this PR do?

This PR turns the `all_attentions` and `all_hidden_states` values into tensors instead of a tuple. This update is to properly allow the dict outputs in TF serving, because the value of each key cannot be something else than a TF tensor.


Checking output format + check raises ValueError

Just making sure we're not changing the format when we apply `function_to_apply`


Remove value error

# What does this PR do?

This PR update the behavior of the input. We should not raise an error if the name is not among the parameters but act like if there was no name, this is more elegant and less annoying.


Hi
When I evaluate the finetune_trainer.py on translation datasets like wmt16-en-cs I always get this error after calling evaluate function, I am using version 3.5.1 of transformer on 1 GPU, this issue is really blokcing me, and happens to be there for  all translation datasets I tried, could you give me some ideas on this? thanks

[libprotobuf FATAL /sentencepiece/src/../third_party/protobuf-lite/google/protobuf/repeated_field.h:1505] CHECK failed: (index) >= (0):
terminate called after throwing an instance of 'google::protobuf::FatalException'
  what():  CHECK failed: (index) >= (0):
Aborted


BertConfig.id2label use list instead of "int: string" dict

In https://github.com/huggingface/transformers/blob/master/src/transformers/configuration_utils.py#L262, use list instead of "int: string" dict maybe is better.
When we use easydict to replace dict, there will be bugs in updating the `.to_dict()` to other easydict object. Because int object can not be the key in easydict.
Solve method:
use ```["LABEL_{}".format(i) for i in range(num_labels)]``` to replace ```self.id2label = {i: "LABEL_{}".format(i) for i in range(num_labels)}```.


[Example] Fix the argument name mismatch in the distillation example

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

This past [PR](https://github.com/huggingface/transformers/pull/6315) replaced the argument name `n_gpu` of the distillation example with `gpus`. This causes a crash in running the example since the rest of the example code still uses the old argument name (`n_gpu`).

This PR solves the issue and get the distillation example to run.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSMT: @stas00
 -->


Model templates overhaul

This is the first PR to make the model templates better. It improves the templates themselves, as well as the testing tools around them:

- Re-instantiates the tests in the CI, this time as a separate test.
- Respects the library style, and tests it. These tests ensure that the templates have not diverged from the code base, especially due to the `# Copied from ...`.
- Implements a decoder model, with support for cross attentions, to be used in the encoder-decoder framework.
- Implements the same decoder model in TensorFlow
- Implements multiple types of position embeddings, similarly to BERT.
- Tests every new feature.
- Tokenizer separation between slow and fast
- Soft dependency on `cookiecutter`
- Adds easily tweakable integration tests for both PyTorch and TensorFlow

Things left for overhaul 2 & 3:

- General tokenizer improvements, I'm not happy with their current state and it's not tested. I find it surprisingly difficult to have a template for a tokenizer that is general enough, so I'm probably going to try to cover as much possible use-cases as possible
- Encoder-decoder w/ @patrickvonplaten

Things to improve for this overhaul (1):

- Probably speeding up the model templates test. It's running on github actions right now, and it's quite slow (8 minutes) even though the downloads are cached. Possible options are
    - use CircleCI instead
    - Cache the whole environment
    - Probably others, thinking about it
- The test runs on each commit + when the branch is opened. That's unnecessary.


[wip] [ci] doc-job-skip take #4 dry-run

This is take 4 on attempting to find a reliable way to get a list of modified files of this PR.

Spent the whole day trying many different ideas, none worked. github API for
`https://api.github.com/repos/${CIRCLE_USERNAME}/${CIRCLE_REPO_NAME}/pulls/${CIRCLE_PR_NUMER}"`
is broken. It gives a bogus `base.sha` at times, e.g. when someone force-pushes into master and then you end up with a base.sha which has nothing to do with the fork. on github website everything is valid, but github API gives bogus info.

So after many attempts I give up on trying to get a reliable way via the SHA information provided via github or circleCI.

The only solution that seems to work is to replicate user's original branch on their fork. Cost is about 3secs.

To do that one has to clone user's repo, switch to their branch and find the branching point identical to what `make fixup` does. This is what the latest incarnation of this PR does.

This PR doesn't enable the skipping yet, just reporting what it would have done and dumps the list of modified files so that we could check if we get some edge cases again which this incarnation doesn't cover.

Please have a look and see if it looks safe to merge and then monitor for a while and if all seems in order then we can enable skipping.

This PR will not be able to handle PRs originating from github direct file edit as it can be seen from https://github.com/huggingface/transformers/pull/8997 as CircleCI fails to pass PR number to the job in this situation :( The whole skip check is skipped in that case the job continues normally - we just don't get the saving on direct doc PRs. I'm still trying to see if CircleCI should be providing this data, since according to https://developer.github.com/webhooks/event-payloads/#pull_request this hook should be sending PR number to circleCI.

When I had the idea first little did I know that this trivial one-liner on user-side (we use it in `make fixup`) will turn out to be such a complicated and unreliable thing on CI.

@LysandreJik


[training] SAVE_STATE_WARNING was removed in pytorch

`SAVE_STATE_WARNING` has been removed from pytorch 3 days ago: pytorch/pytorch#46813

I had to add redundant ()'s to avoid a terrible auto-formatter outcome.

Fixes: #8232

@sgugger, @LysandreJik


Deepcopy and pickling fails for modeling_outputs

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.0.0
- Python version: 3.8
- PyTorch version (GPU?):   1.6.0
- Using GPU in script?: N/A

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten @TevenLeScao
 Blenderbot: @patrickvonplaten
 Bart: @patrickvonplaten
 Marian: @patrickvonplaten
 Pegasus: @patrickvonplaten
 mBART: @patrickvonplaten
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 RAG: @patrickvonplaten, @lhoestq
 FSMT: @stas00
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

## To reproduce

Steps to reproduce the behavior:

```
>>> from transformers.modeling_outputs import BaseModelOutput
>>> import torch
>>> import copy
>>> x = BaseModelOutput(last_hidden_state=torch.ones(1,))
>>> z = copy.deepcopy(x)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/lib/python3.8/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/opt/lib/python3.8/copy.py", line 263, in _reconstruct
    y = func(*args)
TypeError: __init__() missing 1 required positional argument: 'last_hidden_state'
>>> import pickle
>>> obj = pickle.dumps(x)
>>> obj_loaded = pickle.loads(obj)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: __init__() missing 1 required positional argument: 'last_hidden_state'
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
No failures when using deepcopy or pickle.dumps/pickle.loads


BertForMaskedLM train

I have a question
When training using BertForMaskedLM, is the train data as below correct?
 - token2idx
```
<pad> : 0, <mask>: 1, <cls>:2, <sep>:3
```
 - max len : 8
 - input token
```
 <cls> hello i <mask> cats <sep>
```
 - input ids
```
 [2, 34,45,1,56,3,0,0]
```
 - attention_mask
```
 [1,1,1,1,1,1,0,0]
```
 - labels
```
 [-100,-100,-100,64,-100,-100,-100,-100]
```

I wonder if I should also assign -100 to labels for padding token.


Check table as independent script

Separated the table check from `check_copies.py` as for the template I need to manage the copies without managing the table.


Update quicktour docs to showcase the use of truncation

# What does this PR do?
Currently, running the tokenizer batch example on https://huggingface.co/transformers/quicktour.html gives an error

```
Asking to truncate to max_length but no maximum length is provided and the model has no predefined maximum length. Default to no truncation.
```

This PR fixes the above by passing the `max_length` param explicitly (instead of depending on it having a default, which might not be the case for all models).

The fix also adds clarity to the statement in the docs above this example
> If your goal is to send them through your model as a batch, you probably want to pad them all to the same length, truncate them to the maximum length the model can accept and get tensors back

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?
@sgugger


Add option to only check copies


Small fix to the run clm script

# What does this PR do?

@LysandreJik pointed out that the scripts will fail with a cryptic error if the tokenizer `model_max_len` is huge and no `block_size` is set. This PR fixes this by clipping the `block_size` to 1024 (when no value is passed).


Removed unused `encoder_hidden_states` and `encoder_attention_mask`

# What does this PR do?

This PR removes unused `encoder_hidden_states` and `encoder_attention_mask` from MobileBert forward methods. These are use for decoder models, but MobileBert does not include a cross-attention mechanism.

Fixes https://github.com/huggingface/transformers/issues/8969

## Who can review?
 albert, bert, XLM: @LysandreJik