[Bart doc] Fix outdated statement

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #9298 . Bart docs should be slightly updated. Thank @forest1988 !

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSMT: @stas00
 -->


`transformers.models.bart.modeling_bart._prepare_bart_decoder_inputs` seems to be renamed but remains in the document

## Environment info

- `transformers` version: 4.1.1
- Platform: Linux-4.15.0-123-generic-x86_64-with-glibc2.10
- Python version: 3.8.3
- PyTorch version (GPU?): 1.7.0 (True)
- Tensorflow version (GPU?): 2.3.1 (True)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help
 Bart: @patrickvonplaten

## Information

Model I am using (Bert, XLNet ...): Bart

The problem arises in the [model_doc/bart.rst](https://github.com/huggingface/transformers/blob/v4.1.1/docs/source/model_doc/bart.rst#implementation-notes)

## To reproduce

In our code using transformers v3.4.0, we have used :
```
from transformers.modeling_bart import _prepare_bart_decoder_inputs
```
I tried to rerewrite it as:
```
try:  # transformers >= v4
    from transformers.models.bart.modeling_bart import _prepare_bart_decoder_inputs
except ModuleNotFoundError:  # transformers == v3.4.0
    from transformers.modeling_bart import _prepare_bart_decoder_inputs
```
but It seems Bart (modeling_bart) in v4.1.1 doesn't have `_prepare_bart_decoder_inputs` in its implementation.
However, [model_doc/bart.rst](https://github.com/huggingface/transformers/blob/v4.1.1/docs/source/model_doc/bart.rst#implementation-notes) says

> The forward pass of :class:`~transformers.BartModel` will create decoder inputs (using the helper function :func:`transformers.models.bart.modeling_bart._prepare_bart_decoder_inputs`) if they are not passed. This is different than some other modeling APIs.

I think maybe the function is renamed in the refactoring of Bart #8900.
I welcome this refactoring as I would love to take advantage of Bart (and other Seq2SeqLMs), but I am wondering how I can fix the old code to get the best performance out of the refactored code.
Do you have any document about how to fix the old code to work well with the new version of the Bart?

## Expected behavior

Maybe model_doc/bart.rst needs to be updated.
I'm sorry if there is already an appropriate documentation.


fix typo in modeling_encoder_decoder.py

Fixed typo.

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes a typo

## Before submitting
- [ x ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSMT: @stas00
 -->


[bert_generation] enable cache by default

# What does this PR do?

- add `use_cache`  to `BertGenerationConfig` with default to `True`
- in `BertGenerationEncoder` if `use_cache` is `None` (this is the default behaviour) then set it using the `config`.

This will enable caching by default in inference for `BertGenerationEncoder`


Good Second Issue: T5 FP16 in Pytorch

Good Second Issue

Issues that are more difficult to do than "Good First" issues - give it a try if you want!

# 🚀 Feature request

This "Good second issue" should revisit some of the problems we were having with FP16 for `T5ForConditionalGeneration`: https://github.com/huggingface/transformers/issues/4586 and help to make T5 compatible with fp16.

**_Requirements:_**

- use transformers master
- use newest pytorch version
- have access to GPU

**_Context:_**

To better explain the context, let's define the three different pre-trained T5 models types we have:

- **T5v1** (original T5): => this corresponds to all those checkpoints: `t5-small`, `t5-base`, `t5-large`, `t5-3b`, `t5-11b`
- **T5v1_1** (improved T5): => this corresponds to all those checkpoints: `google/t5-v1_1-small`, `google/t5-v1_1-base`, `google/t5-v1_1-large`, `google/t5-v1_1-xl`, `google/t5-v1_1-xxl`. **T5v1_1** has a slightly different architecture than **T5v1**. More info on differences can be found here: https://github.com/huggingface/transformers/issues/6285
- **MT5** (multi-lingual T5): => this model is identical in architecture to **T5v1_1** but has different pre-trained weights and a much larger word embedding matrix.

As shown in this issue https://github.com/huggingface/transformers/issues/4586 , training **T5v1** in fp16 mode led in the past to numerical overflow in the `T5LayerFF` forward pass: https://github.com/huggingface/transformers/blob/6189ae99603bd5dc14c5631f1b4562f78e24d575/src/transformers/models/t5/modeling_t5.py#L279.

At the time of this issue: https://github.com/huggingface/transformers/issues/4586, **T5v1** was added with a small bug that led to slightly wrong outputs that was only fixed by this PR: https://github.com/huggingface/transformers/pull/8518.

Also, now there are new T5 checkpoints, notably the **T5v1_1** and **MT5** checkpoints, where it would be very interesting to see whether fp16 can work with those.

**_Feature Request_**

So for this feature request, we should two scenarios:

1) Inference:

For each T5 model type we should test when the models break during inference. This can be as easy as testing the following script for a bunch of different checkpoints on different `input_str`:

```python
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
import torch

checkpoint = "t5-small"  # "google/mt5-small", "google/t5-v1_1-small"

model = AutoModelForSeq2SeqLM.from_pretrained(checkpoint).to('cuda')
tokenizer = AutoTokenizer.from_pretrained(checkpoint)

input_str = "Hello there. This is the input."  # here it would be better to test much larger inputs

input_ids = tokenizer(input_str, return_tensors="pt").input_ids.to('cuda')

# FP32
output_fp32 = model.generate(input_ids)

# FP16
model.half()
output_fp16 = model.generate(input_ids)

if output_fp32.tolist() == output_fp16.tolist():
    print("SUCCESS: Output is equal!")
else:
    print("Output is different!")
    print("FP32", output_fp32)
    print("FP16", output_fp16)
```

2) Training (the more interesting part):

This is probably more important and will require more time / skill. In order to check how T5 does in FP16 training, I'd recommend to use the newly added `Seq2SeqTrainer`: https://github.com/huggingface/transformers/blob/6189ae99603bd5dc14c5631f1b4562f78e24d575/src/transformers/trainer_seq2seq.py#L38. I would recommend to train on a summarization task, such as CNN/Dailymail. One could closely follow, this notebook: https://colab.research.google.com/drive/1Ekd5pUeCX7VOrMx94_czTkwNtLN32Uyu?usp=sharing, but replacing Bert2Bert with the different T5 models. Ideally different "fp16 backends" should be tested: https://github.com/huggingface/transformers/blob/6189ae99603bd5dc14c5631f1b4562f78e24d575/src/transformers/training_args.py#L216 and one should try to see whether hacks as proposed in https://github.com/huggingface/transformers/issues/4586#issuecomment-748336815 can solve the problem. It would be very interesting to see whether the error happens only for **T5v1** or also for **T5v1_1** and **MT5** and it what point. For each type it would be great to test for "small", "base" and if possible even "large". Ideally, one should first create a short summarization fine-tuning script (happy to help here) and then run a bunch of different experiments with different fp16 backends and different models.

**_Possible Outcome_**

The results of those experiments should be documented here or even better on https://discuss.huggingface.co/. Ideally, a solution to the problem is found and one could publish a nice blog post explaining how to effectively train T5.

## Motivation

T5 is one of the most widely used models of Transformers at the moment so that more results to this issue would be extremely useful for the community. In addition, this issue can be a great opportunity to learn more about the limits of fp16 and why some models still do require full fp32 support (or at least until bfloat16 is better supported in torch). This is not an easy issue to tackle, but an extremely important one.

## Your contribution

I'm happy to help along the way, starting with making a nice T5 summarization training pipeline that lets one easily test on different models, and fp16 backends.


Fix TF input for np.ndarray

# What does this PR do?

This PR allows `np.ndarray` datatype as input of the models.

# Fixes
#9248


Update tokenization_utils_base.py

Missing "s" typo in the error message, which is an invalid argument.


Fix TF Flaubert

# What does this PR do?

This PR fixes Flaubert to make able to be executed in graph mode.


Fix TF CTRL

# What does this PR do?

This PR fixes the model inputs in TF CTRL.


Problem converting slow tokenizer to fast: token out of vocabulary

Core: Tokenization

Internals of the library; Tokenization.

When I try to use a [Dutch RoBERTa model](https://huggingface.co/pdelobelle/robbert-v2-dutch-base/tree/main#how-to-use) as suggested, the library tries to convert the old (slow) tokenizer to the fast one. However, this leads to issues. (I can just keep the slow one, but I need to use the offset and word_ids functionality which is only available in the fast tokenizers.)

```python
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("pdelobelle/robbert-v2-dutch-base")
```

Error trace:

```
 File "C:/dev/python/jasper-tok2vec/main.py", line 10, in main
    tokenizer = AutoTokenizer.from_pretrained("pdelobelle/robbert-v2-dutch-base")
  File "C:\Users\bramv\.virtualenvs\jasper-tok2vec-4kc9ajCV\lib\site-packages\transformers\models\auto\tokenization_auto.py", line 378, in from_pretrained
    return tokenizer_class_fast.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
  File "C:\Users\bramv\.virtualenvs\jasper-tok2vec-4kc9ajCV\lib\site-packages\transformers\tokenization_utils_base.py", line 1804, in from_pretrained
    return cls._from_pretrained(
  File "C:\Users\bramv\.virtualenvs\jasper-tok2vec-4kc9ajCV\lib\site-packages\transformers\tokenization_utils_base.py", line 1877, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
  File "C:\Users\bramv\.virtualenvs\jasper-tok2vec-4kc9ajCV\lib\site-packages\transformers\models\roberta\tokenization_roberta_fast.py", line 160, in __init__
    super().__init__(
  File "C:\Users\bramv\.virtualenvs\jasper-tok2vec-4kc9ajCV\lib\site-packages\transformers\models\gpt2\tokenization_gpt2_fast.py", line 133, in __init__
    super().__init__(
  File "C:\Users\bramv\.virtualenvs\jasper-tok2vec-4kc9ajCV\lib\site-packages\transformers\tokenization_utils_fast.py", line 89, in __init__
    fast_tokenizer = convert_slow_tokenizer(slow_tokenizer)
  File "C:\Users\bramv\.virtualenvs\jasper-tok2vec-4kc9ajCV\lib\site-packages\transformers\convert_slow_tokenizer.py", line 642, in convert_slow_tokenizer
    return converter_class(transformer_tokenizer).converted()
  File "C:\Users\bramv\.virtualenvs\jasper-tok2vec-4kc9ajCV\lib\site-packages\transformers\convert_slow_tokenizer.py", line 262, in converted
    BPE(
Exception: Error while initializing BPE: Token `Ċ` out of vocabulary
```

### Environment

- `transformers` version: 4.1.1
- Platform: Windows-10-10.0.19041-SP0
- Python version: 3.8.2
- PyTorch version (GPU?): 1.7.1+cu110 (True)
- Tensorflow version (GPU?): not installed (NA)

### Who can help

@mfuntowicz


Fix typo in file_utils.py

# What does this PR do?

Fix typo of `add_code_sample_docstrings` in `file_utils.py`

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSMT: @stas00
 -->


[doc] How To Request Support document stab

As discussed it'd be great to have a clear document with guidelines at how to create great Issues that are easy to understand, reproduce and resolve. I wrote this stab of a document to get things started.

Please feel free to edit it further to your satisfaction. I'm not attached to any parts it, just brain dumping what was coming based on my experience. So feel free to add remove, reformat, etc. Best commit your edits directly into this branch or second best via suggestions. I don't need to be the middleman.

Please tag others that you think may want to contribute to this document.

If this is successful, perhaps a similar document will be needed for PRs - or perhaps down the road it will be a single document as there is a lot of overlap between writing a good Issue and a good PR. But let's start simple.

@sgugger, @patrickvonplaten, @LysandreJik


SummarizationModule, Trainer and BertPreTrainedModel

Hi,

I wonder what's the relationship between SummarizationModule (SummarizationDistiller), Trainer and BertPreTrainedModel? I want to reimplement the distillation.py of seq2seq example and run it on the glue dataset. But I'm confused by the relationship between these three classes.

The model parameter of Trainer is BertPreTrainedModel and the Trainer train the model. But in the SummarizationDistiller there isn't a training function. I didn't find the training process in the distillation.py and finetune.py. Should I pass SummarizationDistiller object to Trainer to train the model? Or how should I train my custom SummarizationDistiller.


Why Bert-chinese use do_lower_case=False?

Some Chinese Text has some English words, for example: "Apples是苹果的复数形式。". I have questions about how to tokenize the text:

1.  why Chinese Bert Case sensitive, but I can't find even 'A' in vocab.txt
2.  Because English words in Chinese vocab.txt is few, should I use wordpiece tokenizer as default, like "['apple', '##s', '是', '苹', ...]"or split to char to tokenize, like "['a', 'p', 'p', 'l', 'e', 's', '是', '苹', ...]"?


TFRobertaModel warning  - `output_attentions`, `output_hidden_states` and `use_cache` cannot be updated

I'm using:
google colab
transformers  4.1.1
tensorflow 2.4.0 (with gpu)
model TFRobertaModel

I keep getting a warning when calling TFRobertaModel. For example:
```
tokenizer = RobertaTokenizer.from_pretrained('roberta-base')
config = RobertaConfig.from_pretrained('roberta-base')
roberta_layer = TFRobertaModel(config)

max_seq_len = 64
ids = tf.keras.layers.Input((max_seq_len,), dtype=tf.int32)
att = tf.keras.layers.Input((max_seq_len,), dtype=tf.int32)
tok = tf.keras.layers.Input((max_seq_len,), dtype=tf.int32)

roberta_inputs = [ids, att, tok]

sequence_output = roberta_layer(ids,attention_mask=att,token_type_ids=tok) # this produces the message
```
Produces the following message:

The parameters `output_attentions`, `output_hidden_states` and `use_cache` cannot be updated when calling a model. They have to be set to True/False in the config object (i.e.: `config=XConfig.from_pretrained('name', output_attentions=True)`).
The parameter `return_dict` cannot be set in graph mode and will always be set to `True`.

I tried to set the variables in the config object but there is no change in the message:
```
tokenizer = RobertaTokenizer.from_pretrained('roberta-base')
config = RobertaConfig.from_pretrained('roberta-base',output_attentions=False,output_hidden_states=False,return_dict =True)
roberta_layer = TFRobertaModel(config)
```

@jplu, @LysandreJik


[Templates] Adapt Bert

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Adapt Bert-like templates following https://github.com/huggingface/transformers/pull/9183. This should fix the templates test on master.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSMT: @stas00
 -->


Fix TF DPR

# What does this PR do?

This PR rework the DPR architecture for its TF version. This rework allows DPR models to be saved as proper saved model.


Adapt to new name of `label_smoothing_factor` training arg

# What does this PR do?

This PR changes `label_smoothing` to its new name `label_smoothing_factor` in the tests and scripts that use it.

Pinging @stas00 for information but will merge when CI is passing.


tapas utils

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSMT: @stas00
 -->


issue with evaluation of seq2seq_trainer.py on multiple gpus

## Environment info
- `transformers` version: 3.5.1
- Platform: Linux
- Python version: 3.7
- PyTorch version (GPU?): 1.7
- Tensorflow version (GPU?): -
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: yes

### Who can help

 Trainer: @sgugger

## Information and commands to reproduce

I am running seq2seq trainer model on multiple GPUS. The problem arises during evaluation, here are my modified seq2seq_trainer.py codes and how to reproduce the error:

```
git clone git@github.com:rabeehk/seq2seq.git
python setup.py develop
cd seq2seq
python -m torch.distributed.launch --nproc_per_node=4 --master_port=9918  finetune_t5_trainer.py temp_configs/mp-lr-3e-02-r-8-l-true.json
```

here is the error I get during the evaluation:

`file outputs/mixture1/meta-adapters-task-projector-new_sampler-num-gpus-4/mp-lr-3e-02-r-8-l-true/config.json not found
`

## Bug description
I figured out in finetune_trainer.py if a user load config and trained model again during the evaluation, this operation results in processes not being able to find the config file, showing this is not multi-gpu safe, I realized if I add `trainer.is_world_process_zero:` beforehand and reload model and config in evaluation part, this resolves the issue, could you please comment on this and assist with this issue? I think this is a bug if the user cannot load the model in the evaluation part in multiple-gpus properly. Thanks.

## Full error stack.
 ```
I 1222 15:38:32.129721  2784 main shadow.py:122] > Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/transformers/configuration_utils.py", line 388, in get_config_dict
file outputs/mixture1/meta-adapters-task-projector-new_sampler-num-gpus-4/mp-lr-3e-02-r-8-l-true/config.json not found
I 1222 15:38:32.129757  2784 main shadow.py:122] > Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/transformers/configuration_utils.py", line 388, in get_config_dict
                                                   Traceback (most recent call last):
I 1222 15:38:32.129800  2784 main shadow.py:122] >
I 1222 15:38:32.130028  2784 main shadow.py:122] >     local_files_only=local_files_only,
I 1222 15:38:32.130069  2784 main shadow.py:122] >   File "/usr/local/lib/python3.6/dist-packages/transformers/file_utils.py", line 962, in cached_path
I 1222 15:38:32.130314  2784 main shadow.py:122] >     raise EnvironmentError("file {} not found".format(url_or_filename))
I 1222 15:38:32.130367  2784 main shadow.py:122] > OSError: file outputs/mixture1/meta-adapters-task-projector-new_sampler-num-gpus-4/mp-lr-3e-02-r-8-l-true/config.json not found

I 1222 15:38:32.130448  2784 main shadow.py:122] > During handling of the above exception, another exception occurred:
I 1222 15:38:32.130505  2784 main shadow.py:122] >
I 1222 15:38:32.130557  2784 main shadow.py:122] > OSError: file outputs/mixture1/meta-adapters-task-projector-new_sampler-num-gpus-4/mp-lr-3e-02-r-8-l-true/config.json not found
I 1222 15:38:32.130720  2784 main shadow.py:122] > Traceback (most recent call last):
OSError: file outputs/mixture1/meta-adapters-task-projector-new_sampler-num-gpus-4/mp-lr-3e-02-r-8-l-true/config.json not found
I 1222 15:38:32.130823  2784 main shadow.py:122] >   File "./finetune_t5_trainer.py", line 328, in <module>
I 1222 15:38:32.130858  2784 main shadow.py:122] >
I 1222 15:38:32.130903  2784 main shadow.py:122] > During handling of the above exception, another exception occurred:
I 1222 15:38:32.130938  2784 main shadow.py:122] >
I 1222 15:38:32.130976  2784 main shadow.py:122] > Traceback (most recent call last):
  File "./finetune_t5_trainer.py", line 207, in main
    main()
  File "/usr/local/lib/python3.6/dist-packages/transformers/configuration_utils.py", line 347, in from_pretrained
    cache_dir=model_args.cache_dir)
  File "/usr/local/lib/python3.6/dist-packages/transformers/configuration_utils.py", line 347, in from_pretrained
100%|??????????| 2100/2100 [36:51<00:00,  1.05s/it]
I 1222 15:38:32.131012  2784 main shadow.py:122] >     cache_dir=model_args.cache_dir)
I 1222 15:38:32.131047  2784 main shadow.py:122] >   File "/usr/local/lib/python3.6/dist-packages/transformers/configuration_utils.py", line 347, in from_pretrained
I 1222 15:38:32.131081  2784 main shadow.py:122] >     config_dict, kwargs = cls.get_config_dict(pretrained_model_name_or_path, **kwargs)
I 1222 15:38:32.131301  2784 main shadow.py:122] >   File "/usr/local/lib/python3.6/dist-packages/transformers/configuration_utils.py", line 400, in get_config_dict
I 1222 15:38:32.131337  2784 main shadow.py:122] >     raise EnvironmentError(msg)
I 1222 15:38:32.131372  2784 main shadow.py:122] > OSError: Can't load config for 'outputs/mixture1/meta-adapters-task-projector-new_sampler-num-gpus-4/mp-lr-3e-02-r-8-l-true'. Make sure that:
I 1222 15:38:32.131432  2784 main shadow.py:122] > - 'outputs/mixture1/meta-adapters-task-projector-new_sampler-num-gpus-4/mp-lr-3e-02-r-8-l-true' is a correct model identifier listed on 'https://huggingface.co/models'
I 1222 15:38:32.131615  2784 main shadow.py:122] >     raise EnvironmentError(msg)
I 1222 15:38:32.131670  2784 main shadow.py:122] > OSError: Can't load config for 'outputs/mixture1/meta-adapters-task-projector-new_sampler-num-gpus-4/mp-lr-3e-02-r-8-l-true'. Make sure that:
```


[Refactor] Splitting pipelines.py into its own module.

# What does this PR do?

Moves various pipelines into their own files.
`pipelines.py` was 3k+ lines of code with feels a bit too much. To go along with the `models` split
into various files, splitting into a cleaner module with subfiles was proposed by @thomwolf
(Can't find the discussion).

There's at least 3 parts than need to be explicited.

- The *glue code* that makes `pipeline` so powerful (loading up the right task for the right model with right task, basically completing all the wholes based on the call signature). That's `__init__.py`
- The main class `Pipeline` that mutualises a lot of the boilerplate. Thats `base.py`.
- All the specialized classes `NerPipeline`, `FeatureExtractionPipeline`, ... that's the other files.

All the tests remains strictly the same to ensure there's no breaking change in there.
The main issue with that PR is that now its a bit harder to check the various code flow.
Some is in the base, some is in a specialized file.

`TranslationPipeline`, `SummarizationPipeline` and `Text2TextPipeline` are in the same file `text2text_generation.py` as they seem to share quit a bit of code, maybe some cleanup and better code sharing is imaginable in a follow-up PR (https://github.com/Narsil/transformers/pull/1), at the very least modifications in one, probably should be duplicated in the others as they use the same underlying models (Seq2Seq).

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

## Who can review?

@LysandreJik
@thomwolf

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSMT: @stas00
 -->


LED

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Adds LongformerEncoderDecoder (LED) from @ibeltagy - see: https://github.com/allenai/longformer#longformer

Todo:

- [x] **Important**: position embeddings have to be cut to correctly convert original Bart-ilke checkpoints to LED. The reason is that Bart uses a position embedding hack because of which the embedding idx 0 and 1 are never used resulting in an embedding matrix that has a length of 1026 instead of 1024, see: https://github.com/huggingface/transformers/blob/88ef8893cd649cc2b4adb9885aba88c750118cff/src/transformers/models/bart/modeling_bart.py#L131. All LED checkpoints are hence cut to remove this hack in LED:
```python
model = LEDForConditionalGeneration.from_pretrained("./led-base-16384")
model.model.encoder.embed_positions.weight = torch.nn.Parameter(model.model.encoder.embed_positions.weight[2:, :])
model.model.decoder.embed_positions.weight = torch.nn.Parameter(model.model.decoder.embed_positions.weight[2:, :])
model.save_pretrained("./led-base-16384")
```

- [x] Make Pytorch integration tests pass. See `LEDIntegrationTests` in `tests/test_modeling_led.py`.
- [x] Add gradient_checkpointing
- [x] Make common tests work
- [x] Add convenient padding function so that input can be of whatever size and add global_attn logic to mask
- [x] Automatically create attention_mask in encoder if not provided
- [x] Finish PT version
- [x] Make TF version work
- [x] Add tips in docs for LED
- [x] Eval notebook: https://colab.research.google.com/drive/12INTTR6n64TzS4RrXZxMSXfrOd9Xzamo?usp=sharing
- [x] Nice to have: Fine-tune notebook: https://colab.research.google.com/drive/12LjJazBl7Gam0XBPy_y0CTOJZeZ34c2v?usp=sharing
- [ ] Make nice model cards

## TODO after PR is merged:
- [ ] Correctly add `# Copied from ....` statements from Bart and Longformer (this probably requires the Bart refactor to be merged before)
- [ ] Open issue regarding problems with TF save_model test
- [ ] Correct templates: delete unnecessary test for tf bart; add gradient checkpointing by default in PT

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSMT: @stas00
 -->


[Seq2Seq Templates] Fix check_repo.py templates file


Vision Transformer

# 🌟 New model addition

## Model description

This issue is adding the Vision Transformer model described in the [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://arxiv.org/abs/2010.11929) paper. If you have any feedback and/or further ideas for the implementation please don't hesitate to mention them.

## Open source status

* [ x] the model implementation is available:

[The official github repo](https://github.com/google-research/vision_transformer) provides the implementation in Jax/Flax.

* [x ] the model weights are available:

See the github repo above.

* [ x] who are the authors: (mention them, if possible by @gh-username)

Google Research, Brain Team


Disable progress bar for Trainer

I am referencing a similar code compared to [run_glue.py]
(https://github.com/huggingface/transformers/blob/master/examples/text-classification/run_glue.py).

I am running a preprocessing function that does the tokenization of text as well as `trainer.predict` on a pandas dataframe. How do I disable the progress bar from showing the progress made on each row of the dataframe? I thought it would work by disabling logging as well as `tqdm`, but it is not the case here. #3050


Loss printed by tensorflow fit() differs from loss using custom loop for RoBERTa

## Environment info
- `transformers` version: 3.3.1
- Platform: Windows 10
- Python version: 3.6
- PyTorch version (GPU?): N/A
- Tensorflow version (GPU?): 2.3.0
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
@jplu, @LysandreJik

## Information

Model I am using (Bert, XLNet ...): RoBERTa

The problem arises when using:
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Run the following script
2. The printed out losses are different

```
import tensorflow as tf
from transformers import RobertaConfig, TFRobertaMainLayer

# 1. Create a class to be able to use fit()

class Transformer(tf.keras.Model):
  def __init__(self):
    super(Transformer, self).__init__()

    config = RobertaConfig(
        vocab_size=100,
        hidden_size=128,
        intermediate_size=128,
        max_position_embeddings=514,
        num_attention_heads=8,
        num_hidden_layers=6,
        type_vocab_size=1,
    )
    self.encoder = TFRobertaMainLayer(config)

  def call(self, inp, training=False):
    return self.encoder(inp)[0]

model = Transformer()

# 2. Calculating loss manually for dummy input

loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
x = tf.constant([[1, 0]])
y_true = tf.constant([[1, 0]])
y_pred = model((x, x))
loss = loss_fn(y_true, y_pred)
print(loss) #                              printing 4.8093767

# 3. Run fit()

model.compile(loss=loss_fn)
model.fit((x, x), y_true) #                printing 4.7854

```
## Expected behavior

The losses should be equal.


Fix param error

# What does this PR do?

Fixes error
```
TypeError: forward() got an unexpected keyword argument 'token_type_ids'
```

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSMT: @stas00
 -->


Fix gpt2 document

# What does this PR do?

Fixes gpt2 document error.
```
AttributeError: 'GPT2DoubleHeadsModelOutput' object has no attribute 'lm_logits'
```

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


allow integer device for BatchEncoding

# What does this PR do?

Fixes #9244

I'm not fully aware of the details behind the Apex guard, in the method, so maybe this is not the solution.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

 tokenizers: @mfuntowicz


# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->
how can I change the AlbertModel's vocab
Thanks

## Motivation

I noticed that I can change the Bert's vocab by change the vocab.txt. But when I use the Albert's API, the document suggests:
tokenizer = AlbertTokenizer.from_pretrained('albert-base-v2'). So how can I change that.
Thanks

## Your contribution


Output probability from model.generate

# 🚀 Feature request

Do we have the option to output the probability of the generated sequence from model.generate function? It will be super useful for evaluating the confidence score of the generated sequence. Thanks so much!


Unable to load LayoutLM from pretrained

## Environment info

- `transformers` version: 3.3.0
- Platform: Linux-4.15.0-76-generic-x86_64-with-glibc2.10
- Python version: 3.8.2
- PyTorch version (GPU?): 1.7.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: NA
- Using distributed or parallel set-up in script?: NA

### Who can help
@sgugger
@LysandreJik

## Information

When I try to load a LayoutLM model with the following script I hit an error.

```
from transformers import LayoutLMForTokenClassification
model = LayoutLMForTokenClassification.from_pretrained('microsoft/layoutlm-base-uncased', from_tf=True)
```
```
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
~/miniconda3/envs/ML38/lib/python3.8/site-packages/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    905                 if resolved_archive_file is None:
--> 906                     raise EnvironmentError
    907             except EnvironmentError:

OSError:

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
<ipython-input-11-9f82aa1a161f> in <module>
----> 1 model = LayoutLMForTokenClassification.from_pretrained('microsoft/layoutlm-base-uncased', from_tf=True)

~/miniconda3/envs/ML38/lib/python3.8/site-packages/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    911                     f"- or '{pretrained_model_name_or_path}' is the correct path to a directory containing a file named one of {WEIGHTS_NAME}, {TF2_WEIGHTS_NAME}, {TF_WEIGHTS_NAME}.\n\n"
    912                 )
--> 913                 raise EnvironmentError(msg)
    914
    915             if resolved_archive_file == archive_file:

OSError: Can't load weights for 'microsoft/layoutlm-base-uncased'. Make sure that:

- 'microsoft/layoutlm-base-uncased' is a correct model identifier listed on 'https://huggingface.co/models'

- or 'microsoft/layoutlm-base-uncased' is the correct path to a directory containing a file named one of pytorch_model.bin, tf_model.h5, model.ckpt.
```


[hf args] shouldn't match partial arg names

For `--label_smoothing_factor` I can pass `--label_smoothing` and it still works - which is a bug, as it should do a full match and not a substring. This is with master.

context: finetune_trainer just switched from `--label_smoothing` to `--label_smoothing_factor` (different functionality) and we were puzzling over why  `--label_smoothing`  still worked.

this is definitely not urgent

@sgugger


Minor documentation revisions from copyediting

# What does this PR do?

Minor changes to the documentation to correct typos and improve readability. I noticed these typos while reading through the docs to familiarize myself with the library for a project, and thought it would be nice to make a PR for them 😊

I've already tested building the docs from these changes, and all changes seem to have taken effect properly 👍

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests), Pull Request section?

## Who can review?

@sgugger would you mind reviewing this PR? I'm happy to make any changes (or remove any changes) you want 🙂


[finetune_trainer]  max length cl args redesign

Splitting of from https://github.com/huggingface/transformers/pull/9241, it's been proposed to refactor the following 4 cl args of `finetune_trainer.py`:

1.   `--max_source_length`
2.   `--max_target_length`
3.   `--val_max_target_length`
4.  `--test_max_target_length`

https://github.com/huggingface/transformers/blob/f38c4ad302dd34faef1137b13e9636f4408b0462/examples/seq2seq/finetune_trainer.py#L82-L110

There are multiple comments wrt this in https://github.com/huggingface/transformers/pull/9241, especially towards the end of it

Let's redesign it here and then do a single breaking change to this probably in the new year.

To summarize the main suggestions so far were:

1. to perhaps remove `--max_source_length` - but we need use cases to see whether this is safe to do
2. collapse cl args 2-4 into a single `max_length` arg to match `generate`'s API.

@sgugger, @patrickvonplaten, @patil-suraj


compute_metrics in the trainer does not seem to be extensible

Hi,

This is more feature request, looking into compute_metrics function defined below:
https://github.com/huggingface/transformers/blob/c89bdfbe720bc8f41c7dc6db5473a2cb0955f224/src/transformers/trainer.py#L204

to me this looks like the design does not allow user easy modification for different applications or I am missing something, please find the explanations below:

Lets assume the user has multiple tasks like in T5 and each task needs multiple different evaluation metrics, which needs to be generated on the fly, then since this function does not accept any more arguments than `EvalPrediction`, this would not allow user to pass further parameters to generate the final evaluation metric on the fly, I appreciate modifying the design to allow easy modifications.

thanks


Adds MuRIL - BERT based model for 17 Indian Languages to the library

# What does this PR do?

This PR adds a TensorFlow-based MuRIL model to the library. More details about the MuRIL model can be found [here](https://tfhub.dev/google/MuRIL/1).

Fixes: #9190
@LysandreJik @patrickvonplaten


Revert renaming in finetune_trainer

# What does this PR do?

As per the discussion in #9241, reverting all renaming in the `finetune_trainer.py` script for now.


[seq2seq] memory regression

#9241 introduced a memory regression - found out via git bisect.

I was able to do: BS=12 before this PR got merged and now only BS=8 with:
```
 export BS=12; rm -r output_dir; CUDA_VISIBLE_DEVICES=0,1 PYTHONPATH=../../src USE_TF=0   python -m torch.distributed.launch --nproc_per_node=2 --master_port=9910  ./finetune_trainer.py --model_name_or_path sshleifer/distill-mbart-en-ro-12-4 --output_dir output_dir --adam_eps 1e-06 --data_dir wmt_en_ro --do_train --freeze_embeds --label_smoothing 0.1 --learning_rate 3e-5 --logging_first_step --logging_steps 1000 --max_source_length 128 --max_target_length 128 --num_train_epochs 1 --overwrite_output_dir --per_device_train_batch_size $BS --sortish_sampler --src_lang en_XX --task translation --tgt_lang ro_RO --val_max_target_length 128 --warmup_steps 500 --n_train 500 --sharded_ddp --fp16
```

We really need to go back to that issue of memory benchmarks in CI and figure out how to make it happen.

The problem is that I started working on it some months back but gave up since each gpu gave different numbers...

For details please see: https://github.com/huggingface/transformers/issues/6045

edit: should also make sure that `--label_smoothing 0.1 --fp16 --fp16_backend apex` works https://github.com/huggingface/transformers/issues/9261#issuecomment-749800880

@patrickvonplaten, should we figure this out in the new year?


Add speed metrics to all example scripts + template

# What does this PR do?

This does the same as #9198 but on all examples scripts and the example template.


Fix script that check objects are documented

# What does this PR do?

Currently, the script that checks objects in the main init are not documented is not really running cause I'm stupid and forgot a pair of `()`...
This PR fixes that and adds the objects introduces without documentation in their proper place.


torch.hub colab doesn't work

https://colab.research.google.com/github/pytorch/pytorch.github.io/blob/master/assets/hub/huggingface_pytorch-transformers.ipynb#scrollTo=T_3y0655Bqbj
```
%%bash
pip install tqdm boto3 requests regex sentencepiece sacremoses
```

Then all cells left don't work!


Pegasus Documentation May Conflict With Seq2Seq ReadMe

Here, under `tips and tricks`.....
https://github.com/huggingface/transformers/blob/master/examples/seq2seq/README.md#tips-and-tricks
`Both finetuning and eval are 30% faster with --fp16. For that you need to install apex.`

But in the documentation...
https://huggingface.co/transformers/master/model_doc/pegasus.html#examples
`FP16 is not supported (help/ideas on this appreciated!).`

Also in the documentation
https://huggingface.co/transformers/master/model_doc/pegasus.html#examples
`Script to fine-tune pegasus on the XSUM dataset.`
leads to a 404: https://github.com/huggingface/transformers/blob/master/examples/seq2seq/finetune_pegasus_xsum.sh


[EncoderDecoder] Make tests more aggressive

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Before merging #9183, we should make sure that the EncoderDecoder and caching tests are aggressive enough to be sure everything works as expected.

In addition, this PR refactors the `_expand_mask` function in Bart making it cleaner and move the responsibility correctly to the `attention_mask` creation.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSMT: @stas00
 -->


Fix link to bertabs/README.md

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSMT: @stas00
 -->


Fix link to old language modeling script


Prediction problem of glue task


Fix TF BART for saved model creation

# What does this PR do?

This PR fixes the graph execution issue in order to make BART able to create a proper saved model.


Model Templates for Seq2Seq

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

This PR adds the possibility to generate Encoder-Decoder models via the cookie-cutter tool.
Model is correctly generated for PT and TF with all tests passing. Two test files are added.
These templates should very much facilitate the addition of Pegasus, Blenderbot, Marian as separate model files as well as adding BigBird, etc...

Please note that for now the safety checks: `# Copied from transformers.models.bart.modeling_bart...` are only added for very few layers because:

- Bart has some hacks that we should not copy for new models, but that we need to keep for backwards compatibility. E.g. positional embeddings have an offset hack leading to slightly too large positional embeddings, which we should not repeat (same as in RoBERTa), automatic creation of `decoder_input_ids` is a special feature and not the default case, Sinusoidal position embeddings are IMO also not general enough to be in the templates
- `modeling_bart.py` still has the `add_layer_norm` hacks which are not copied to the model templates. When Bart is separated into Pegasus, etc... those if-else hacks can be deleted from `modeling_bart.py` at which point some more `# Copied from transformers.models.bart.modeling_bart...` should be added to the Seq2Seq model templates


 @mfuntowicz

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: Latest  transformers==4.2.0.dev0
- Platform:  Colab
- Python version:  Python 3.6.9
- PyTorch version (GPU?): torch==1.7.0+cu101
- Tensorflow version (GPU?):
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
@mfuntowicz
## Information

The following code indicated in the latest HF news letter seems to have isssues when I tried
I get tokenizer error both under Fast and Slow (True/Flase tokenizer parameter) conditions when I had checked

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [ ]
```python
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("mrm8488/mT5-small-finetuned-tydiqa-for-xqa",use_fast=False )

model = AutoModelForSeq2SeqLM.from_pretrained("mrm8488/mT5-small-finetuned-tydiqa-for-xqa")

context = "HuggingFace won the best Demo paper at EMNLP2020."
question = "What won HuggingFace?"
input_text = 'question: %s context: %s' % (question, context)
features = tokenizer([input_text], return_tensors='pt')
output = model.generate(**features)
tokenizer.decode(output[0])
```

## To reproduce

Steps to reproduce the behavior:

1. Run the above code on Google Colab

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     -->
**ERROR reported**

`ValueError                                Traceback (most recent call last)
<ipython-input-3-87256159791c> in <module>()
     10 from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
     11
---> 12 tokenizer = AutoTokenizer.from_pretrained("mrm8488/mT5-small-finetuned-tydiqa-for-xqa",use_fast=False )
     13
     14 model = AutoModelForSeq2SeqLM.from_pretrained("mrm8488/mT5-small-finetuned-tydiqa-for-xqa")

/usr/local/lib/python3.6/dist-packages/transformers/models/auto/tokenization_auto.py in from_pretrained(cls, pretrained_model_name_or_path, *inputs, **kwargs)
    358             if tokenizer_class is None:
    359                 raise ValueError(
--> 360                     "Tokenizer class {} does not exist or is not currently imported.".format(tokenizer_class_candidate)
    361                 )
    362             return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)

ValueError: Tokenizer class T5Tokenizer does not exist or is not currently imported.`


GPT2 distributed TPU pre-training using run_clm.py

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.2.0dev0
- Platform: Linux-4.9.0-14-amd64-x86_64-with-debian-9.13
- Python version: 3.6.10
- PyTorch version (GPU?): 1.8.0a0+5c3788d (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: yes using V3-8 TPUs

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten @TevenLeScao
 Blenderbot: @patrickvonplaten
 Bart: @patrickvonplaten
 Marian: @patrickvonplaten
 Pegasus: @patrickvonplaten
 mBART: @patrickvonplaten
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 RAG: @patrickvonplaten, @lhoestq
 FSMT: @stas00
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->
 albert, bert, GPT2, XLM: @LysandreJik
Trainer: @sgugger

## Information

Model I am using (Bert, XLNet ...): GPT2

The problem arises when using:
* [ 1] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ 1] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1.Create a GCP with pytorch/xla support
2.create a V3-8 TPU
3.install transformers and datasets libs and then run code:

```ruby
python3 transformers/examples/xla_spawn.py --num_cores=8 \
   transformers/examples/language-modeling/run_clm.py \
    --model_name_or_path gpt2 \
    --dataset_name wikitext \
    --dataset_config_name wikitext-2-raw-v1 \
    --do_train \
    --do_eval \
    --output_dir /tmp/test-clm
```
then this error comes up:
```ruby
Traceback (most recent call last):
  File "transformers/examples/xla_spawn.py", line 85, in <module>
    main()
  File "transformers/examples/xla_spawn.py", line 81, in main
    xmp.spawn(mod._mp_fn, args=(), nprocs=args.num_cores)
  File "/anaconda3/envs/torch-xla-nightly/lib/python3.6/site-packages/torch_xla/distributed/xla_multiprocessing.py", line 394, in spawn
    start_method=start_method)
  File "/anaconda3/envs/torch-xla-nightly/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 205, in start_processes
    while not context.join():
  File "/anaconda3/envs/torch-xla-nightly/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 160, in join
    exit_code=exitcode
torch.multiprocessing.spawn.ProcessExitedException: process 0 terminated with exit code 17
```
<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

and those exceptions come up before the error:
```ruby
     [[{{node XRTCompile}}]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

  (1) Resource exhausted: Ran out of memory in memory space hbm. Used 16.61G of 15.98G hbm. Exceeded hbm capacity by 645.48M.

Total hbm usage >= 16.63G:
    reserved         18.00M
    program          13.93G
    arguments         2.68G (100.0% utilization)

Output size 192.01M (100.0% utilization); shares 0B with arguments.

Program hbm requirement 13.93G:
    global             4.0K
    HLO temp         13.93G (91.5% utilization: Unpadded (12.75G) Padded (13.93G), 0.0% fragmentation (2.60M))

  Largest program allocations in hbm:

  1. Size: 1.53G
     Shape: pred[8,1023,50257]{1,2,0:T(8,128)E(32)}
     Unpadded size: 392.25M
     Extra memory due to padding: 1.15G (4.0x expansion)
     XLA label: %broadcast.4850.remat3 = pred[8,1023,50257]{1,2,0:T(8,128)E(32)} broadcast(pred[]{:T(256)E(32)} %constant.4065), dimensions={}
     Allocation type: HLO temp
     ==========================

  2. Size: 785.38M
     Shape: bf16[8,1023,50257]{1,2,0:T(8,128
```

## Expected behavior
I don't think this is an OOM problem since I am using 8 cores TPU, so it must be an XLA multiprocessing problem.
<!-- A clear and concise description of what you would expect to happen. -->


numpy ndarray type is not allowed on process pytorch model

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.1.1
- Platform: macOS-10.16-x86_64-i386-64bit
- Python version: 3.8.6
- PyTorch version (GPU?): 1.7.1 (False)
- Tensorflow version (GPU?): 2.3.1 (False)
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help
 tensorflow: @jplu
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten @TevenLeScao
 Blenderbot: @patrickvonplaten
 Bart: @patrickvonplaten
 Marian: @patrickvonplaten
 Pegasus: @patrickvonplaten
 mBART: @patrickvonplaten
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 RAG: @patrickvonplaten, @lhoestq
 FSMT: @stas00
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

## Information

Model I am using (Bert, XLNet ...): Bert

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [o] my own modified scripts: (give details below)

Changed 1 line that `from_pt`'s default value from `False` to `True`((https://github.com/huggingface/transformers/blob/ec07da65e25562040581febaf9b400a462962961/src/transformers/modeling_tf_utils.py#L947))

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [o] my own task or dataset: (give details below)

HuggingFace: monologg/kobert

I loaded my pertained pytorch model, and an error occurred on input_processing function

## To reproduce

Steps to reproduce the behavior:

1. run input_processing
2. numpy ndarray is not allowed type(https://github.com/huggingface/transformers/blob/ec07da65e25562040581febaf9b400a462962961/src/transformers/modeling_tf_utils.py#L331)
3. So ndarray cannot be processed(https://github.com/huggingface/transformers/blob/ec07da65e25562040581febaf9b400a462962961/src/transformers/modeling_tf_utils.py#L354)
4. Of course there is not any proper changing for ndarray not as other types(dict, Tensor and etc)
5. Error occurred like below

```
  File "/Users/ys/dev/rasa/venv/lib/python3.8/site-packages/transformers/modeling_tf_utils.py", line 357, in input_processing
    raise ValueError(f"Data of type {type(v)} is not allowed only {allowed_types} is accepted for {k}.")
ValueError: Data of type <class 'numpy.ndarray'> is not allowed only (<class 'tensorflow.python.framework.ops.Tensor'>, <class 'bool'>, <class 'int'>, <class 'transformers.file_utils.ModelOutput'>, <class 'tuple'>, <class 'list'>, <class 'dict'>) is accepted for attention_mask.
```
<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

Train success!


T5 tokenizer.vocab_size and config.vocab_size mismatch?

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.1.1
- Python version: 3.8.5
- PyTorch version (GPU?): 1.7.1
- tokenizers: 0.9.4
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

## Information
Hi @patrickvonplaten
I am trying to train a "t5-base" model and I directly use from_pretrained tokenizer, config and model. However, I found the vocabulary size given by the tokenizer and config is different (see to reproduce). Does this is expected?

If I use the model `T5ForConditionalGeneration.from_pretrained('t5-base', config=config)` to do predictions, this will result in the last dimension of lm_logits is different from `tokenizer.vocab_size`.

## To reproduce

Steps to reproduce the behavior:

```
>>> from transformers import T5Tokenizer, T5Config
>>> tokenizer = T5Tokenizer.from_pretrained("t5-base")
>>> config = T5Config.from_pretrained("t5-base")
>>> print(tokenizer.vocab_size)
32100
>>> print(config.vocab_size)
32128
```

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
```
>>> print(tokenizer.vocab_size)
32128
>>> print(config.vocab_size)
32128
```


torch: 1.6.0
transformers: 3.5.1
OS: centos 7
GPU: A100

I trained a sentencepiece bpe model. There is no problem if I load it with `XLMRobertaTokenizer`. But when I load with `XLMRobertaTokenizerFast`, it cost a long time in `transformers/convert_slow_tokenizer.py`. After save the tokenizer and load it `from_pretrained`, the error occurs:
```
/ProjectRoot/tp_origin/pyenv/lib/python3.7/site-packages/transformers/tokenization_utils_base.py in _from_pretrained(cls, resolved_vocab_files, pretrained_model_name_or_path, init_configuration, *init_inputs, **kwargs)
   1758             for token, index in added_tok_encoder_sorted:
   1759                 assert index == len(tokenizer), (
-> 1760                     f"Non-consecutive added token '{token}' found. "
   1761                     f"Should have index {len(tokenizer)} but has index {index} in saved vocabulary."
   1762                 )

AssertionError: Non-consecutive added token '<pad>' found. Should have index 40002 but has index 40000 in saved vocabulary.
```
I find that a file `added_tokens.json` is created with content `{"<pad>": 40000, "<mask>": 40001}`.

my tokenizer
```
PreTrainedTokenizerFast(name_or_path='/ProjectRoot/tp_origin/distillation/tmp/student_init_model3/0_Transformer', vocab_size=40000, model_max_len=514, is_fast=True, padding_side='right', special_tokens={'bos_token': '<s>', 'eos_token': '</s>', 'unk_token': '<unk>', 'sep_token': '</s>', 'pad_token': '<pad>', 'cls_token': '<s>', 'mask_token': '<mask>'})
```


[s2s] test_finetune_trainer_slow fails when run in group

On dual gpu when running `test_finetune_trainer.py::TestFinetuneTrainer::test_finetune_trainer_slow` alone - all is good.

when running it with all the other tests in that file it fails:

```
RUN_SLOW=1 pytest -sv test_finetune_trainer.py
[...]
self = <seq2seq.test_finetune_trainer.TestFinetuneTrainer testMethod=test_finetune_trainer_slow>

    @slow
    def test_finetune_trainer_slow(self):
        # There is a missing call to __init__process_group somewhere
        output_dir = self.run_trainer(
            eval_steps=2, max_len="128", model_name=MARIAN_MODEL, num_train_epochs=10, distributed=False
        )

        # Check metrics
        logs = TrainerState.load_from_json(os.path.join(output_dir, "trainer_state.json")).log_history
        eval_metrics = [log for log in logs if "eval_loss" in log.keys()]
        first_step_stats = eval_metrics[0]
        last_step_stats = eval_metrics[-1]

>       assert first_step_stats["eval_bleu"] < last_step_stats["eval_bleu"]  # model learned nothing
E       AssertionError: assert 0.0 < 0.0

test_finetune_trainer.py:130: AssertionError
----------------------------------------------------------------------- Captured log call -----------------------------------------------------------------------
WARNING  seq2seq.finetune_trainer:finetune_trainer.py:160 Process rank: -1, device: cuda:0, n_gpu: 2, distributed training: False, 16-bits training: False

==================================================================== short test summary info ====================================================================
FAILED test_finetune_trainer.py::TestFinetuneTrainer::test_finetune_trainer_slow - AssertionError: assert 0.0 < 0.0
================================================ 1 failed, 7 passed, 1 skipped, 17 warnings in 102.82s (0:01:42) ================================================
```

For some reason it fails to learn anything when some other tests run before it.

tested with pytorch-nightly + py38.


BatchEncoding.to accepted types too restrictive

## Environment info

- `transformers` version: 4.1.1
- Platform: Linux-4.14.81.bm.15-amd64-x86_64-with-debian-9.11
- Python version: 3.7.9
- PyTorch version (GPU?): 1.7.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: yes

### Who can help

## Information

In `BatchEncoding.to`, the only accepted class types are `str` and `torch.device`. I think some libraries like pytorch-lightning call `.to` with the integer value for the GPU number, and HF complains about this when it is perfectly valid:

```
>>> x = torch.zeros(1)
>>> x.to(0)
tensor([0.], device='cuda:0')
```

## Expected behavior

Also allow int values in BatchEncoding.to


AssertionError with model_parallel in run_clm.py

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.1.1
- Platform: AWS Sagemaker
- Python version:  3.6
- PyTorch version (GPU?): 1.6
- Tensorflow version (GPU?):
- Using GPU in script?: YES
- Using distributed or parallel set-up in script?: YES

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten @TevenLeScao
 Blenderbot: @patrickvonplaten
 Bart: @patrickvonplaten
 Marian: @patrickvonplaten
 Pegasus: @patrickvonplaten
 mBART: @patrickvonplaten
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 RAG: @patrickvonplaten, @lhoestq
 FSMT: @stas00
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->
@LysandreJik  @sgugger @alexorona

## Information

Model I am using (Bert, XLNet ...): GPT2

The problem arises when using:
* [X] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [X] my own task or dataset: (give details below)

I am fine-tuning GPT2 using my own dataset, using the examples/language-modelling/run_clm.py script, and I want to use the new model_parallel feature in v4.1.1. I am using a multi-gpu instance (AWS p3.8xlarge - with 4 gpus).

But I get this error:
AssertionError: DistributedDataParallel device_ids and output_device arguments only work with single-device CUDA modules, but got device_ids [2], output_device 2, and module parameters {device(type='cpu')}.

## To reproduce

Steps to reproduce the behavior:

1. Run run_clm.py with the following params:
```
python -m torch.distributed.launch \
    --nproc_per_node 4 run_clm.py \
    --do_train \
    --do_eval \
    --fp16 \
    --logging_first_step \
    --model_parallel \
    --evaluation_strategy epoch \
    --logging_steps 50 \
    --model_name_or_path gpt2 \
    --model_type gpt2 \
    --num_train_epochs 1 \
    --output_dir /opt/ml/model/ \
    --per_device_eval_batch_size 2 \
    --per_device_train_batch_size 2 \
    --save_steps 50 \
    --save_total_limit 1 \
    --train_file /opt/ml/input/data/data/train.txt \
    --validation_file /opt/ml/input/data/data/val.txt
```

2. I get this error when training starts:
```
Traceback (most recent call last):
  File "run_clm.py", line 374, in <module>
    main()
  File "run_clm.py", line 344, in main
    trainer.train(model_path=model_path)
  File "/opt/conda/lib/python3.6/site-packages/transformers/trainer.py", line 681, in train
    else True
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/parallel/distributed.py", line 282, in __init__
#015 95%|ââââââââââ| 62/65 [00:04<00:00, 12.58ba/s]    ).format(device_ids, output_device, {p.device for p in module.parameters()})
AssertionError: DistributedDataParallel device_ids and output_device arguments only work with single-device CUDA modules, but got device_ids [1], output_device 1, and module parameters {device(type='cpu')}.
#015 98%|ââââââââââ| 64/65 [00:05<00:00, 12.26ba/s]#015100%|ââââââââââ| 65/65 [00:05<00:00, 12.61ba/s]
Traceback (most recent call last):
  File "run_clm.py", line 374, in <module>
    main()
  File "run_clm.py", line 344, in main
    trainer.train(model_path=model_path)
  File "/opt/conda/lib/python3.6/site-packages/transformers/trainer.py", line 681, in train
    else True
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/parallel/distributed.py", line 282, in __init__
    ).format(device_ids, output_device, {p.device for p in module.parameters()})
AssertionError: DistributedDataParallel device_ids and output_device arguments only work with single-device CUDA modules, but got device_ids [2], output_device 2, and module parameters {device(type='cpu')}.
#015 95%|ââââââââââ| 62/65 [00:05<00:00, 11.84ba/s]#015 98%|ââââââââââ| 64/65 [00:05<00:00, 12.18ba/s]#015100%|ââââââââââ| 65/65 [00:05<00:00, 12.60ba/s]
Traceback (most recent call last):
  File "run_clm.py", line 374, in <module>
    main()
  File "run_clm.py", line 344, in main
    trainer.train(model_path=model_path)
  File "/opt/conda/lib/python3.6/site-packages/transformers/trainer.py", line 681, in train
    else True
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/parallel/distributed.py", line 282, in __init__
    ).format(device_ids, output_device, {p.device for p in module.parameters()})
AssertionError: DistributedDataParallel device_ids and output_device arguments only work with single-device CUDA modules, but got device_ids [3], output_device 3, and module parameters {device(type='cpu')}.
#015 98%|ââââââââââ| 64/65 [00:05<00:00, 11.50ba/s]#015100%|ââââââââââ| 65/65 [00:05<00:00, 12.48ba/s]
[INFO|trainer.py:388] 2020-12-22 00:54:34,892 >> The following columns in the training set don't have a corresponding argument in `GPT2LMHeadModel.forward` and have been ignored: .
[INFO|trainer.py:388] 2020-12-22 00:54:34,892 >> The following columns in the evaluation set don't have a corresponding argument in `GPT2LMHeadModel.forward` and have been ignored: .
Traceback (most recent call last):
  File "run_clm.py", line 374, in <module>
    main()
  File "run_clm.py", line 344, in main
    trainer.train(model_path=model_path)
  File "/opt/conda/lib/python3.6/site-packages/transformers/trainer.py", line 681, in train
    else True
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/parallel/distributed.py", line 282, in __init__
    ).format(device_ids, output_device, {p.device for p in module.parameters()})
AssertionError: DistributedDataParallel device_ids and output_device arguments only work with single-device CUDA modules, but got device_ids [0], output_device 0, and module parameters {device(type='cpu')}.
Traceback (most recent call last):
  File "/opt/conda/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/opt/conda/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/opt/conda/lib/python3.6/site-packages/torch/distributed/launch.py", line 261, in <module>
    main()
  File "/opt/conda/lib/python3.6/site-packages/torch/distributed/launch.py", line 257, in main
    cmd=cmd)
subprocess.CalledProcessError: Command '['/opt/conda/bin/python', '-u', 'run_clm.py', '--local_rank=3', '--do_train', '--do_eval', '--fp16', '--logging_first_step', '--model_parallel', '--evaluation_strategy', 'epoch', '--logging_steps', '50', '--model_name_or_path', 'gpt2', '--model_type', 'gpt2', '--num_train_epochs', '1', '--output_dir', '/opt/ml/model/', '--per_device_eval_batch_size', '2', '--per_device_train_batch_size', '2', '--save_steps', '50', '--save_total_limit', '1', '--train_file', '/opt/ml/input/data/data/train.txt', '--validation_file', '/opt/ml/input/data/data/val.txt']' returned non-zero exit status 1.
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
I expect the fine-tuning script to run successfully.

If I remove the --model_parallel in the args, then it does run successfully in distributed mode. But I want to use this new feature to reduce memory usage, and increase batch_size


Load from a TF 1.0 checkpoint in modeling_tf_utils.py

In the file modeling_utils.py, we can load a TF 1.0 checkpoint as is indicated in this [line](https://github.com/huggingface/transformers/blob/fb650df8590f796663226132482d09da5b0fb613/src/transformers/modeling_utils.py#L930). However, in the file modeling_tf_utils.py, which is the same version for TF, we can not load models from TF 1.0, and it says expecifically that you can as:
`            >>> model = BertModel.from_pretrained('./tf_model/my_tf_checkpoint.ckpt.index', from_tf=True, config=config)`
But there is no _if_ for
`os.path.isfile(os.path.join(pretrained_model_name_or_path, TF_WEIGHTS_NAME + ".index"))`


Seq2seq trainer

# What does this PR do?

This PR graduates `Seq2SeqTrainer` and moves it inside the transformers library. By doing so, it moves some of the features of the `Seq2SeqTrainer` inside the main `Trainer` and leaves some in the subclass. More precisely, the following features will be available in the general Trainer:
- label smoothing is passed to the main `Trainer` (so it can be used in all classification problems), with an easier API, bug fixes (the current implementation did not work with -100 has an ignore index for the loss, now it does; also the current implementation returns a loss that is not averaged thus being too big, see below)
- the ability to pick any scheduler
- the ability to use Adafactor instead of AdamW

The sortish-sampling and predict with generate are left in the subclass.

There are also a few breaking changes in the `Seq2SeqTrainer` API to make its init match the one of `Trainer` mainly the init does not take a `config` and a `DataArguments`, instead:
- the token IDs are taken from the tokenizer
- the arguments for generation are passed to `evaluate` or `predict`
- the `ignore_pad_token_for_loss` is passed in the init but is deprecated since it should be removed once BART and subclasses use -100.

About label smoothing and the mean vs sum. The current implementation takes the sum over the batch size and sequence length (only counting tokens that have a label != -100). This gives something that does not have the same scale as the usual cross entropy loss (which is the mean on the same dimensions) thus would require a special learning rate to be useful. With the fix, the label-smoothed loss had the same scale as the non-smoothed loss, which means the same command with the same learning rate should produce comparable results.


Help: How to deploy a fine tuned t5 model in production

Hi All,

I am trying to deploy a fine-tuned t5 model in production. This is something new to me, to deploy a PyTorch model in production. I went through the presentation from Hugging Face on youtube, about how they deploy the model. And some of the other blog posts.

It is mentioned by HF that they deploy the model on Cython environment as it gives a ~100 times boost to the inference. So, is it always advisable to run a model in production on Cython?
Converting a model in Pytorch to TF does it help and is advisable or not?
What is the preferred container approach to adopt to run multiple models on a set of GPUs?

I know some of these questions would be basic, I apologize for it, but I want to make sure that I follow the correct guidelines to deploy a model in production.

Thank you
Amit


Adding performer fine-tuning research exampke

# What does this PR do?

This PR adds a performer fine-tuning research example based on `run_mlm_flax.py`. The user can fine-tune a Performer/FAVOR+ Bert starting from the Bert checkpoint or blank model of their choice, and compare it to a vanilla Bert model, also from a checkpoint or blank.

@patrickvonplaten