[DOC] Improve pipeline() docstrings for config and tokenizer

As currently written, it was not clear to me which arguments were needed when using a non-default model in `pipeline()`. It seemed that when you provided a non-default `model`, that you still needed to manually change the `config` and `tokenizer` because otherwise the "task's default will be used". In practice, though, the pipeline is smart enough to automatically choose the right config/tokenizer for the given model. This PR clarifies that a bit in the docstrings/documentation, by explaining exactly which priorities are used when loading the tokenizer. A small change was made for `config`, too.

Admittedly, the wording for the tokenizer part is a bit off (programmatical, even), but I think it should make clear how the right tokenizer is loaded.

cc @sgugger


behaviour of ZeroShotClassification using facebook/bart-large-mnli is different on online demo vs local machine

## Environment info

- `transformers` version: 3.4.0
- Platform: Ubuntu 20.04
- Python version: 3.7.7
- PyTorch version (GPU?): 1.6.0 (GPU:Yes)
- Tensorflow version (GPU?): No
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 blenderbot: @mariamabarham
 Bart: @sshleifer
 Marian: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

@sshleifer

## Information

Model I am using (Bert, XLNet ...): facebook/bart-large-mnli

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [X] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [X] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

First I tried the hosted demo online at huggingface, which gives me a very high score of **0.99 for travelling (as expected)**:
![Screenshot from 2020-10-29 00-02-40](https://user-images.githubusercontent.com/62788745/97464214-949e3480-197b-11eb-9fc8-72d5fcdcc49c.png)

Then I tried to run the code on my local machine, which returns **very different scores for all labels** (poor scores):

```
from transformers import pipeline
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("facebook/bart-large-mnli")
model = AutoModel.from_pretrained("facebook/bart-large-mnli")

zsc = pipeline(task='zero-shot-classification', tokenizer=tokenizer, model=model)

sequences = 'one day I will see the world'
candidate_labels = ['travelling', 'cooking', 'dancing']

results = zsc(sequences=sequences, candidate_labels=candidate_labels, multi_class=False)
print(results)
>>>{'sequence': 'one day I will see the world',
'labels': ['travelling', 'dancing', 'cooking'],
'scores': [0.5285395979881287, 0.2499372661113739, 0.22152313590049744]}
```

I **got this warning message when initializing the model**:
`model = AutoModel.from_pretrained("facebook/bart-large-mnli")`

```
Some weights of the model checkpoint at facebook/bart-large-mnli were not used when initializing BartModel: ['model.encoder.version', 'model.decoder.version']
- This IS expected if you are initializing BartModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPretraining model).
- This IS NOT expected if you are initializing BartModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
```

## Expected behavior

Code on my local machine's **_score_** to be quite similar to the online demo.


fix(trainer_callback]: typo

# What does this PR do?

Fix a typo in `trainer_callback`
<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR. @sgugger

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


Rename add_start_docstrings_to_callable

# What does this PR do?

This PR rename all `add_start_docstrings_to_callable` to a more explicit `add_start_docstrings_to_model_forward`. This should avoid confusion on the use of this decorator.

(It's an internal function so there should be no breaking change.)


feat(wandb): save model as artifact

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

**EDIT**

The logic has been simplified.
Model is just saved to a temporary folder and uploaded as artifact at the end of training.

**ORIGINAL message**

Save trained model as artifact.

A few different possibilities:

* log model at `on_save` callback -> the issue is there could quickly be too many checkpoints to upload, high bandwidth…

* log model at `on_train_end`

   * when we have access to `state.best_model_checkpoint`, we should just upload that folder
   * we could upload entire `output_dir` but it could be very large (same problem as `on_save`, ideally we only upload one model
   * we can save last model in a separate folder and upload it -> issue is that we don't have access to `Trainer.save_model` (where are the 2-way callbacks 😉)
   * we could just use `_sorted_checkpoints` and log only last element (which would also be the best model when metrics are given)

I'm thinking I should actually go with the last option (use of `_sorted_checkpoints`). What do you think?

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [X] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR. @sgugger

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 Trainer: @sgugger
 -->


Document the various LM Auto models

# What does this PR do?

This PR adds the documentation for the three classes of mdoels for LM (`AutoModelForCausalLM`, `AutoModelForMaskedLM` and `AutoModelForSeq2SeqLM`) and their TF equivalent. It also removes the documentation of `AutoModelWithLMHead` which is deprecated.


fast tokenizer issue on most user uploaded models

## Environment info

- `transformers` version: 3.4.0
- Platform: Linux-5.8.0-25-generic-x86_64-with-glibc2.32
- Python version: 3.8.6
- PyTorch version (GPU?): 1.6.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: yes

### Who can help
@mfuntowicz @julien-c

## Information

Found the bug on `camembert/camembert-base-ccnet` but probably common to many models uploaded by users.
On camembert base model, it works out of the box (there is no bug).

## To reproduce

Since `tokenizer` 0.9, it's possible to load the many unigram based tokenizers with the fast Rust implementation.
It appears that the file `tokenizer_config.json` of some of them is not up to date, in particular the information `"model_max_length": 512` is missing.

Because of that, the value of `model_max_length` is a very big integer.

```python
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("camembert/camembert-base-ccnet", use_fast=True)
tokenizer.model_max_length
# Out[4]: 1000000000000000019884624838656
```

To fix it, the field model_max_length has to be added to the config file.

## Expected behavior

I would expect `tokenizer.model_max_length` to be equal to 512.


Add labels padding in tokenization_utils_base.py

# What does this PR do?

This PR makes `tokenizer.pad()` also pad `'labels'`.

I tried to use this:
https://github.com/huggingface/transformers/blob/8065fea87007fbf7542fc060ff8ddd0b5df567da/src/transformers/data/data_collator.py#L69

But since labels is not padded, the result cannot turn into a tensor. `ValueError: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same lengt
h.
`
This patch solves the problem.

It seems logical to me that `tokenizer.pad()` should also pad `'labels'`.

This portion of code is last changed in #4015 @n1t0 @thomwolf @LysandreJik


Fix eval ref miss in Chinese WWM.

Sorry for my reckless: I didn't add param for `eval_ref_file` when Chinese WWM.
It was found by @johnsonice in [here](https://github.com/huggingface/transformers/pull/7925#issuecomment-717701325).

So I fix it and update readme for Chinese WWM.


Pegasus: Error when training with increased input length

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.4.0
- Platform: MacOS Mojave (10.14.6)
- Python version: 3.7.9
- PyTorch version (GPU?): 1.6.0 (True)
- Tensorflow version (GPU?): NA
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
@sshleifer
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 blenderbot: @mariamabarham
 Bart: @sshleifer
 Marian: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

## Information

Model I am using (Bert, XLNet ...): Pegasus

The problem arises when using:
* [ ] the official example scripts: NA
* [x] my own modified scripts:

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: NA
* [x] my own task or dataset: Long input training on the standard CNN/DM dataset.

## To reproduce

Steps to reproduce the behavior:

1. I am trying to train a Pegasus model using the following script on a larger input length.
```python
from transformers import PegasusConfig, PegasusForConditionalGeneration, PegasusTokenizer
from transformers import Trainer, TrainingArguments
from examples.seq2seq.utils import Seq2SeqDataset

config = PegasusConfig(
    max_length=2048,
    max_position_embeddings=2048,
    encoder_layers=16,
    decoder_layers=4,
    num_beams=2
)

tokenizer = PegasusTokenizer.from_pretrained("sshleifer/distill-pegasus-cnn-16-4")
model = PegasusForConditionalGeneration(config=config)

dataset = Seq2SeqDataset(data_dir='data/cnn_dm', tokenizer=tokenizer, max_source_length=2048, max_target_length=150)

training_args = TrainingArguments(
    output_dir="./data/output",
    overwrite_output_dir=True,
    num_train_epochs=1,
    save_steps=10,
    save_total_limit=2,
)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=dataset,
    prediction_loss_only=True,
)

trainer.train()

```

2. I am getting the following error message
```
/Users/sdasgupta02/code/summarization/summarization-long/transformers/src/transformers/trainer.py:263: FutureWarning: Passing `prediction_loss_only` as a keyword argument is deprecated and won't be possible in a future version. Use `args.prediction_loss_only` instead. Setting `args.prediction_loss_only=True
  FutureWarning,
  0%|          | 0/2 [00:00<?, ?it/s]Traceback (most recent call last):
  File "/Users/sdasgupta02/code/summarization/summarization-long/transformers/train_scratch.py", line 34, in <module>
    trainer.train()
  File "/Users/sdasgupta02/code/summarization/summarization-long/transformers/src/transformers/trainer.py", line 756, in train
    tr_loss += self.training_step(model, inputs)
  File "/Users/sdasgupta02/code/summarization/summarization-long/transformers/src/transformers/trainer.py", line 1056, in training_step
    loss = self.compute_loss(model, inputs)
  File "/Users/sdasgupta02/code/summarization/summarization-long/transformers/src/transformers/trainer.py", line 1080, in compute_loss
    outputs = model(**inputs)
  File "/anaconda3/envs/transformers/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
TypeError: forward() missing 1 required positional argument: 'input_ids'
  0%|          | 0/2 [00:00<?, ?it/s]
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

Expected behaviour is to be able to train this Pegasus model on CNN/DM dataset on longer input sequences (> 1024).

<!-- A clear and concise description of what you would expect to happen. -->


[WIP] Add Tapas model

# What does this PR do?

Since the beginning of August, I'm working in my free time on incorporating the [Tapas](https://arxiv.org/abs/2004.02349) algorithm by Google AI in the Transformers library (because this library is awesome and I want to contribute to it!). Tapas is basically a BERT model with some clever modifications for natural language understanding related to **tabular data** (structured data like tables, or even HTML). Adding this model could foster research in this area 😄

Demo's of my current implementation:
* [colab notebook](https://colab.research.google.com/drive/1feRe1Jyjtw7iZVRiKWulP6WjBW6hBIJE?usp=sharing) to showcase `TapasForQuestionAnswering` on WTQ (WikiTable Questions by Stanford University)
* [colab notebook](https://colab.research.google.com/drive/1CDPUr7c8uCNnCtAmmFfj91j-sIzdcqym?usp=sharing) to showcase `TapasForQuestionAnswering`on SQA (Sequential Question Answering by Microsoft Research)
* [colab notebook](https://colab.research.google.com/drive/1JDwWrwHSt8KhGBQ57BDlCFZEe0xMGTQA?usp=sharing) to showcase `TapasForSequenceClassification` on TabFact (Table Fact checking, introduced at ICLR this year)

The model weights are available on the [original Github repository](https://github.com/google-research/tapas), and I wrote a conversion script (similar to other models in the Transformers library) to load them into their PyTorch counterpart.
I suggest reading the [paper](https://arxiv.org/abs/2004.02349) as well as my [notes](https://docs.google.com/document/d/1WIdZX6of1l-c4AmT909PT7Dpj57EfqUh8BBPaf9ztOw/edit?usp=sharing) to gain a full understanding of how the model works and how I implemented it. There's also a [blog post](https://ai.googleblog.com/2020/04/using-neural-networks-to-find-answers.html) by Google AI as well as a [video](https://www.youtube.com/watch?v=cIUtRNhY6Rw&ab_channel=YannicKilcher) by Yannic Kilcher explaining how the algorithm works.

The main classes are `TapasConfig`, `TapasModel`, `TapasForQuestionAnswering` and `TapasForSequenceClassification` which can all be found in `modeling_tapas.py`. I'm quite sure the models are OK, the output is the same as the Tensorflow implementation. I added a very extensive documentation (docstrings) to all classes, which you can view by running the `make html` command from the docs directory. Feedback appreciated!

However, there are 2 things for which I need some help/opinions to finish this work:

## 1. Making TapasTokenizer fully Transformers-compliant
To implement `TapasTokenizer`, I need some help/opinions. I suggest using Pandas dataframes as the central object for tabular data (as shown in the Colab notebooks above). and let the API be as follows:

```
from transformers import TapasTokenizer
import pandas as pd

data = {'Actors': ["Brad Pitt", "Leonardo Di Caprio", "George Clooney"],
        'Age': ["56", "45", "59"],
        'Number of movies': ["87", "53", "69"],
        'Date of birth': ["18 december 1963", "11 november 1974", "6 may 1961"]}
table = pd.DataFrame.from_dict(data)
queries = ["When was Brad Pitt born?",
           "Which actor appeared in the least number of movies?",
           "What is the average number of movies?"]

tokenizer = TapasTokenizer.from_pretrained("tapas-base-finetuned-wtq")
inputs = tokenizer(table=table, queries=queries)
```

Currently I've only implemented the `batch_encode_plus` method of TapasTokenizer, because it's not really clear to me how to make it fully compatible with the Transformers library, since the way that data is prepared for the model is a bit different compared to BERT/RoBERTa/etc (see also my notes above). It's also not straightforward to make it compatible with the different padding/truncation strategies of Transformers. Currently, the way it works is as follows: there’s a function `_get_token_budget` in `tokenization_tapas.py` that calculates the number of tokens left for the flattened table after tokenizing a question. This is currently set to `self.model_max_length - (len(question_tokens) + 2)` (+ 2 for the CLS and SEP tokens), as was done in the original implementation. There is a hyperparameter when initializing `TapasTokenizer` called `drop_rows_to_fit` which drops rows of the table to fit into the token budget if set to `True`. If it’s set to `False` and a table is too big, it throws a `ValueError` indicating 'too many rows'.

## 2. Testing
Currently I've written `test_modeling_tapas.py` (23 tests passed, 5 failed) and `test_modeling_tapas_utilities.py` (9 tests passed). However, there are 4 different settings to use `TapasForQuestionAnswering` (see my notes) and these all need to be tested (currently only 1 setting is tested) - some help here would be great. Besides this, tests should be added to see whether the model can be properly trained, as well as adding `test_tokenization_tapas.py` (which depends on how TapasTokenizer will be implemented).

Fixes the following issues (people requesting to add the model):
- #4166
- #4288

## Who can review?
I suggest @sgugger @LysandreJik since we already discussed this on the forum [here](https://discuss.huggingface.co/t/adding-a-new-model-to-transformers-with-additional-dependencies/916/15).
 tokenizers: @mfuntowicz

DISCLAIMER: this is my first PR of my life, never done this before, hopefully I don't mess up anything (just got the Pro Git book 😛). I assume I should not use `git rebase` anymore now since this branch submitted as PR and should only use `git add`, `git commit` and `git push -u origin tapas_v3`? And `git pull origin tapas_v3` in case others make commits to my branch?
Is there a Slack channel where people can help me out in case I have git issues?


Documentation code snippet has extra ) after model code

Documentation at https://huggingface.co/transformers/model_doc/roberta.html#tfrobertaforsequenceclassification has code snippet

```
>> from transformers import RobertaTokenizer, TFRobertaForSequenceClassification
>> import tensorflow as tf
>> tokenizer = RobertaTokenizer.from_pretrained('roberta-base')
>> model = TFRobertaForSequenceClassification.from_pretrained('roberta-base', return_dict=True))
>> inputs = tokenizer("Hello, my dog is cute", return_tensors="tf")
>> inputs["labels"] = tf.reshape(tf.constant(1), (-1, 1)) # Batch size 1
>> outputs = model(inputs)
>> loss = outputs.loss
>> logits = outputs.logits
```

in the fourth line there is extra `)` at the end.

This is issue is also for other model code snippet as well


[Model] mT5 Cross-Lingual Model

# 🌟 New model addition

## Model description

<!-- Important information -->

Multilingual T5 (mT5) is a massively multilingual pretrained text-to-text transformer model, trained following a similar recipe as T5.

Weights, code are available.

Github Repo: [mT5 Weights and Code](https://github.com/google-research/multilingual-t5)

Paper: [mT5: A massively multilingual pre-trained text-to-text transformer](https://arxiv.org/abs/2010.11934)

## Open source status

* [x] the model implementation is available: [Implementation](https://github.com/google-research/multilingual-t5)
* [x] the model weights are available: [checkpoints](https://github.com/google-research/multilingual-t5#released-model-checkpoints)
* [x] who are the authors: (@craffel, @adarob)


[gh actions] run artifacts job always

I see that the recently added artifacts job won't run if the test job failed, which defeats the purpose. ([example](https://github.com/huggingface/transformers/runs/1317972448?check_suite_focus=true))

After some research it appears that adding `if: always()` may do the trick. Supposedly such a job should always be run regardless of the outcome of the previous jobs. Found it [here](https://github.community/t/continue-on-error-allow-failure-ui-indication/16773/2?u=stas00), documented [here](https://docs.github.com/en/free-pro-team@latest/actions/reference/context-and-expression-syntax-for-github-actions#always).

Let's merge and see if it fixes the issue.

@LysandreJik, @sgugger, @sshleifer


T5Tokenizer: decode does not show special tokens

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.4.0
- Platform: macOS-10.15.7-x86_64-i386-64bit
- Python version: 3.8.5
- PyTorch version (GPU?): 1.7.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: N/a
- Using distributed or parallel set-up in script? N/a

### Who can help
  examples/seq2seq: @sshleifer
 -->

## Information

Model I am using (Bert, XLNet ...): T5Tokenizer

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [X ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ X] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:
```python
from transformers import T5Tokenizer
input = "word  <pad> <unk> </s> </s>"
t5tokenizer = T5Tokenizer.from_pretrained('t5-small')
tokenized = t5tokenizer.batch_encode_plus([input], max_length=10, padding="longest", return_tensors="pt").input_ids
print(t5tokenizer.batch_decode(tokenized, skip_special_tokens=False, clean_up_tokenization_spaces=False))
```
IDs output: ` _word  <pad> <unk> </s> </s>`
decode output: `word  ⁇ `

## Expected behavior
The tokens should be shown in the decoded output, but everything except for the unknown token is dropped (no pad or EOS).
`convert_ids_to_tokens` followed by `convert_tokens_to_string` also drops the tokens.
<!-- A clear and concise description of what you would expect to happen. -->


Support various BERT relative position embeddings

# What does this PR do?

The default BERT model `bert-base-uncased` was pre-trained with absolute position embeddings. We provide three pre-trained models which were pre-trained on the same training data (BooksCorpus and English Wikipedia) as in the BERT model training, but with different relative position embeddings (Shaw et al., Self-Attention with Relative Position Representations, https://arxiv.org/abs/1803.02155 and Huang et al., Improve Transformer Models with Better Relative Position Embeddings, https://arxiv.org/abs/2009.13658, accepted in findings of EMNLP 2020). We show how to fine-tune these pre-trained models on SQuAD1.1 data set and we also report the EM and F1 score on SQuAD1.1 dev dataset. See examples/question-answering/README.md for more details.

Fixes # (issue)

N/A

## Before submitting

- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

@LysandreJik @julien-c @patrickvonplaten


[testing] port test_trainer_distributed to distributed pytest + TestCasePlus enhancements

This PR:

* [x] ports `test_trainer_distributed` to run with pytest - it will skip if gpus < 2.

* [x] includes various improvements via refactoring now 3 use cases of distributed testing by extending `TestCasePlus` with a whole set of convenient features:

Feature 1: A set of fully resolved important file and dir path accessors.

In tests often we need to know where things are relative to the current test file, and it's not trivial since the test could be invoked from more than one directory or could reside in different sub-directories. This class solves this problem by sorting out all the basic paths and provides easy accessors to them:

* ``pathlib`` objects (all fully resolved):

   - ``test_file_path`` - the current test file path (=``__file__``)
   - ``test_file_dir`` - the directory containing the current test file
   - ``tests_dir`` - the directory of the ``tests`` test suite
   - ``examples_dir`` - the directory of the ``examples`` test suite
   - ``repo_root_dir`` - the directory of the repository
   - ``src_dir`` - the directory of ``src`` (i.e. where the ``transformers`` sub-dir resides)

* stringified paths - same as above but these return a string, rather than a ``pathlib`` object

   - ``test_file_path_str``
   - ``test_file_dir_str``
   - ``tests_dir_str``
   - ``examples_dir_str``
   - ``repo_root_dir_str``
   - ``src_dir_str``

Feature 2: Get a copy of the ``os.environ`` object that sets up ``PYTHONPATH``  correctly, depending on the test suite it's invoked from. This is useful for invoking external programs from the test suite - e.g. distributed training.

```
    def test_whatever(self):
        env = self.get_env()
        # now call the external program, passing ``env`` to it
```

All these are also documented in `testing.rst`.

Fixes: #8058

@sgugger, @LysandreJik, @sshleifer


Move installation instructions to the top

# What does this PR do?

This PR clarifies the instructions to run the examples by moving the source install to the top and putting it in bold.


New run_clm script

# What does this PR do?

This PR adds an example of a causal language modeling fine-tuning (or training from scratch) using the 🤗 Datasets library. It supports loading a dataset via its name (from the hub) or local files. A test of training on a small text is added.


RagSequenceForGeneration how to get document texts retrieved  in response to a query

When I run the retriever separately, how can I find out the text of the documents (from the doc_ids ?) that are retrieved ?

I created the retriever using:
retriever = RagRetriever.from_pretrained(rag_example_args.rag_model_name,index_name="custom",passages_path=passages_path,index_path=index_path,n_docs=8)
tokenizer = RagTokenizer.from_pretrained(rag_example_args.rag_model_name)

model = RagSequenceForGeneration.from_pretrained(rag_example_args.rag_model_name,index_name="custom",indexed_dataset=dataset)

question_hidden_states = model.question_encoder(input_ids)[0]
docs_dict = retriever(input_ids.numpy(), question_hidden_states.detach().numpy(), return_tensors="pt")


run_language_modeling crashes with import cannot import name 'DataCollatorForWholeWordMask' from 'transformers'

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.4.0
- Platform: linux
- Python version: 3.8
- PyTorch version (GPU?): 1.5
- Tensorflow version (GPU?):
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: No

## Information
I am trying to run the example for [language modeling](https://github.com/huggingface/transformers/tree/master/examples/language-modeling) but can't get it to start. Import fails
Traceback (most recent call last):
  File "run_language_modeling.py", line 32, in <module>
    from transformers import (
ImportError: cannot import name 'DataCollatorForWholeWordMask' from 'transformers' (/home/spacemanidol/miniconda3/envs/prunetransformer/lib/python3.8/site-packages/transformers/__init__.py)
Model I am using (Bert, XLNet ...):

The problem arises when using:
* [ X] the official example scripts: (give details below)
running the language modeling example script

The tasks I am working on is:
* [ X] an official GLUE/SQUaD task: (give the name)
Language modeling
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Create conda enviorment. install transformers from the source.
2. Run language modeling example script

## Expected behavior
Example runs.


Adjust setup so that all extras run on Windows

# What does this PR do?

This removes some of the extra deps if they don't exist on Windows, so that the install doesn't fail.


should be BartConfig.prefix  None?

For bart-large-xsum/bart-large-cnn, currently set to `' '`

+  does not help evaluation performance vs setting to `None` (checked xsum and CNN)

+ It could help fine-tuning performance by serving as a work-around for the `add_prefix_space` issue?
Putting here in case others have thoughts.


Rename swish to silu to give appropriate credit

The swish was originally coined the "SiLU" in https://arxiv.org/pdf/1606.08415.pdf and https://arxiv.org/abs/1702.03118 long before the swish paper. Renaming other peoples' exact same ideas is unacceptable and huggingface's naming convention implicitly erases the research and work of people outside of Google.
This request inspired by a [discussion](https://www.reddit.com/r/MachineLearning/comments/hkiyir/r_google_has_a_credit_assignment_problem_in/) and a recent [tensorflow issue](https://github.com/tensorflow/tensorflow/issues/41066), but this problem has been brought up every few months for the past few years. In light of recent efforts to make the ML community more equitable and _fair_, this is a no-brainer and long overdue.

**Will this change the current api? How?**
The API would replace the "swish" argument with the "silu" argument and deprecate the swish.
[PyTorch 1.7](https://pytorch.org/docs/1.7.0/generated/torch.nn.SiLU.html?highlight=silu) added the SiLU. Tensorflow added the [SiLU](https://github.com/tensorflow/tensorflow/blob/27d26a8d86bceda282ad9ba3e3116a00759d4ebc/tensorflow/python/ops/nn_impl.py#L517) and should be in the next version. Jax has already added the SiLU;
jax.nn.swish will eventually be deprecated and jax.nn.silu will be added and both of the aforementioned papers will be cited in the documentation.


Reformer implementation in Tensorflow

# 🚀 Feature request

Since there is an implementation of the Reformer in Pytorch, my question is if there will be an implementation for Tensorflow too?


## Environment info
- `transformers` version: 3.4.0
- Platform: Linux-3.10.0-957.el7.x86_64-x86_64-with-debian-stretch-sid
- Python version: 3.6.9
- PyTorch version (GPU?): 1.6.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No
 @TevenLeScao

## Information
I am using TransfoXLModel. The problem arises when running the code below (if I do not fill in vocab_size=256, it works fine):
* the example scripts:
```python
from transformers import TransfoXLConfig, TransfoXLModel
configuration = TransfoXLConfig(vocab_size=256)
model = TransfoXLModel(configuration)
```

## Error I get:
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-323-7039580347ad> in <module>
      3 configuration = TransfoXLConfig(vocab_size=256)
      4 # Initializing a model from the configuration
----> 5 model = TransfoXLModel(configuration)

/opt/conda/lib/python3.6/site-packages/transformers/modeling_transfo_xl.py in __init__(self, config)
    736
    737         self.word_emb = AdaptiveEmbedding(
--> 738             config.vocab_size, config.d_embed, config.d_model, config.cutoffs, div_val=config.div_val
    739         )
    740

/opt/conda/lib/python3.6/site-packages/transformers/modeling_transfo_xl.py in __init__(self, n_token, d_embed, d_proj, cutoffs, div_val, sample_softmax)
    421                 l_idx, r_idx = self.cutoff_ends[i], self.cutoff_ends[i + 1]
    422                 d_emb_i = d_embed // (div_val ** i)
--> 423                 self.emb_layers.append(nn.Embedding(r_idx - l_idx, d_emb_i))
    424                 self.emb_projs.append(nn.Parameter(torch.FloatTensor(d_proj, d_emb_i)))
    425

/opt/conda/lib/python3.6/site-packages/torch/nn/modules/sparse.py in __init__(self, num_embeddings, embedding_dim, padding_idx, max_norm, norm_type, scale_grad_by_freq, sparse, _weight)
    107         self.scale_grad_by_freq = scale_grad_by_freq
    108         if _weight is None:
--> 109             self.weight = Parameter(torch.Tensor(num_embeddings, embedding_dim))
    110             self.reset_parameters()
    111         else:

RuntimeError: Trying to create tensor with negative dimension -199744: [-199744, 16]


[wip/s2s] Aggregate Rouge Deterministically

Take randomness/sampling out of `calculate_rouge_score`.
Not ready for merge as the default should be changed back to not using this.


Create README.md

model card

Related to pretrained model cards

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


Fix IterableDataset with __len__ in Trainer

# What does this PR do?
Fix #8087
Bring back support for `IterableDataset` with `__len__` in Trainer. Changed in #7858

@sgugger


Documentation error in question-answering pipeline

Hi,
The [QuestionAnsweringPipeline](https://huggingface.co/transformers/main_classes/pipelines.html?highlight=pipelines#transformers.QuestionAnsweringPipeline.__call__) is returning the start and end position of the context string and not according to "the tokenized version of the input." as mentioned in the doc.

```python
from transformers import AutoTokenizer, AutoModelForQuestionAnswering, pipeline
tokenizer = AutoTokenizer.from_pretrained("bert-large-uncased-whole-word-masking-finetuned-squad")
qa_pipeline = pipeline("question-answering")
text = r"""
🤗 Transformers (formerly known as pytorch-transformers and pytorch-pretrained-bert) provides general-purpose
architectures (BERT, GPT-2, RoBERTa, XLM, DistilBert, XLNet…) for Natural Language Understanding (NLU) and Natural
Language Generation (NLG) with over 32+ pretrained models in 100+ languages and deep interoperability between
TensorFlow 2.0 and PyTorch.
"""

questions = [
    "How many pretrained models?"
]

result = qa_pipeline(question=questions, context=text)
print(result)

#this is the correct answer
print(text[int(result["start"]):int(result["end"])])

#this is not correct
print(tokenizer.tokenize(text[int(result["start"]):int(result["end"])))
```

Sorry if I misunderstood the doc


Fully remove codecov

Fully removes codecov as we don't have any full test suite in circleci, alongside https://github.com/huggingface/transformers/commit/829b9f8cc321aa28396e6203e0f21eed26b132f7

If we want to put it back up, we can merge the two slow tests (TF + PT) and run coverage on that, but we should first take care of the inconsistencies in coverage as explained in https://github.com/huggingface/transformers/issues/6317

cc @stas00


Fix DeBERTa docs

Fix the DeBERTa docs


Fix assertion error message for MLflowCallback

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


Update README.md

Related to pretrained model cards


Create README.md

Related to pretrained model cards


#7858 breaks IterableDataset with __len__ in Trainer

https://github.com/huggingface/transformers/blob/08f534d2da47875a4b7eb1c125cfa7f0f3b79642/src/transformers/trainer.py#L381-L382

This used to be (before #7858)
```python
if isinstance(self.train_dataset, torch.utils.data.IterableDataset):
```

I am using IterableDataset with __len__ in Trainer. This change makes it return a sampler and results in an error later. `ValueError: DataLoader with IterableDataset: expected unspecified sampler option, but got sampler=<torch.utils.data.sampler.RandomSampler object at 0x7fa32c57b340>`

Maybe change to this?
```python
if (isinstance(self.train_dataset, torch.utils.data.IterableDataset) or
        not isinstance(self.train_dataset, collections.abc.Sized)):
```

@j-rossi-nl @sgugger


Hello world example fail with transformers-3.4

## Environment info

- `transformers` version:3.4
- Platform: Mac
- Python version: 3.6
- PyTorch version (GPU?):
- Tensorflow version (GPU?): 2.3
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger

## Information

Model I am using (Bert, XLNet ...): Bert

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

## To reproduce

Steps to reproduce the behavior:

Download all the code of branch-3.4.0

```
from transformers import AutoTokenizer, TFAutoModel

tokenizer = AutoTokenizer.from_pretrained("/Users/gt/Desktop/transformers-3.4.0/my_code/bert-base-uncased-tokenizer.json")
model = TFAutoModel.from_pretrained("/Users/gt/Desktop/transformers-3.4.0/my_code/bert-base-uncased-pytorch_model.bin")

inputs = tokenizer("Hello world!", return_tensors="tf")
outputs = model(**inputs)
print(outputs)
```

Model downloaded from https://mirrors.tuna.tsinghua.edu.cn/hugging-face-models/

Get error:
```
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte
```
in line 4


Merge pull request #1 from huggingface/master

Version track

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


Fix tf export path type in notebooks/04-onnx-export.ipynb


FastFormers into transformers

# 🌟 New model addition

## Model description

We just open-sourced [FastFormers](https://arxiv.org/abs/2010.13382) which are our SustaiNLP 2020 systems (FastFormers: Highly Efficient Transformer Models for Natural Language Understanding [paper](https://arxiv.org/abs/2010.13382)).

Currently, we are hosting this on our repository, but would like to merge it back to the transformers repository as an example.
our repo - https://github.com/microsoft/fastformers

For the purpose of the shared task, this is purely implemented with SuperGLUE data set.
So, it's dependent on Alex Wang(@W4ngatang)'s SuperGLUE data processing pipeline.
Also, many parts of the implementations are based on Alex'.
(https://github.com/W4ngatang/transformers/tree/superglue)

What would be the best way to merge this back?

## Open source status

* [x] the model implementation is available: https://github.com/microsoft/fastformers/blob/main/examples/fastformers/run_superglue.py
* [x] the model weights are available: demo systems are uploaded. https://github.com/microsoft/fastformers/releases/tag/v0.1-model
* [x] who are the authors: @ykim362


Fix doc examples

Fix many `{model_class}.from_pretrained())`  -> `{model_class}.from_pretrained()`. Hope it helps.

 documentation: @sgugger


Move style_doc to extra_quality_checks

Previously, if you have a doc style error in a .rst file,
```bash
python utils/style_doc.py $(modified_py_files) --max_len 119;
```
 wouldn't catch it


Pre style


Best practice to use this great repo for industry application.

# ❓ Questions & Help

To download all the source code and put them into pycharm or To use pip-install and use the API code?

## Details

I want to pretrain and fine-tune the models here on our own dataset.


Hope more GPT Chinese pretrained model.

# 🚀 Feature request

Hope more GPT Chinese pretrained model.

![image](https://user-images.githubusercontent.com/4702353/97251450-e0998e00-1842-11eb-9268-9dfa837f6b29.png)

Thank you very much.