CUDA Out of Memory After Several Epochs

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.1.1
- Platform: Linux-4.14.105-1-tlinux3-0013-x86_64-with-centos-7.2-Final
- Python version: 3.7.7
- PyTorch version (GPU?): 1.5.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: True
- Using distributed or parallel set-up in script?: using nn.data_parallel

### Who can help

- gpt2: @patrickvonplaten
- trainer: @sgugger

## Information

Model I am using (Bert, XLNet ...): GPT2

The problem arises when using:
* [- ] the official example scripts: run_clm.py

The tasks I am working on is:
* [ -] my own task or dataset: zh_wikitext

## To reproduce

The strange thing is that the scripts runs ok in the first 12 epochs, and ends with error in the middle of 12 epochs. I have checked that the trainer doesn't cache training loss tensor, so I am quite puzzled by the error. Any help are highly appreciated.

Steps to reproduce the behavior:

1. `python run_clm.py config.json`

Several useful config in `config.json` are:
```
block_size: 512
check_point_name: "gpt2_result/checkpoint-100000"
per_device_train_batch_size: 12
learning_rate: 0.00005
weight_decay: 0
adam_beta1: 0.9
adam_beta2: 0.98
adam_epsilon: 1e-8
max_grad_norm: 1
num_train_epochs: 50
max_steps: -1
warmup_steps: 0
```

Model Config are:
```
Model config GPT2Config {
  "activation_function": "gelu_new",
  "architectures": [
    "GPT2LMHeadModel"
  ],
  "attn_pdrop": 0.1,
  "bos_token_id": 50256,
  "embd_pdrop": 0.1,
  "eos_token_id": 50256,
  "gradient_checkpointing": false,
  "initializer_range": 0.02,
  "layer_norm_epsilon": 1e-05,
  "model_type": "gpt2",
  "n_ctx": 512,
  "n_embd": 768,
  "n_head": 12,
  "n_inner": null,
  "n_layer": 12,
  "n_positions": 512,
  "resid_pdrop": 0.1,
  "summary_activation": null,
  "summary_first_dropout": 0.1,
  "summary_proj_to_labels": true,
  "summary_type": "cls_index",
  "summary_use_proj": true,
  "task_specific_params": {
    "text-generation": {
      "do_sample": true,
      "max_length": 50
    }
  },
  "use_cache": true,
  "vocab_size": 21128
}
```

The tokenizer used is `BertTokenizer.from_pretrained('Bert-base-chinese')`.

The error log are following:
```
[INFO|trainer.py:703] 2021-02-10 11:30:39,997 >> ***** Running training *****
[INFO|trainer.py:704] 2021-02-10 11:30:39,997 >>   Num examples = 744899
[INFO|trainer.py:705] 2021-02-10 11:30:39,997 >>   Num Epochs = 50
[INFO|trainer.py:706] 2021-02-10 11:30:39,997 >>   Instantaneous batch size per device = 12
[INFO|trainer.py:707] 2021-02-10 11:30:39,997 >>   Total train batch size (w. parallel, distributed & accumulation) = 96
[INFO|trainer.py:708] 2021-02-10 11:30:39,997 >>   Gradient Accumulation steps = 1
[INFO|trainer.py:709] 2021-02-10 11:30:39,997 >>   Total optimization steps = 388000
[INFO|trainer.py:725] 2021-02-10 11:30:40,011 >>   Continuing training from checkpoint, will skip to saved global_step
[INFO|trainer.py:726] 2021-02-10 11:30:40,011 >>   Continuing training from epoch 12
[INFO|trainer.py:727] 2021-02-10 11:30:40,011 >>   Continuing training from global step 100002
  0%|                                                                                | 0/388000 [00:00<?, ?it/s]/data/miniconda3/lib/python3.7/site-packages/torch/nn/parallel/_functions.py:61: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
  warnings.warn('Was asked to gather along dimension 0, but all '
 26%|███████████████████████▋                                                                    | 100003/388000 [00:17<00:50, 5746.78it/s]Traceback (most recent call last):
  File "run_clm.py", line 321, in <module>
    main()
  File "run_clm.py", line 291, in main
    trainer.train(model_path=model_path)
  File "/data/miniconda3/lib/python3.7/site-packages/transformers/trainer.py", line 799, in train
    tr_loss += self.training_step(model, inputs)
  File "/data/miniconda3/lib/python3.7/site-packages/transformers/trainer.py", line 1139, in training_step
    loss = self.compute_loss(model, inputs)
  File "/data/miniconda3/lib/python3.7/site-packages/transformers/trainer.py", line 1163, in compute_loss
    outputs = model(**inputs)
  File "/data/miniconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/data/miniconda3/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 156, in forward
    return self.gather(outputs, self.output_device)
  File "/data/miniconda3/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 168, in gather
    return gather(outputs, output_device, dim=self.dim)
  File "/data/miniconda3/lib/python3.7/site-packages/torch/nn/parallel/scatter_gather.py", line 68, in gather
    res = gather_map(outputs)
  File "/data/miniconda3/lib/python3.7/site-packages/torch/nn/parallel/scatter_gather.py", line 62, in gather_map
    for k in out))
  File "<string>", line 9, in __init__
  File "/data/miniconda3/lib/python3.7/site-packages/transformers/file_utils.py", line 1412, in __post_init__
    for element in iterator:
  File "/data/miniconda3/lib/python3.7/site-packages/torch/nn/parallel/scatter_gather.py", line 62, in <genexpr>
    for k in out))
  File "/data/miniconda3/lib/python3.7/site-packages/torch/nn/parallel/scatter_gather.py", line 63, in gather_map
    return type(out)(map(gather_map, zip(*outputs)))
  File "/data/miniconda3/lib/python3.7/site-packages/torch/nn/parallel/scatter_gather.py", line 55, in gather_map
    return Gather.apply(target_device, dim, *outputs)
  File "/data/miniconda3/lib/python3.7/site-packages/torch/nn/parallel/_functions.py", line 68, in forward
    return comm.gather(inputs, ctx.dim, ctx.target_device)
  File "/data/miniconda3/lib/python3.7/site-packages/torch/cuda/comm.py", line 165, in gather
    return torch._C._gather(tensors, dim, destination)
RuntimeError: CUDA out of memory. Tried to allocate 288.00 MiB (GPU 0; 23.88 GiB total capacity; 22.53 GiB already allocated; 86.38 MiB free; 23.21 GiB reserved in total by PyTorch)
```


Possible bug in RAG Tokenizer

On this line
input_dict = tokenizer.prepare_seq2seq_batch(question, return_tensors="pt")
the following error is being generated
AttributeError: 'super' object has no attribute 'prepare_seq2seq_batch'


Bug in RAG Sequence generate

IMHO there is a bug in the RAG Sequence model, in the generate function.
The shapes mismatch all the time. I looked into the code and found the issue in the following loop.

https://github.com/huggingface/transformers/blob/85395e4901f87b880f364bcd6424fe37da94574b/src/transformers/models/rag/modeling_rag.py#L936

Kindly let me know if there is indeed a bug or is it just my code problem.
Thanks.


Fix tokenizers training in notebooks

The `train` method has been updated in `tokenizers` v0.10, and it includes a breaking change from the previous versions (reordered arguments). This modification ensures it works for all versions.

cc @sgugger @LysandreJik


Git does not find the model folder and does not commit model files in the hugging face

I work for Google Colab. This is how I save my training model files
````
trainer.save_model("./kvantorium-small")
tokenizer.save_pretrained("/content/For_tokenize", legacy_format=False)
````

Next, I want to commit my files to the hugging face repository. As shown in the guide (https://huggingface.co/welcome)

all these lines of code compile successfully:

````
!sudo apt-get install git-lfs
!pip install huggingface_hub
!huggingface-cli login
!huggingface-cli repo create simple-small-kvantorium

!git lfs install
!git clone https://huggingface.co/Fidlobabovic/simple-small-kvantorium
````
But when I want to make a push request to save files to the repository, I get an error that there is no such repository. How do I rewrite the request to publish the files to the repository? Is this a problem that I work for Google Colab? Thanks a lot in advance, you are helping me a lot

#9878

````
!git add .
!git commit -m "commit from $Fidlobabovic/simple-small-kvantorium"
!git push

fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
````


Non-JSON-serializable tokenizer config with `save_pretrained`

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.3.1
- Platform: Linux
- Python version: 3.7.9
- PyTorch version (GPU?): 1.7.1 (GPU)
- Tensorflow version (GPU?): 2.1.2 (GPU)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
@LysandreJik
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @jplu

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

## Information

Model I am using (Bert, XLNet ...): Bert

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

Using a minimal example with loading/saving a tokenizer.

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

Again, this is just a minimal example.

## To reproduce

Steps to reproduce the behavior:

1. Instantiate a `BertConfig` and a `BertTokenizer` based on the config.
2. Try and save the tokenizer with `save_pretrained`

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

Minimal example:

```
from transformers import BertConfig, BertTokenizer

config = BertConfig.from_pretrained("./configs/bert-small.json", cache_dir=".")
tokenizer = BertTokenizer.from_pretrained("vocab/", cache_dir=".", config=config)
tokenizer.save_pretrained('new_save')
```

Error:

```
Traceback (most recent call last):
  File "test.py", line 5, in <module>
    tokenizer.save_pretrained('new_save')
  File "/cluster/envs/mult/lib/python3.7/site-packages/transformers/tokenization_utils_base.py", line 1979, in save_pretrained
    f.write(json.dumps(tokenizer_config, ensure_ascii=False))
  File "/cluster/envs/mult/lib/python3.7/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/cluster/envs/mult/lib/python3.7/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/cluster/envs/mult/lib/python3.7/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/cluster/envs/mult/lib/python3.7/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type BertConfig is not JSON serializable
```

## Expected behavior

Tokenizer should be saveable. I'm guessing this could be happening because the bit that's supposed to be saving the config is using the `json` library directly, instead of calling `to_json_file` on the `BertConfig`, but I'm not sure.

<!-- A clear and concise description of what you would expect to happen. -->


Remove speed metrics from default compute objective [WIP]

# What does this PR do?

This PR removes speed metrics (e.g. `eval_runtime`) from the default compute objective (`default_compute_objective`). `default_compute_objective` is used when no `compute_objective` is passed to `Trainer.hyperparameter_search`. `Trainer` adds speed metrics such as `eval_runtime` and `eval_samples_per_second` to the metrics and `default_compute_objective` returns the sum of metrics as the objective so these speed metrics will be included in the objective.

I still need to add unit test for `default_compute_objective` to avoid having such metrics in the objective in the future.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?
@sgugger


Revert "Fix TFConvBertModelIntegrationTest::test_inference_masked_lm Test"

Reverts huggingface/transformers#10104


PruneTrain: Fast Neural Network Training by Dynamic Sparse Model Reconfiguration

Feature request

Request for a new feature

# 🚀 Feature request

PruneTrain. {...} By using a structured-pruning approach and additional reconfiguration techniques we introduce, the pruned model can still be efficiently processed on a GPU accelerator. Overall, **PruneTrain achieves a reduction of 39% in the end-to-end training time of ResNet50 for ImageNet by reducing computation cost by 40% in FLOPs, memory accesses by 37% for memory bandwidth bound layers, and the inter-accelerator communication by 55%.**

## Motivation

I'm pre-training some midsize language models from scratch. If you tell me that I can pretrain a network with 1% drop in performance while cutting down the energy demand of the training by up to 40% and speeding inference time at the same time, I will buy it.

## Your contribution

https://arxiv.org/abs/1901.09290. I can not understand why the authors did not open source the code, since it could reduce the global warming, speedup experimentation and reduce energy consumption.


Fix TFConvBertModelIntegrationTest::test_inference_masked_lm Test


Fix Faiss Import

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

All RAG related tests are skipped on circle ci at the moment because `faiss-cpu` is not passing the `is_faiss_available()` function. Sadly this didn't make us realize that RAG is currently broken on master. This should be merged with https://github.com/huggingface/transformers/pull/10094 to fix RAG.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @jplu

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Replace faiss cpu by faiss

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @jplu

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Change dependency from faiss-cpu to faiss


Fix some edge cases in report_to and add deprecation warnings

# What does this PR do?

This PR adds two new values for the `report_to` TrainingArguments:
- "all" for all integrations installed
- "none" for none (necessary when using in the CLI and we can't pass an empty list)

It also starts warning the user (with an info to not be too spammy) of the upcoming change of default in v5.


Issue training Longformer

Hello, apologies if this is the wrong place to ask for help, I'm currently trying to fine-tune longformer on a text classification task. My script is below.

When I use ```for param in model.longformer.encoder.parameters():
        param.requires_grad = False``` to not train the encoder layer but just the classification head and the embeddings, training works as expected. When I don't freeze the encoder layers, the model doesn't train at all, and when I try to do inference on it, it gives constant output, regardless of what data I put in. I've been reading all the papers to find what I have done wrong, can anyone point me in the right direction? Thank you so much for your help! Tom

```import logging
import pandas as pd
from transformers import AdamW, LongformerTokenizerFast, TrainingArguments, Trainer,LongformerForSequenceClassification
import torch
from torch.utils.data import DataLoader
import numpy as np

from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score

def compute_metrics(pred):
  labels = pred.label_ids
  preds = pred.predictions.argmax(-1)
  # calculate accuracy using sklearn's function
  acc = accuracy_score(labels, preds)
  f1 = f1_score(labels,preds)
  precision = precision_score(labels,preds)
  recall = recall_score(labels,preds)
  return {
      'accuracy': acc,
      'f1': f1,
      'precision': precision,
      'recall': recall
  }

class SupremeDataset(torch.utils.data.Dataset):
    def __init__(self, encodings, labels):
        self.encodings = encodings
        self.labels = labels

    def __getitem__(self, idx):
        item = {key: torch.tensor(val[idx]) for key, val in self.encodings.items()}
        item['labels'] = torch.tensor(self.labels[idx])
        return item

    def __len__(self):
        return len(self.labels)

def main():

    # Setup logging:
    logging.basicConfig(
        format="%(asctime)s - %(levelname)s - %(name)s -   %(message)s",
        datefmt="%m/%d/%Y %H:%M:%S",
        level=logging.INFO,
    )

    logging.info("*** Data processing ***")

    logging.info("importing data")
    data_train = pd.read_csv("../../../shared/benchmarking/supreme_train.csv").dropna()
    data_val = pd.read_csv("../../../shared/benchmarking/supreme_val.csv").dropna()

    tokenizer = LongformerTokenizerFast.from_pretrained('allenai/longformer-base-4096')

    logging.info("tokenizing data")

    train_encodings = tokenizer(list(data_train.content_decode),truncation=True,padding=True,return_tensors="pt")
    val_encodings = tokenizer(list(data_val.content_decode),truncation=True,padding=True,return_tensors="pt")

    train_encodings['global_attention_mask'] = torch.zeros_like(train_encodings['input_ids'])
    val_encodings['global_attention_mask'] = torch.zeros_like(val_encodings['input_ids'])

    train_encodings['global_attention_mask'][train_encodings['input_ids']==0] = 1
    val_encodings['global_attention_mask'][val_encodings['input_ids']==0] = 1

    train_labels = data_train.label.tolist()
    val_labels = data_val.label.tolist()

    logging.info("creating datasets")

    train_dataset = SupremeDataset(train_encodings, train_labels)
    val_dataset = SupremeDataset(val_encodings, val_labels)

    logging.info("*** Training ***")

    training_args = TrainingArguments(
    output_dir='./results',          # output directory
    num_train_epochs=3,              # total number of training epochs
    per_device_train_batch_size=1,  # batch size per device during training
    per_device_eval_batch_size=1,   # batch size for evaluation
    warmup_steps=500,                # number of warmup steps for learning rate scheduler
    weight_decay=0.01,               # strength of weight decay
    logging_dir='./logs',            # directory for storing logs
    logging_steps=200,
    do_eval=True,
    load_best_model_at_end=True,
    metric_for_best_model="accuracy",
    evaluation_strategy = "steps",
    )

    logging.info("loading model")

    model = LongformerForSequenceClassification.from_pretrained('allenai/longformer-base-4096')

    for param in model.longformer.encoder.parameters():
        param.requires_grad = False

    logging.info("loading trainer")

    trainer = Trainer(
    model=model,                         # the instantiated 🤗 Transformers model to be trained
    args=training_args,                  # training arguments, defined above
    train_dataset=train_dataset,         # training dataset
    eval_dataset=val_dataset,
    compute_metrics = compute_metrics             # evaluation dataset
    )

    logging.info("starting training")

    trainer.train()

    torch.save(model, 'supremecourt_fullmodel.pt')

if __name__ == "__main__":
    main()
```


Adding support for TFEncoderDecoderModel

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

This PR will add tf2 support for EncoderDecoderModel upon completion.

<!-- Remove if not applicable -->

Fixes #9863

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

@patrickvonplaten
<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @jplu

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


DeBERTa v2 throws "TypeError: stat: path should be string...", v1 not

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.3.1
- Platform: Linux-5.4.0-54-generic-x86_64-with-glibc2.29
- Python version: 3.8.5
- PyTorch version (GPU?): 1.7.1+cpu (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: false
- Using distributed or parallel set-up in script?: false

### Who can help

@BigBird01 @patil-suraj

## Information

Model I am using (DeBERTa v2):

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Create this file:
```
from transformers import AutoTokenizer, AutoModel
import torch

tokenizer = AutoTokenizer.from_pretrained('microsoft/deberta-xlarge-v2')
model = AutoModel.from_pretrained('microsoft/deberta-xlarge-v2')

inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
outputs = model(**inputs)

last_hidden_states = outputs.last_hidden_state
print(outputs)
```

2. Run the file

3. You'll get:
```
(venv) root@16gb:~/deberta# python3 index.py
Traceback (most recent call last):
  File "index.py", line 4, in <module>
    tokenizer = AutoTokenizer.from_pretrained('microsoft/deberta-xlarge-v2')
  File "/root/deberta/venv/lib/python3.8/site-packages/transformers/models/auto/tokenization_auto.py", line 398, in from_pretrained
    return tokenizer_class_py.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
  File "/root/deberta/venv/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 1788, in from_pretrained
    return cls._from_pretrained(
  File "/root/deberta/venv/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 1860, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
  File "/root/deberta/venv/lib/python3.8/site-packages/transformers/models/deberta/tokenization_deberta.py", line 542, in __init__
    if not os.path.isfile(vocab_file):
  File "/usr/lib/python3.8/genericpath.py", line 30, in isfile
    st = os.stat(path)
TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType
```

I tried this with the DeBERTa v1 models and there was no error. I've the same behavior when using `DebertaTokenizer, DebertaModel`

## Expected behavior

No error.


Fix example in Wav2Vec2 documentation

Fixes an example in Wav2Vec2 documentation


Fix naming in TF MobileBERT

# What does this PR do?

This PR fixes a naming issue in the `TFMobileBertForMaskedLM` model.

# Fixes

#10088


[RAG] fix generate

# What does this PR do?

#9984 introduced a new `encoder_no_repeat_ngram_size`  `generate` param, but it was missing from `RagTokenForGeneration.generate`,  it's a required argument for `_get_logits_processor` which is called inside `RagTokenForGeneration.generate`.

This PR adds the argument to `RagTokenForGeneration.generate` and passes it to `_get_logits_processor`.


Pre-Training for Question Generation

Hi,
How to pre-train any of language generation models (T5, BART or GPT ) for Question Generation task where I had passage, question and answer?


Logging propagation

This PR enables logs propagation by default with transformers' logging system, in a similar fashion to https://github.com/huggingface/datasets/pull/1845.

Unlike `datasets` however, we will not remove the default handler in transformers' logging system: this handler is heavily used in all examples, and removing the default handler would prevent the formatting from being correctly applied to the examples.
Since this is the best practice shown in examples, I believe this change would be breaking for users that have copy/pasted this change across their codebases, and this change would therefore be breaking for these users.

Furthermore, any user that does not want the default handler may use the `disable_default_handler` method in order to disable that behavior. These two methods are added to the documentation in this PR.

cc @lhoestq @sgugger @patrickvonplaten


How to run distributed training on multiple machines?

## Environment info

- `transformers` version: 4.3.0
- Platform: PyTorch
- Python version: 3.7
- PyTorch version (GPU?): 1.7.1
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Yes

##Who can help:
Examples:
- maintained examples (not research project or legacy): @sgugger, @patil-suraj

## Information

Model I am using (Bert, XLNet ...): Roberta

## To reproduce
The script I'm working with is https://github.com/huggingface/transformers/blob/master/examples/language-modeling/run_mlm.py
I know that we can run the distributed training on multiple GPUs in a single machine by
`python -m torch.distributed.launch --nproc_per_node=8 run_mlm.py --sharded_dpp`

But what if I can multiple machines with multiple GPUs, let's say I have two machines and each is with 8 GPUs, what is the expected command to run on these 16 GPUs?


[question] Are the tensorflow bert weights same as the original repo ?

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- `transformers` version: 4.2.2
- Platform: Linux-4.15.0-133-generic-x86_64-with-debian-stretch-sid
- Python version: 3.6.8
- PyTorch version (GPU?): not installed (NA)
- Tensorflow version (GPU?): 2.5.0-dev20210204 (True)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: distributed

### Who can help

Models:

- albert, bert, xlm: @LysandreJik

## Information
I'm using the pretrained bert-base-chinese [here](https://huggingface.co/bert-base-chinese). I print out the pooler_output and the result is different from the tensorflow2.0 published hub saved model [here](https://tfhub.dev/tensorflow/bert_zh_L-12_H-768_A-12/3).
I want to confirm whether the two checkpoints weights are the same.

## To reproduce

Steps to reproduce the behavior:
```
from transformers import TFBert
bert = bert = TFBertModel.from_pretrained('bert-base-chinese', output_hidden_states=True)
output = bert(input_ids=tf.convert_to_tensor([[ 101,  791, 1921, 1921, 3698, 2582,  720, 3416,  102]]),
                   attention_mask=tf.convert_to_tensor([[1, 1, 1, 1, 1, 1, 1, 1, 1]]), training=False)
print(output.pooler_output)
```
```
# print the beginning weights:
array([[ 0.99749047,  0.9999622 ,  0.99657625,  0.96953416,  0.8489984 ,
         0.06474952,
```

The tensorflow hub output can be produced from https://tfhub.dev/tensorflow/bert_zh_L-12_H-768_A-12/3.
The input text is "今天天气怎么样"
```
[[ 9.97488916e-01  9.99962687e-01  9.96489942e-01  9.69992220e-01
   8.49602520e-01  6.62192404e-02 ,
```

The tensorflow part claims they use the original bert checkpoints from tf1.X.
There is no training=True/False option in tensorflow hub so I'm confused if the difference is due to this option? (I've set trainable=False in tensorflow hub)

## Expected behavior
Expect the outputs to be the same.


Deprecate Wav2Vec2ForMaskedLM and add Wav2Vec2ForCTC

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Deprecates `Wav2Vec2ForMaskedLM` -> the name was very badly chosen since it's currently used for CTC classification which is very different from `MaskedLM`. Also `MaskedLM` is not a good name for pretraining where it should rather be something like `ForMaskedSpeechModeling`, so IMO the best idea is to deprecate the whole class.

Right after this PR is merged and there is a patch, I will update all configs.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @jplu

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Language modelling head has zero weights in pretrained TFMobileBertForMaskedLM

## Description
The `TFMobileBertForMaskedLM` example returns all zero logits while `MobileBertForMaskedLM` example works fine.
https://huggingface.co/transformers/model_doc/mobilebert.html#tfmobilebertformaskedlm

I checked language modeling head weights for both models and found that TF pretrained model has zero weights.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.3.0
- Platform: Google Colab
- Python version: 3.6.9
- PyTorch version (GPU?): 1.7.0+cu101
- Tensorflow version (GPU?): 2.4.1
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
@jplu @patrickvonplaten

## Information

Model I am using (Bert, XLNet ...): TFMobileBertForMaskedLM

The problem arises when using:
- [x] the official example scripts: (give details below)

## To reproduce

```
from transformers import MobileBertForMaskedLM
from transformers import TFMobileBertForMaskedLM

# PyTorch
model = MobileBertForMaskedLM.from_pretrained('google/mobilebert-uncased')
print(list(model.cls.parameters())[0])

# Parameter containing:
# tensor([-7.2946, -7.4302, -7.5401,  ..., -7.4850, -7.4503, -2.7798],
#       requires_grad=True)

# TensorFlow
model = TFMobileBertForMaskedLM.from_pretrained('google/mobilebert-uncased')
print(model.layers[1].get_weights()[0])

# array([0., 0., 0., ..., 0., 0., 0.], dtype=float32)
```
## Expected behavior
Language modeling head for TFMobileBertForMaskedLM should have the same weights as of MobileBertForMaskedLM


remove adjust_logits_during_generation method

# What does this PR do?

This PR is the first split of #9811.

This PR
1.  introduces two new `generate`  and `config` arguments  and `LogitsProcessor`
    - `forced_bos_token_id` and `forced_eos_token_id`,  to force a specific start and end token. This is particularly useful for many to many and one to many translation models, so we can pass different language tokens as `forced_bos_token_id` to `generate`,
    - `ForcedBOSTokenLogitsProcessor` and `ForcedEOSTokenLogitsProcessor`
2. Remove `adjust_logits_during_generation` method from all models (except `Marian`) and handle that use case using the newly introduced logits processors.
4. remove the `force_bos_token_to_be_generated` argument from `BartConfig`

For `Marian` we still need to keep the `adjust_logits_during_generation` method to force the model to not generate pad token. Adding the pad token to `bad_words_ids` does not resolve this issue, the score of `pad_token_id` needs to be set to `-inf` before calling `log_softmax`


doc: update W&B related doc

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Updates W&B related documentation:
* remove outdated examples
* update urls
* add config parameters

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @jplu

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->
@sgugger


[examples/s2s] add test set predictions

# What does this PR do?

This PR adds the `do_predict` option to the `run_seq2seq.py` script for test set predictions.

Fixes #10032

cc. @stas00


Tapas not working with tables exceeding token limit

## Environment info

- `transformers` version: 4.3.0
- Platform: MacOS
- Python version: 3.7
- PyTorch version (GPU?): 1.7.1
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
@LysandreJik @sgugger @NielsRogge

## Information

Model I am using (Bert, XLNet ...): TaPas

## To reproduce
When executing the following code, using this [table](https://gist.github.com/bogdankostic/387d1c7a0e8ce25ea302395756df11b3), I get an `IndexError: index out of range in self`.
```python
from transformers import AutoTokenizer, AutoModelForTableQuestionAnswering
import pandas as pd

tokenizer = AutoTokenizer.from_pretrained("google/tapas-base-finetuned-wtq", drop_rows_to_fit=True)
model = AutoModelForTableQuestionAnswering.from_pretrained("google/tapas-base-finetuned-wtq")

df = pd.read_csv("table.tsv", sep="\t").astype(str)

queries = ["How big is Ardeen?"]
inputs = tokenizer(table=df, queries=queries, padding="max_length", truncation=True, return_tensors="pt")

outputs = model(**inputs)
```

I am not completely sure about the cause of the error but I suspect that the column rank vectors are not correctly generated. (`torch.max(token_type_ids[:, :, 4])` returns 298 and `torch.max(token_type_ids[:, :, 5])` returns 302, the Embedding Models for column rank and inverse column rank, however, allow a max value of 255)


model.generate needs BART config update

### Who can help
@patrickvonplaten @patil-suraj

```
model = BartForConditionalGeneration.from_pretrained('facebook/bart-large-cnn')

tokenizer = BartTokenizer.from_pretrained('facebook/bart-large-cnn')

ARTICLE_TO_SUMMARIZE = "My friends are cool but they eat too many carbs."
inputs = tokenizer([ARTICLE_TO_SUMMARIZE], max_length=1024, return_tensors='pt')

# Generate Summary
summary_ids = model.generate(inputs['input_ids'], num_beams=4, max_length=5, early_stopping=True)
```

model.generate runs into errors. num_beam_groups, return_dict_in_generate and encoder_no_repeat_ngram_size are not defined in BART config. Should they be added?


Supporting truncation from both ends of the sequence in BertTokenizerFast

Request for a new feature

# 🚀 Feature request

For `BertTokenizerFast` (inherited from `PreTrainedTokenizerFast`), it seems like `__call__` only supports truncating from the end of the sequences if we set `truncation` to be `longest_first`, `only_first` or `only_second`. For example, assuming `max_length` is 6 and `truncation` is `longest_first`:

(`I have a pen`, `I have an apple`) --> truncation --> (`I have a`, `I have an`)

However, if we take a closer look at [Google's original data-preprocessing script for BERT](https://github.com/google-research/bert/blob/master/create_pretraining_data.py#L430), truncation can happen at both ends of the sequence randomly:

(`I have a pen`, `I have an apple`) --> truncation --> (`I have a`, `have an apple`) or (`have a pen`, `I have an`) or (`I have a`, `I have an`) or (`have a pen`, `have an apple`)

For `BertTokenizer`, perhaps I could reassigned its `truncate_sequences` member function (https://github.com/huggingface/transformers/blob/master/src/transformers/tokenization_utils_base.py#L2887) with a new function that implements Google's truncation scheme; however, for `BertTokenizerFast`, truncation is handled completely in Rust, about which I can't do anything.

An alternative is to call `tokenize` first, then truncate the sequence using Google's scheme, ~~then call `__call__` and passing `is_split_into_words` as `True`~~. However, this approach has significant performance impact comparing to calling `__call__` on a batch of sequences directly (the average total tokenization latency doubled in our experiments).

> PS: Turned out `is_split_into_words` doesn't work this way (since when it sees a subword `##abc`, `__call__` would further tokenize it into `#` `#` `abc` even if `is_split_into_words==True`). Thus, the actual (but slow) alternative is to 1) call `tokenize` 2) implement the truncation scheme and making sure a subword starting with `##` won't be at the boundary 3) call `convert_tokens_to_string` 4) call `__call__`. Effectively, this alternative tokenizes the same sequence twice.

I'm wondering if's possible to add official support for random truncation from both ends of the sequence?

## Motivation

To match Google's truncation scheme exactly and minimizing artificial impacts on pretraining convergence.

## Your contribution

Unfortunately I'm not very familiar with Rust (I can read it, but I neve learned/wrote Rust before), thus I can't help much.


pipeline("sentiment-analysis') - index out of range in self

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.2.2
- Platform:  Manjaro Linux (Feb 2021)
- Python version: 3.8.5
- PyTorch version (GPU?): 1.7.1 (GPU)
- Tensorflow version (GPU?):
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.-->

Library:

- tokenizers: @n1t0, @LysandreJik
- pipelines: @LysandreJik

## Information

Model I am using (Bert, XLNet ...): distilbert-base-uncased-finetuned-sst-2-english

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: sentiment analysis
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

My dataset consists blog articles and comments on them. Sometimes there are  non-english characters, code snippets or other weird sequences.

Error occurs when:

1. Initialize the default pipeline("sentiment-analysis") with device 0 or -1
2. Run inference classifier with truncation=True of my dataset
3. After some time the classifier returns the following error:

CPU: `Index out of range in self`

GPU:  ``/opt/conda/conda-bld/pytorch_1607370172916/work/aten/src/ATen/native/cuda/Indexing.cu:658: indexSelectLargeIndex: block: [56,0,0], thread: [0,0,0] Assertion `srcIndex < srcSelectDimSize` failed.``

## Expected behavior

I thought at first that my data was messing up the tokenization process or the model because sometimes there are strange sequences in the data e.g. code, links or stack traces.

However, if you name the model and tokenizer during pipeline initialization, inference from the model works fine for the same data:

`classifier = pipeline('sentiment-analysis', model='distilbert-base-uncased-finetuned-sst-2-english', tokenizer='distilbert-base-uncased', device=0)`


[deepspeed tests] transition to new tests dir

as discussed at https://github.com/huggingface/transformers/issues/10076 relocating deepspeed tests a dedicated area and out of the scripts area.

I went right ahead and create a dedicated sub-folder for deepspeed tests.

I no longer can use the libraries from `seq2seq` since I can't do a relative import from the script.

The only thing is that I will need to update all of my comments/posts to mention that `ds_config.json` has moved.

fairscale will be probably next if this looks good.

Fixes: https://github.com/huggingface/transformers/issues/10076

@sgugger


Unclear error "NotImplementedError:  "while saving tokenizer. How fix it?

Here is my tokenizer code and how I save it to a json file" /content/bert-datas7.json"

````
from tokenizers import normalizers
from tokenizers.normalizers import Lowercase, NFD, StripAccents

bert_tokenizer.pre_tokenizer = Whitespace()

from tokenizers.processors import TemplateProcessing

bert_tokenizer.post_processor = TemplateProcessing(
    single="[CLS] $A [SEP]",
    pair="[CLS] $A [SEP] $B:1 [SEP]:1",
    special_tokens=[
        ("[CLS]", 1),
        ("[SEP]", 2),
        ("[PAD]", 3),
    ],

)
from tokenizers.trainers import WordPieceTrainer

trainer = WordPieceTrainer(
    vocab_size=30522, special_tokens=["[UNK]", "[CLS]", "[SEP]", "[PAD]", "[MASK]"], pad_to_max_length=True
)
files = [f"/content/For_ITMO.txt" for split in ["test", "train", "valid"]]
bert_tokenizer.train(trainer, files)

model_files = bert_tokenizer.model.save("data", "/content/For_ITMO.txt")

bert_tokenizer.model = WordPiece.from_file(*model_files, unk_token="[UNK]",  pad_to_max_length=True)

bert_tokenizer.save("/content/bert-datas7.json")
````

When I output tokenizer name_or_path = nothing is displayed. This is normal?

````
tokenizer = PreTrainedTokenizerFast(tokenizer_file='/content/bert-datas7.json')
tokenizer.add_special_tokens({'pad_token': '[PAD]'})

print(tokenizer)
>>> PreTrainedTokenizerFast(name_or_path='', vocab_size=1435, model_max_len=1000000000000000019884624838656, is_fast=True, padding_side='right', special_tokens={'pad_token': '[PAD]'})
````
Also, when I try to save my tokenizer, I get an error without explanation. How can I rewrite the code so that all this???
#9658
#10039
[For_ITMO.txt-vocab (1) (1).txt](https://github.com/huggingface/transformers/files/5945659/For_ITMO.txt-vocab.1.1.txt)

````
tokenizer.save_pretrained("/content/tokennizerrrr")

NotImplementedError                       Traceback (most recent call last)
<ipython-input-11-efc48254a528> in <module>()
----> 1 tokenizer.save_pretrained("/content/tokennizerrrr")

2 frames
/usr/local/lib/python3.6/dist-packages/transformers/tokenization_utils_base.py in save_vocabulary(self, save_directory, filename_prefix)
   2042             :obj:`Tuple(str)`: Paths to the files saved.
   2043         """
-> 2044         raise NotImplementedError
   2045
   2046     def tokenize(self, text: str, pair: Optional[str] = None, add_special_tokens: bool = False, **kwargs) -> List[str]:

NotImplementedError:
````


Replace strided slice with tf.expand_dims

# What does this PR do?

This PR aims to replace the strided slice notation by its TF operator counterpart. As proposed by @mfuntowicz https://github.com/huggingface/transformers/pull/9890#discussion_r571939682


Update tokenizers requirement

Bump `tokenizers` version requirement to use the latest release, and also accept any new version before the next one possibly breaking.


[tests] where to put deepspeed + fairscale tests

As a split off from this comment https://github.com/huggingface/transformers/pull/10039#pullrequestreview-585482462 we need to find a new home for deepspeed + fairscale tests.

Currently there are under `examples/seq2seq` because they rely on `finetune_trainer.py` ( `run_seq2seq.py` once the transition is over).

@sgugger suggests to keep the `seq2seq` folder as simple as possible. We also have `ds_config.json` there that could be moved too.

Seeing what's happening in the fairscale land - I think we will need a bunch of various tests there in the future too.

So where should we put the deepspeed + fairscale tests?

Ideally they should be put under main `tests`, since they are part of the trainer core, but I'm not sure whether reaching across the tests suite is a clean approach.

My fantasy is that one day transformers will have a few essential tools that aren't examples, and those will then leave somewhere in the main tree, perhaps `src/transformers/apps` and then it'd be easy to have such tests under `tests`.

So suggestions for now:

1. create `examples/deepspeed` and `examples/fairscale`
2. create `examples/distributed` and perhaps have all those extensions tested in one folder
3. create a new 3rd test suite for integrations
4. create `tests/deepspeed` - but as voiced earlier I'm not sure how reaching across a test suite will work - need to try - also this proposes to change the current flat structure of `tests`.

Perhaps you have other ideas.

@sgugger, @patrickvonplaten, @LysandreJik, @patil-suraj


Trying to train a binary classifier over sentence pairs with custom dataset throws a Tensroflow error.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: `4.2.2`
- Platform: Ubuntu 18.04
- Python version: `3.7.5`
- PyTorch version (GPU?):
- Tensorflow version (GPU): `2.3.1`
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Nope

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @jplu

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

## Information

Model I am using (TFRoberta, TFXLMRoberta...):

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: https://huggingface.co/transformers/training.html#fine-tuning-in-native-tensorflow-2
* [x] my own task or dataset:

## To reproduce

Steps to reproduce the behavior:
```python
from transformers import TFAutoModelForSequenceClassification, AutoTokenizer
from keras.metrics import Precision, Recall
import tensorflow as tf

def build_dataset(tokenizer, filename):
    data = [[], [], []]
    with open(filename, 'r') as file_:
        for line in file_:
            fields = line.split('\t')
            data[0].append(fields[0].strip())
            data[1].append(fields[1].strip())
            data[2].append(int(fields[2].strip()))
    sentences = tokenizer(data[0], data[1],
                          padding=True,
                          truncation=True)

    return tf.data.Dataset.from_tensor_slices((dict(sentences),
                                             data[2]))
settings = {
    "model": 'roberta-base',
    "batch_size": 8,
    "n_classes": 1,
    "epochs": 10,
    "steps_per_epoch": 128,
    "patience": 5,
    "loss": "binary_crossentropy",
    "lr": 5e7,
    "clipnorm": 1.0,
}
tokenizer = AutoTokenizer.from_pretrained(settings["model"])

train_dataset = build_dataset(tokenizer, 'train.head')
train_dataset = train_dataset.shuffle(
        len(train_dataset)).batch(settings["batch_size"])

dev_dataset = build_dataset(tokenizer, 'dev.head').batch(
        settings["batch_size"])

model = TFAutoModelForSequenceClassification.from_pretrained(
            settings['model'],
            num_labels=1)
model.compile(optimizer='adam',
              #loss='binary_crossentropy',
              loss=model.compute_loss,
              metrics=[Precision(name='p'), Recall(name='r')])
model.summary()
model.fit(train_dataset,
          epochs=settings["epochs"],
          #steps_per_epoch=steps_per_epoch,
          validation_data=dev_dataset,
          batch_size=settings["batch_size"],
          verbose=1)
```

Gives the following output
```
Some layers of TFRobertaForSequenceClassification were not initialized from the model checkpoint at roberta-base and are newly initialized: ['classifier']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Model: "tf_roberta_for_sequence_classification"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
roberta (TFRobertaMainLayer) multiple                  124055040
_________________________________________________________________
classifier (TFRobertaClassif multiple                  591361
=================================================================
Total params: 124,646,401
Trainable params: 124,646,401
Non-trainable params: 0
_________________________________________________________________
Epoch 1/10
The parameters `output_attentions`, `output_hidden_states` and `use_cache` cannot be updated when calling a model.They have to be set to True/False in the config object (i.e.: `config=XConfig.from_pretrained('name', output_attentions=True)`).
The parameter `return_dict` cannot be set in graph mode and will always be set to `True`.
The parameters `output_attentions`, `output_hidden_states` and `use_cache` cannot be updated when calling a model.They have to be set to True/False in the config object (i.e.: `config=XConfig.from_pretrained('name', output_attentions=True)`).
The parameter `return_dict` cannot be set in graph mode and will always be set to `True`.
Traceback (most recent call last):
  File "finetune.py", line 52, in <module>
    verbose=1)
  File "/work/user/bicleaner-neural/venv/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py", line 108, in _method_wrapper
    return method(self, *args, **kwargs)
  File "/work/user/bicleaner-neural/venv/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py", line 1098, in fit
    tmp_logs = train_function(iterator)
  File "/work/user/bicleaner-neural/venv/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 780, in __call__
    result = self._call(*args, **kwds)
  File "/work/user/bicleaner-neural/venv/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 840, in _call
    return self._stateless_fn(*args, **kwds)
  File "/work/user/bicleaner-neural/venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 2829, in __call__
    return graph_function._filtered_call(args, kwargs)  # pylint: disable=protected-access
  File "/work/user/bicleaner-neural/venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 1848, in _filtered_call
    cancellation_manager=cancellation_manager)
  File "/work/user/bicleaner-neural/venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 1924, in _call_flat
    ctx, args, cancellation_manager=cancellation_manager))
  File "/work/user/bicleaner-neural/venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 550, in call
    ctx=ctx)
  File "/work/user/bicleaner-neural/venv/lib/python3.7/site-packages/tensorflow/python/eager/execute.py", line 60, in quick_execute
    inputs, attrs, num_outputs)
tensorflow.python.framework.errors_impl.InvalidArgumentError: 2 root error(s) found.
  (0) Invalid argument:  assertion failed: [predictions must be >= 0] [Condition x >= y did not hold element-wise:] [x (tf_roberta_for_sequence_classification/classifier/out_proj/BiasAdd:0) = ] [[0.153356239][0.171548933][0.121127911]...] [y (Cast_3/x:0) = ] [0]
         [[{{node assert_greater_equal/Assert/AssertGuard/else/_1/assert_greater_equal/Assert/AssertGuard/Assert}}]]
         [[assert_greater_equal_1/Assert/AssertGuard/pivot_f/_31/_205]]
  (1) Invalid argument:  assertion failed: [predictions must be >= 0] [Condition x >= y did not hold element-wise:] [x (tf_roberta_for_sequence_classification/classifier/out_proj/BiasAdd:0) = ] [[0.153356239][0.171548933][0.121127911]...] [y (Cast_3/x:0) = ] [0]
         [[{{node assert_greater_equal/Assert/AssertGuard/else/_1/assert_greater_equal/Assert/AssertGuard/Assert}}]]
0 successful operations.
0 derived errors ignored. [Op:__inference_train_function_20780]

Function call stack:
train_function -> train_function
```
The dataset examples look like this:
```python
print(list(train_dataset.take(1).as_numpy_iterator()))
```
```
[({'input_ids': array([[    0,   133,   864, ...,     1,     1,     1],
       [    0,   133,   382, ...,     1,     1,     1],
       [    0,  1121,   645, ...,     1,     1,     1],
       ...,
       [    0,   133,   864, ...,     1,     1,     1],
       [    0,  1121,   144, ...,     1,     1,     1],
       [    0,   495, 21046, ...,     1,     1,     1]], dtype=int32), 'attention_mask': array([[1, 1, 1, ..., 0, 0, 0],
       [1, 1, 1, ..., 0, 0, 0],
       [1, 1, 1, ..., 0, 0, 0],
       ...,
       [1, 1, 1, ..., 0, 0, 0],
       [1, 1, 1, ..., 0, 0, 0],
       [1, 1, 1, ..., 0, 0, 0]], dtype=int32)}, array([0, 0, 0, 0, 1, 0, 0, 0], dtype=int32))]
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->


I have environment with following

```
torch=1.7.1+cpu
tensorflow=2.2.0
transformers=4.2.2
Python=3.6.12
```

and I am using below command

```
input_ids = tokenizer.encode('accident', return_tensors='tf')
greedy_output = model.generate(input_ids, max_length=50)
print("Output:\n" + 100 * '-')
```

but I get the error below

```
  File "/home/anaconda3/envs/myenv/lib/python3.6/site-packages/transformers/generation_utils.py", line 368, in _prepare_attention_mask_for_generation
    return input_ids.new_ones(input_ids.shape)

AttributeError: 'tensorflow.python.framework.ops.EagerTensor' object has no attribute 'new_ones'
```

Earlier, until October, the same was working perfectly. May I ask help which error or conflict I am making./?


Added integration tests for Pytorch implementation of the ELECTRA model

Fixes #9949

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

@LysandreJik


Fixing model templates


Fix mlflow param overflow clean

# What does this PR do?

This PR fixes the issue #8849 where MLflow logging failed due to parameters logged being too long. Now the MLflow logger also fetches the limits directly from MLflow validation utility.

Fixes #8849

An example using run_seq2seq.py: https://colab.research.google.com/drive/1Sof7YtueI5MNcm9rn0wOKkFvWSeqK-Sy?usp=sharing

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @jplu

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->
@sgugger


remove token_type_ids from TokenizerBertGeneration output

# What does this PR do?
this PR fixes #10045  removes `token_type_ids` as it's not needed for `BertGenerationModel`

@LysandreJik


Fix TF template

# What does this PR do?

Fix the TF template.


Integrating GPT-2 model with Web page

Hi,

I would like to integrate GPT-2 model with web technology such as html, and javascript in order to build a similar editor https://transformer.huggingface.co/doc/gpt2-large

Can you please guide me how can I achieve it?


I am trying to execute this command after installing all the required modules and I ran into this error:
NOTE  : We are running this on HPC cluster.

`python -c "from transformers import pipeline; print(pipeline('sentiment-analysis')('I love you'))"
`
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/civil/phd/cez198233/anaconda3/envs/lang2/lib/python3.7/site-packages/transformers/pipelines/__init__.py", line 340, in pipeline
    framework = framework or get_framework(model)
  File "/home/civil/phd/cez198233/anaconda3/envs/lang2/lib/python3.7/site-packages/transformers/pipelines/base.py", line 66, in get_framework
    model = AutoModel.from_pretrained(model, revision=revision)
  File "/home/civil/phd/cez198233/anaconda3/envs/lang2/lib/python3.7/site-packages/transformers/models/auto/modeling_auto.py", line 724, in from_pretrained
    pretrained_model_name_or_path, return_unused_kwargs=True, **kwargs
  File "/home/civil/phd/cez198233/anaconda3/envs/lang2/lib/python3.7/site-packages/transformers/models/auto/configuration_auto.py", line 360, in from_pretrained
    config_dict, _ = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/home/civil/phd/cez198233/anaconda3/envs/lang2/lib/python3.7/site-packages/transformers/configuration_utils.py", line 420, in get_config_dict
    use_auth_token=use_auth_token,
  File "/home/civil/phd/cez198233/anaconda3/envs/lang2/lib/python3.7/site-packages/transformers/file_utils.py", line 1056, in cached_path
    local_files_only=local_files_only,
  File "/home/civil/phd/cez198233/anaconda3/envs/lang2/lib/python3.7/site-packages/transformers/file_utils.py", line 1235, in get_from_cache
    "Connection error, and we cannot find the requested files in the cached path."
ValueError: Connection error, and we cannot find the requested files in the cached path. Please try again or make sure your Internet connection is on.

**#CONDA LIST OUTPUT**
conda list
# packages in environment at /home/civil/phd/cez198233/anaconda3/envs/lang2:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main
blas                      1.0                         mkl
brotlipy                  0.7.0           py37hb5d75c8_1001    conda-forge
ca-certificates           2020.12.5            ha878542_0    conda-forge
certifi                   2020.12.5        py37h89c1867_1    conda-forge
cffi                      1.14.4           py37h261ae71_0
chardet                   4.0.0            py37h89c1867_1    conda-forge
click                     7.1.2              pyh9f0ad1d_0    conda-forge
cryptography              2.9.2            py37hb09aad4_0    conda-forge
cudatoolkit               10.0.130                      0
dataclasses               0.7                pyhb2cacf7_7    conda-forge
filelock                  3.0.12             pyh9f0ad1d_0    conda-forge
freetype                  2.10.4               h5ab3b9f_0
gperftools                2.7                  h767d802_2    conda-forge
idna                      2.10               pyh9f0ad1d_0    conda-forge
importlib-metadata        3.4.0            py37h89c1867_0    conda-forge
intel-openmp              2020.2                      254
joblib                    1.0.0              pyhd8ed1ab_0    conda-forge
jpeg                      9b                   h024ee3a_2
lcms2                     2.11                 h396b838_0
ld_impl_linux-64          2.33.1               h53a641e_7
libedit                   3.1.20191231         h14c3975_1
libffi                    3.3                  he6710b0_2
libgcc-ng                 9.1.0                hdf63c60_0
libpng                    1.6.37               hbc83047_0
libstdcxx-ng              9.1.0                hdf63c60_0
libtiff                   4.1.0                h2733197_1
lz4-c                     1.9.3                h2531618_0
mkl                       2020.2                      256
mkl-service               2.3.0            py37he8ac12f_0
mkl_fft                   1.2.0            py37h23d657b_0
mkl_random                1.1.1            py37h0573a6f_0
ncurses                   6.2                  he6710b0_1
ninja                     1.10.2           py37hff7bd54_0
numpy                     1.19.2           py37h54aff64_0
numpy-base                1.19.2           py37hfa32c7d_0
olefile                   0.46                     py37_0
openssl                   1.1.1i               h27cfd23_0
packaging                 20.9               pyh44b312d_0    conda-forge
perl                      5.32.0               h36c2ea0_0    conda-forge
pillow                    8.1.0            py37he98fc37_0
pip                       20.3.3           py37h06a4308_0
pycparser                 2.20                       py_2
pyopenssl                 19.1.0                   py37_0    conda-forge
pyparsing                 2.4.7              pyh9f0ad1d_0    conda-forge
pysocks                   1.7.1            py37h89c1867_3    conda-forge
python                    3.7.9                h7579374_0
python_abi                3.7                     1_cp37m    conda-forge
pytorch                   1.1.0           py3.7_cuda10.0.130_cudnn7.5.1_0    pytorch
readline                  8.1                  h27cfd23_0
regex                     2020.11.13       py37h4abf009_0    conda-forge
requests                  2.25.1             pyhd3deb0d_0    conda-forge
sacremoses                0.0.43             pyh9f0ad1d_0    conda-forge
sentencepiece             0.1.92           py37h99015e2_0    conda-forge
setuptools                52.0.0           py37h06a4308_0
six                       1.15.0           py37h06a4308_0
sqlite                    3.33.0               h62c20be_0
tk                        8.6.10               hbc83047_0
tokenizers                0.9.4            py37h17e0dd7_1    conda-forge
torchvision               0.3.0           py37_cu10.0.130_1    pytorch
tqdm                      4.56.0             pyhd8ed1ab_0    conda-forge
transformers              4.2.2              pyhd8ed1ab_0    conda-forge
typing_extensions         3.7.4.3                    py_0    conda-forge
urllib3                   1.26.3             pyhd8ed1ab_0    conda-forge
wheel                     0.36.2             pyhd3eb1b0_0
xz                        5.2.5                h7b6447c_0
zipp                      3.4.0                      py_0    conda-forge
zlib                      1.2.11               h7b6447c_3
zstd                      1.4.5                h9ceee32_0

**conda info --all output**

conda info --all

     active environment : lang2
    active env location : /home/civil/phd/cez198233/anaconda3/envs/lang2
            shell level : 1
       user config file : /home/civil/phd/cez198233/.condarc
 populated config files : /home/civil/phd/cez198233/.condarc
          conda version : 4.8.3
    conda-build version : 3.18.11
         python version : 3.7.6.final.0
       virtual packages : __glibc=2.17
       base environment : /home/civil/phd/cez198233/anaconda3  (writable)
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /home/civil/phd/cez198233/anaconda3/pkgs
                          /home/civil/phd/cez198233/.conda/pkgs
       envs directories : /home/civil/phd/cez198233/anaconda3/envs
                          /home/civil/phd/cez198233/.conda/envs
               platform : linux-64
             user-agent : conda/4.8.3 requests/2.22.0 CPython/3.7.6 Linux/3.10.0-957.el7.x86_64 centos/7.6.1810 glibc/2.17
                UID:GID : 86941:11302
             netrc file : None
           offline mode : False

# conda environments:
#
base                     /home/civil/phd/cez198233/anaconda3
9pytorch                 /home/civil/phd/cez198233/anaconda3/envs/9pytorch
lang2                 *  /home/civil/phd/cez198233/anaconda3/envs/lang2
tf-gpu                   /home/civil/phd/cez198233/anaconda3/envs/tf-gpu

sys.version: 3.7.6 (default, Jan  8 2020, 19:59:22)
...
sys.prefix: /home/civil/phd/cez198233/anaconda3
sys.executable: /home/civil/phd/cez198233/anaconda3/bin/python
conda location: /home/civil/phd/cez198233/anaconda3/lib/python3.7/site-packages/conda
conda-build: /home/civil/phd/cez198233/anaconda3/bin/conda-build
conda-convert: /home/civil/phd/cez198233/anaconda3/bin/conda-convert
conda-debug: /home/civil/phd/cez198233/anaconda3/bin/conda-debug
conda-develop: /home/civil/phd/cez198233/anaconda3/bin/conda-develop
conda-env: /home/civil/phd/cez198233/anaconda3/bin/conda-env
conda-index: /home/civil/phd/cez198233/anaconda3/bin/conda-index
conda-inspect: /home/civil/phd/cez198233/anaconda3/bin/conda-inspect
conda-metapackage: /home/civil/phd/cez198233/anaconda3/bin/conda-metapackage
conda-render: /home/civil/phd/cez198233/anaconda3/bin/conda-render
conda-server: /home/civil/phd/cez198233/anaconda3/bin/conda-server
conda-skeleton: /home/civil/phd/cez198233/anaconda3/bin/conda-skeleton
conda-verify: /home/civil/phd/cez198233/anaconda3/bin/conda-verify
user site dirs: ~/.local/lib/python3.6
                ~/.local/lib/python3.7

CIO_TEST: <not set>
CONDA_DEFAULT_ENV: lang2
CONDA_EXE: /home/civil/phd/cez198233/anaconda3/bin/conda
CONDA_PREFIX: /home/civil/phd/cez198233/anaconda3/envs/lang2
CONDA_PROMPT_MODIFIER: (lang2)
CONDA_PYTHON_EXE: /home/civil/phd/cez198233/anaconda3/bin/python
CONDA_ROOT: /home/civil/phd/cez198233/anaconda3
CONDA_SHLVL: 1
HTTPS_PROXY: <set>
HTTP_PROXY: <set>
MANPATH: /usr/share/Modules/3.2.10/share/man::/opt/pbs/19.2.4/share/man
MODULEPATH: /home/soft/modules
PATH: /home/civil/phd/cez198233/anaconda3/bin:/home/civil/phd/cez198233/anaconda3/envs/lang2/bin:/home/civil/phd/cez198233/anaconda3/bin:/home/civil/phd/cez198233/anaconda3/bin:/home/civil/phd/cez198233/anaconda3/condabin:/opt/am/bin:/opt/am/sbin:/opt/pbs/default/bin:/opt/pbs/default/sbin:/usr/share/Modules/3.2.10/bin:/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/opt/ibutils/bin:/root/bin:/opt/pbs/19.2.4/bin:/home/civil/phd/cez198233/bin
REQUESTS_CA_BUNDLE: <not set>
SSL_CERT_FILE: <not set>
ftp_proxy: <set>
http_proxy: <set>
https_proxy: <set>


Removing run_pl_glue.py from text classification docs, include run_xnli.py & run_tf_text_classification.py

# What does this PR do?

Since `run_pl_glue.py` is not part of `text-classification` examples after #9010, this PR removes it from the text-classification docs. Also, it adds `run_xnli.py` and `run_tf_text_classification.py` scripts, which are in that folder now.
<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

@sgugger may be interested in the PR, he's responsible for docs and the author of #9010


Model templates tests are run twice

The CI currently runs the model template tests twice when opening a PR from a branch of the huggingface/transformers repo.

The `pull_request_target` should only trigger on external pull requests, or we should remove the `push` target so that the suite only runs once.

Additionally, the `pull_request_target` suite doesn't take into account the changes the PR is doing.

ETA until resolved ~ 2 weeks.


Fix model template typo

Fix typo introduced in https://github.com/huggingface/transformers/pull/10033


[Finetune Seq2Seq Trainer] fix bert2bert test

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

This PR fixes the slow test: `tests/test_trainer_seq2seq.py::Seq2seqTrainerTester::test_finetune_bert2bert` which was failing because `rouge_score` was not added to the dependencies. In this PR I remove the usage of `datasets.load("rouge")` since it is very much unnecessary here => we are only testing whether training does not throw an error and for this it doesn't matter whether we use the rouge metric or accuracy. Removing `rouge` removes a dependency, which is the better way here IMO.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @jplu

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Disable temporarily too slow tests (Longformer/LED)

# What does this PR do?

The tests about SavedModel takes way too long for Longformer and LED. To do not timeout the CI, we disable them and will see how to better handle them.


Dimension error while finetuning longformer with roberta-large EncoderDecoderModel

## Environment info
- `transformers` version: 4.2.2
- Platform: Linux-3.10.0-1160.6.1.el7.x86_64-x86_64-with-debian-buster-sid
- Python version: 3.6.11
- PyTorch version (GPU?): 1.7.1 (True)
- Tensorflow version (GPU?): 2.2.0 (True)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help
- maintained examples (not research project or legacy): @patrickvonplaten

Models:
- allenai/longformer-base-4096 with roberta-large
- allenai/longformer-base-4096 with xlm-roberta-large

## To reproduce

Steps to reproduce the behavior:
I have follow the steps on https://huggingface.co/patrickvonplaten/longformer2roberta-cnn_dailymail-fp16
but switched model roberta-base to roberta-large

**CODE:-**
```
model = EncoderDecoderModel.from_encoder_decoder_pretrained("allenai/longformer-base-4096", "roberta-large")
tokenizer = LongformerTokenizer.from_pretrained("allenai/longformer-base-4096")
```
model params
```
# enable gradient checkpointing for longformer encoder
model.encoder.config.gradient_checkpointing = True

# set decoding params
model.config.decoder_start_token_id = tokenizer.bos_token_id
model.config.eos_token_id = tokenizer.eos_token_id
model.config.max_length = 142
model.config.min_length = 56
model.config.no_repeat_ngram_size = 3
model.early_stopping = True
model.length_penalty = 2.0
model.num_beams = 4

encoder_length = 2048
decoder_length = 128
batch_size = 16
```
training parms
```
training_args = TrainingArguments(
    output_dir="./",
    per_device_train_batch_size=batch_size,
    per_device_eval_batch_size=batch_size,
    #predict_from_generate=True,
    #evaluate_during_training=True,
    do_train=True,
    do_eval=True,
    logging_steps=1000,
    save_steps=1000,
    eval_steps=1000,
    overwrite_output_dir=True,
    warmup_steps=2000,
    save_total_limit=3,
    fp16=True,
)

# instantiate trainer
trainer = Trainer(
    model=model,
    args=training_args,
    compute_metrics=compute_metrics,
    train_dataset=train_dataset,
    eval_dataset=val_dataset,
)

# start training
trainer.train()
```

**ERROR:-**
```
RuntimeError: Caught RuntimeError in replica 0 on device 0.
Original Traceback (most recent call last):
File "/home/jovyan/.local/lib/python3.6/site-packages/torch/nn/parallel/parallel_apply.py", line 61, in _worker
output = module(*input, **kwargs)
File "/home/jovyan/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/jovyan/.local/lib/python3.6/site-packages/transformers/models/encoder_decoder/modeling_encoder_decoder.py", line 430, in forward
**kwargs_decoder,
File "/home/jovyan/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/jovyan/.local/lib/python3.6/site-packages/transformers/models/roberta/modeling_roberta.py", line 928, in forward
return_dict=return_dict,
File "/home/jovyan/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/jovyan/.local/lib/python3.6/site-packages/transformers/models/roberta/modeling_roberta.py", line 808, in forward
return_dict=return_dict,
File "/home/jovyan/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/jovyan/.local/lib/python3.6/site-packages/transformers/models/roberta/modeling_roberta.py", line 505, in forward
output_attentions,
File "/home/jovyan/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/jovyan/.local/lib/python3.6/site-packages/transformers/models/roberta/modeling_roberta.py", line 424, in forward
output_attentions,
File "/home/jovyan/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/jovyan/.local/lib/python3.6/site-packages/transformers/models/roberta/modeling_roberta.py", line 328, in forward
output_attentions,
File "/home/jovyan/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/jovyan/.local/lib/python3.6/site-packages/transformers/models/roberta/modeling_roberta.py", line 198, in forward
key_layer = self.transpose_for_scores(self.key(encoder_hidden_states))
File "/home/jovyan/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/jovyan/.local/lib/python3.6/site-packages/torch/nn/modules/linear.py", line 93, in forward
return F.linear(input, self.weight, self.bias)
File "/home/jovyan/.local/lib/python3.6/site-packages/torch/nn/functional.py", line 1692, in linear
output = input.matmul(weight.t())
RuntimeError: mat1 dim 1 must match mat2 dim 0
```


[BART Tests] Fix Bart mask filling pipeline tests

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Following the PR: https://github.com/huggingface/transformers/pull/9783/ some slow Bart tests were not updated. This PR updates the mask-filling bart tests accordingly.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @jplu

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Fix slow dpr test