[WIP] Add Linformer

# What does this PR do?

This PR adds [Linformer: Self-Attention with Linear Complexity](https://arxiv.org/abs/2006.04768) by Facebook AI. Contrary to the regular Transformer, it has linear complexity in both space and time w.r.t. the sequence length, allowing it to be trained on much larger sequence lengths.

I've created both a PT and TF version using the CookieCutter template (BERT-based, encoder-only). The only difference with BERT is that:
- `LinformerSelfAttention` looks a bit different compared to `BertSelfAttention`. The Linformer model adds 2 projection matrices to the keys and values, which project the sequence dimension (which is of size 512 by default) to a lower dimension k (such as `k=256`). As you'll see, I do not use the `extended_attention_mask` function which is used by other models. Instead, I cast the attention mask across the different attention heads by simply writing `attention_mask[:,None,:,None]`, which I then multiply by the keys and values, before projecting to the lower dimension.
- One can choose to use the same projection matrix for both keys and values. This is determined by the `share_projection` attribute of `LinformerConfig`.
- Linformer comes with 2 limitations: it cannot be used in the autoregressive case (see [this](https://github.com/lucidrains/linformer/issues/4#issuecomment-777781537)) - hence I removed the `is_decoder` logic, and it assumes a fixed sequence length. The latter is determined by the `seq_length` attribute of LinformerConfig, which is set to 512 by default. So if you provide `input_ids` and so on to the model, their length must be equal to the value of `seq_length`.

Fixes the following issues:
#4967
#5201

## Before submitting
- [ ] Did you write any new necessary tests? Yes I did, however I need some help to make sure all pass. Current status: for PyTorch, 30 are passing, 7 are failing. For Tensorflow, 26 passed, 9 failed. Note that there are currently no integration tests, as no weights were shared by the authors yet. However, they plan to release them as seen [here](https://github.com/pytorch/fairseq/issues/2795#issuecomment-720680448).
- [ ] Also, it seems that the original authors relied on RoBERTa rather than BERT, so we might have to let `LinformerTokenizer` inherit from `RobertaTokenizer` rather than `BertTokenizer`.

## Who can review?

@patil-suraj @LysandreJik


from_pretrained: check that the pretrained model is for the right model architecture

# What does this PR do?
Adding Checks to the from_pretrained workflow to check the model name passed belongs to the model being initiated.
Same checks need to be added for Tokenizer.

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes https://github.com/huggingface/transformers/issues/10293

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @jplu

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


[run_seq2seq] fix nltk lookup

Hmm, CI crashes every so often on
```
try:
    nltk.data.find("tokenizers/punkt")
except LookupError:
```

introduced in this PR: https://github.com/huggingface/transformers/pull/10407

https://app.circleci.com/pipelines/github/huggingface/transformers/20635/workflows/989fde0b-e543-4620-9d9a-f213ad53dd9b/jobs/176742
```
__________________ ERROR collecting examples/test_examples.py __________________
examples/test_examples.py:51: in <module>
    import run_seq2seq
examples/seq2seq/run_seq2seq.py:54: in <module>
    nltk.data.find("tokenizers/punkt")
../.local/lib/python3.6/site-packages/nltk/data.py:539: in find
    return FileSystemPathPointer(p)
../.local/lib/python3.6/site-packages/nltk/compat.py:41: in _decorator
    return init_func(*args, **kwargs)
../.local/lib/python3.6/site-packages/nltk/data.py:315: in __init__
    raise IOError("No such file or directory: %r" % _path)
E   OSError: No such file or directory: '/home/circleci/nltk_data/tokenizers/punkt/PY3'
```
which is odd, re-running the job fixed the problem.

So trying to mend it with:

```
try:
    nltk.data.find("tokenizers/punkt")
except (LookupError, OSError):
```
not sure why the Exception was different here.

@sgugger, @LysandreJik


[examples tests] various fixes

This PR is fixing slow examples tests that currently fail on scheduled CI

2 more tests will be fixed by https://github.com/huggingface/transformers/pull/10551

This PR:

Sharded DDP issues:
* fixes fully sharded ddp enum - and the corresponding tests
* 2 sharded ddp tests currently hang with master fairscale - add skip until this is sorted out - didn't want to step on @sgugger's toes - so for now just skipping

Tests:
* changes a large group of tests to check loss is not nan
* fix `test_run_seq2seq_slow` test
   - missed by my PR https://github.com/huggingface/transformers/pull/10428
  - make it more resilient - was failing quality-wise on multi-gpu
* then we have an issue with apex - once run in a worker directly it breaks other tests running directly in the same pytest worker:

```
        # XXX: apex breaks the trainer if it's run twice e.g. run_seq2seq.main() from the same
        # program and it breaks other tests that run from the same pytest worker, therefore until this is
        # sorted out it must be run only in an external program, that is distributed=True in this
        # test and only under one or more gpus - if we want cpu will need to make a special test
        #
        # specifically to the problem traced it to self.optimizer.step() - if it's run 2nd time via
        # 2nd main() call it botches the future eval.
```
I'm not quite sure what happens but I think no-one will run into this in a normal situation, here we basically end up running:
```
main()
main()
```
in the same process. We have some internal state that doesn't get reset.

So as I wrote above I used a workaround of running the apex integration test in a separate process so that it doesn't affect the rest of the test suite.

Of course if you have ideas on what the problem is and how to fix it I'm all ears. It's very simple to reproduce it:
```
    def test_run_seq2seq_apex(self):
        self.run_seq2seq_quick(distributed=True, extra_args_str="--fp16 --fp16_backend=apex")
        # test 2nd time - was getting eval_loss': nan'
        # to reproduce the problem set distributed=False
        self.run_seq2seq_quick(distributed=True, extra_args_str="--fp16 --fp16_backend=apex")
```

I can move it out into a separate PR if it's too much for one.

@LysandreJik, @sgugger


[trainer] fix double wrapping + test

We have an issue with
```
trainer.train()
trainer.train()
```
under any environment that requires model wrapping - we currently get the wrapping multiple times - and things may kind of work - but most of the time it breaks badly - thanks to apex for complaining noisily when it's being wrapped second time.

i.e. we get things like `DataParallel(DataParallel(model))`

This PR fixes this problem and adds a test.

@sgugger, @LysandreJik


wrong model used for BART Summarization example

I'm pretty sure that `bart-large` was not trained for summarization, I replaced it with `bart-large-cnn` which is a model that was fine-tuned for summarization

# What does this PR do?
replace model used in Summarization example


wav2vec2: support datasets other than LibriSpeech

# What does this PR do?
Building on #10145, I'm adding support for the two other speech datasets (besides LibriSpeech) for ASR at the time of writing (`timit_asr` and `arabic_speech_corpus`), which require the following:
* Custom validation split name
* On-the-fly resampling support to target feature extractor's sampling rate (via `librosa` -- see `requirements.txt`)
* Max duration (in seconds) filter to remove outliers (which may crash GPU training when running OOM)
* Verbose logging to help debug custom datasets, including reverse transliteration (via `lang-trans` -- see `requirements.txt`)
* Pre-processing: tokenization and text normalization using orthography rules
  * Casing (`do_lower_case`)
  * Custom vocab for tokens used in orthography (e.g., Buckwalter transliteration for Arabic)
  * Custom word delimiter token (when the default, `"|"`, is used in orthography)
  * Transformations similar to those in `jiwer` to normalize transcripts for training
  * Removing special words (e.g., "sil" which can be used to indicate silence)
  * Translation table (e.g., "-" -> " " to break compounds like "quarter-century-old")
  * Cleaning up characters not in vocab (after applying the rules above)

Arabic model: https://huggingface.co/elgeish/wav2vec2-large-xlsr-53-arabic
TIMIT models: https://huggingface.co/elgeish/wav2vec2-base-timit-asr and https://huggingface.co/elgeish/wav2vec2-large-lv60-timit-asr

## Who can review?
@patrickvonplaten @sgugger @LysandreJik @patil-suraj @SeanNaren


Issue when customizing loss in Trainer

Hi everyone,

I am a student and therefore not yet very familiar with the way issues report work on git, so I aplogize in advance if this is not the proper place to post this message.
I'm trying to customize the loss to use a weighted CrossEntropyLoss, I browsed the issues reports and saw that this matter was already mentionned and a solution was brought by the developpers (I think it was @sgugger) of the Transformers library, I tried to follow their code snipets as much as possible but always ended up with the same error.

I'm working with the last version of Transformers and this is my code:

```
config = AutoConfig.from_pretrained("bert-base-cased", num_labels=2, finetuning_task="SST-2")

# Test with modified trainer for weighted CrossEntropyLoss

model = AutoModelForSequenceClassification.from_pretrained(
        "dmis-lab/biobert-base-cased-v1.1",
        from_tf=False,
        config=config)

from torch import FloatTensor

classDistribution_raw = [97, 3]
classDistribution = [0.8, 0.2]
normedWeights = [1 - (x / sum(classDistribution)) for x in classDistribution]
normedWeights = FloatTensor(normedWeights).cuda()

from torch.nn import CrossEntropyLoss

class MyTrainer(Trainer):
    def compute_loss(self, model, inputs, return_outputs=False):

        if "labels" in inputs:
          labels = inputs.pop("labels")

        outputs = model(**inputs)
        logits = outputs.logits
        loss_function = CrossEntropyLoss(weight = normedWeights)

        if self.args.past_index >= 0:
            self._past = outputs[self.args.past_index]

        if labels is not None:
            loss = loss_function(logits, labels)
        else:
            # We don't use .loss here since the model may return tuples instead of ModelOutput.
            loss = outputs["loss"] if isinstance(outputs, dict) else outputs[0]

        return (loss, outputs) if return_outputs else loss

trainer = MyTrainer(
        model=model,
        args=training_args,
        train_dataset=train_dataset,
        eval_dataset=eval_dataset,
        compute_metrics=compute_metrics_fn,
        tokenizer=tokenizer,
    )
```
And this is the error I keep getting:
`'NoneType' object has no attribute 'detach'`

I'm probably doing something wrong but I can't understand what.

Thanks in advance for your answers, I stay availble if u need any more details about my set up or my code,
Best regards,
Arthur Ledaguenel


request about deepspeed tutorial

Dear @stas00

You have created this great tutorial here that without you it was really very hard near impossible to be able to train these large models, thank you so much

https://github.com/huggingface/transformers/issues/8771

Do you mind updating your comment, including how numbers would change if you add distributed training and deepspeed and how the command should be in this case? so to further speedup deepspeed on multiple GPUs

I also get this message with deepspeed with transformer 4.3.3 running the tutorial:
[deepscale] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 131072.0, reducing to 65536.0

could you tell me if this shows any issue in training? sorry I am very unfamiliar with deepspeed.

One more question, in tutorial you did not use --fp16, could you add a few comments if we can use it with deepspeed?
this is such a great toturial and would be great if we could have all info in one place.

thank you so much for the hard work and a lot of help


Why HFArgumentParser.parse_dict(TrainerArguments) return tuple instead of dict?

I guess it is not certainly bug however I could not almost understand why` HfArgumentParser.parsedict() `return `(*outputs,)`. As can be seen in [docs](https://huggingface.co/transformers/_modules/transformers/hf_argparser.html#HfArgumentParser
)
I am trying to fine-tune BERT for Token Classification using Trainer class and my aim is to turn argparse object to TrainingArguments. I converted  ArgumentParser object to dictionary then using HfArgumentParser.parsedict() turn it to TrainingArguments object. However I realized that ` HfArgumentParser.parsedict() ` returns `(TrainingArguments, )` tuple  which cause following error in Trainer initializer.

```console
Traceback (most recent call last):
  File "transformers_ner.py", line 344, in <module>
    finetune_model(args)
  File "transformers_ner.py", line 308, in finetune_model
    args=train_args
  File "/home/akali/.local/lib/python3.6/site-packages/transformers/trainer.py", line 237, in __init__
    set_seed(self.args.seed)
AttributeError: 'tuple' object has no attribute 'seed'

```

I get train_args by:

```python
    # ArgParser --> Training Arguments
    HFParser = HfArgumentParser(TrainingArguments)
    train_args = HFParser.parse_dict(args)
```

I know I can get TrainingArguments object by doing` train_args[0] `however, isn't it meaningless that `HfArgumentParser.parsedict()` return tuple instead of directly TrainingArguments.

Thanks.


seq2seq example with T5 does not run due to issue with loading tokernizer

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.3.3
- Platform: linux
- Python version: 3.7
- PyTorch version (GPU?): 1.8
- Tensorflow version (GPU?): -
- Using GPU in script?: -
- Using distributed or parallel set-up in script?: -

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @jplu

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->
@patrickvonplaten, @patil-suraj
## Information
Hi
I am trying to run run_seq2seq.py example on mt5 model

` python run_seq2seq.py --model_name_or_path google/mt5-small --dataset_name wmt16 --dataset_config_name ro-en --source_prefix "translate English to Romanian: " --task translation_en_to_ro   --output_dir /test/test_large  --do_train --do_eval --predict_with_generate   --max_train_samples 500  --max_val_samples 500 --tokenizer_name google/mt5-small
`

getting this error:

```
Traceback (most recent call last):
  File "run_seq2seq.py", line 539, in <module>
    main()
  File "run_seq2seq.py", line 309, in main
    use_auth_token=True if model_args.use_auth_token else None,
  File "/dara/libs/anaconda3/envs/fast/lib/python3.7/site-packages/transformers/models/auto/tokenization_auto.py", line 379, in from_pretrained
    return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
  File "/dara/libs/anaconda3/envs/fast/lib/python3.7/site-packages/transformers/tokenization_utils_base.py", line 1789, in from_pretrained
    resolved_vocab_files, pretrained_model_name_or_path, init_configuration, *init_inputs, **kwargs
  File "/dara/libs/anaconda3/envs/fast/lib/python3.7/site-packages/transformers/tokenization_utils_base.py", line 1860, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
  File "/dara/libs/anaconda3/envs/fast/lib/python3.7/site-packages/transformers/models/t5/tokenization_t5_fast.py", line 147, in __init__
    **kwargs,
  File "/dara/libs/anaconda3/envs/fast/lib/python3.7/site-packages/transformers/tokenization_utils_fast.py", line 103, in __init__
    "Couldn't instantiate the backend tokenizer from one of: "
ValueError: Couldn't instantiate the backend tokenizer from one of: (1) a `tokenizers` library serialization file, (2) a slow tokenizer instance to convert or (3) an equivalent slow tokenizer class to instantiate and convert. You need to have sentencepiece installed to convert a slow tokenizer to a fast one.
```

thank you for your help


Movement pruning for DistilGPT2 - pre_trained model, issue while using dynamic_quantization

## Environment info

- `transformers` version: 4.3.3
- Platform: Ubuntu 20.04
- Python version: 3.8.8
- PyTorch version (GPU?): 1.4.0 (False)
- Tensorflow version (GPU?): 2.4.1 (False)
- Using GPU in script?: False
- Using distributed or parallel set-up in script?: False

### Who can help

[@VictorSanh](https://github.com/VictorSanh)

## Information

I am following through the Saving PruneBERT [notebook](https://github.com/huggingface/transformers/blob/b11386e158e86e62d4041eabd86d044cd1695737/examples/movement-pruning/Saving_PruneBERT.ipynb) from the *examples/movement-pruning/* directory, to have a pruned and quantized model for DistilGPT2.

In cell 4:

```
# Elementary representation: we decompose the quantized tensors into (scale, zero_point, int_repr).
# See https://pytorch.org/docs/stable/quantization.html

# We further leverage the fact that int_repr is sparse matrix to optimize the storage: we decompose int_repr into
# its CSR representation (data, indptr, indices).

elementary_qtz_st = {}
for name, param in qtz_st.items():
    if "dtype" not in name and param.is_quantized:
        print("Decompose quantization for", name)
        # We need to extract the scale, the zero_point and the int_repr for the quantized tensor and modules
        scale = param.q_scale()                                # torch.tensor(1,) - float32
        zero_point = param.q_zero_point()                      # torch.tensor(1,) - int32
        elementary_qtz_st[f"{name}.scale"] = scale
        elementary_qtz_st[f"{name}.zero_point"] = zero_point

        # We assume the int_repr is sparse and compute its CSR representation
        # Only the FCs in the encoder are actually sparse
        int_repr = param.int_repr()                         # torch.tensor(nb_rows, nb_columns) - int8
        int_repr_cs = sparse.csr_matrix(int_repr)           # scipy.sparse.csr.csr_matrix

        elementary_qtz_st[f"{name}.int_repr.data"] = int_repr_cs.data                  # np.array int8
        elementary_qtz_st[f"{name}.int_repr.indptr"] = int_repr_cs.indptr              # np.array int32
        assert max(int_repr_cs.indices) < 65535 # If not, we shall fall back to int32
        elementary_qtz_st[f"{name}.int_repr.indices"] = np.uint16(int_repr_cs.indices) # np.array uint16
        elementary_qtz_st[f"{name}.int_repr.shape"] = int_repr_cs.shape                # tuple(int, int)
    else:
        elementary_qtz_st[name] = param
```

my model throws the below error:

```AttributeError: 'NoneType' object has no attribute 'is_quantized'```

which on digging into quantizing step in cell 2, shows a significant difference between the used BERT and DistilGPT2 (the one am using) quantized version:

1. The BERT quantized first few layers look like:

   ```
   bert.embeddings.position_ids
   bert.embeddings.word_embeddings.weight
   bert.embeddings.position_embeddings.weight
   bert.embeddings.token_type_embeddings.weight
   bert.embeddings.LayerNorm.weight
   bert.embeddings.LayerNorm.bias
   bert.encoder.layer.0.attention.self.query.scale
   bert.encoder.layer.0.attention.self.query.zero_point
   bert.encoder.layer.0.attention.self.query._packed_params.weight
   bert.encoder.layer.0.attention.self.query._packed_params.bias
   bert.encoder.layer.0.attention.self.key.scale
   bert.encoder.layer.0.attention.self.key.zero_point
   bert.encoder.layer.0.attention.self.key._packed_params.weight
   bert.encoder.layer.0.attention.self.key._packed_params.bias
   bert.encoder.layer.0.attention.self.value.scale
   bert.encoder.layer.0.attention.self.value.zero_point
   bert.encoder.layer.0.attention.self.value._packed_params.weight
   bert.encoder.layer.0.attention.self.value._packed_params.bias
   ```

2. The quantized DistilGPT2 first few layers look like:

   ```
   transformer.wte.weight
   transformer.wpe.weight
   transformer.h.0.ln_1.weight
   transformer.h.0.ln_1.bias
   transformer.h.0.attn.bias
   transformer.h.0.attn.masked_bias
   transformer.h.0.attn.c_attn.weight
   transformer.h.0.attn.c_attn.bias
   transformer.h.0.attn.c_proj.weight
   transformer.h.0.attn.c_proj.bias
   transformer.h.0.ln_2.weight
   transformer.h.0.ln_2.bias
   ```

3. As you would notice, there is a clear difference in the way layers are formed after quantization

   a) BERT has ```.scale``` and ```.zero_point``` added to every layer after embeddings, whereas the DistilGPT2 layers do not get these 2 extras.

   b) any ```.weight``` and ```.bias``` are converted to ```._packed_params.weight``` and ```._packed_params.bias``` respectively.

4. I beleive this is why when processing the cell 4:

   a) It does not even go to all layers that are missing ```._packed_params``` and just tries to process the last layer which is

   ```
   lm_head.scale
   lm_head.zero_point
   lm_head._packed_params.weight
   lm_head._packed_params.bias
   ```

   b) Where it fails with the error mentioned just before point 1.

## To reproduce

Steps to reproduce the behavior:

1. Clone `transformers` and follow the steps to install the `movement-pruning` example
2. Upgrade torch to v1.4
3. Try to run the `Saving_PruneBERT.ipynb` notebook with 1 change, in the cell 2 instantiate the model class with below line
   ```model = AutoModelForCausalLM.from_pretrained('distilgpt2')```


bug in run_finetune

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.3.3
- Platform: linux
- Python version: 3.7
- PyTorch version (GPU?): 1.8
- Tensorflow version (GPU?): -
- Using GPU in script?: -
- Using distributed or parallel set-up in script?: -

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @jplu

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

@patrickvonplaten, @patil-suraj

## Information
I am running  run_seq2seq.py getting

ImportError: cannot import name 'is_offline_mode' from 'transformers.file_utils'

## To reproduce

Steps to reproduce the behavior:

python run_seq2seq.py

thnaks


The dimension of Feature extraction


Update data_collator.py

加上中文註解


Import error for class Speech2TextProcessor, Speech2TextTransformerForConditionalGeneration

## Environment info
- `transformers` version: 4.4.0.dev0
- Platform: Windows-10-10.0.19041-SP0
- Python version: 3.8.8
- PyTorch version (GPU?): 1.7.1+cpu (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help

Models:
- Speech2TextProcessor, Speech2TextTransformerForConditionalGeneration @patil-suraj

## Information

Model I am using (Speech2TextProcessor, Speech2TextTransformerForConditionalGeneration):

The problem arises when using:
* trying to import the model

## To reproduce

Steps to reproduce the behavior:

1. first install master brach
2. then try to import the  Speech2TextProcessor, Speech2TextTransformerForConditionalGeneration class then showing import error
```
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-8149e8a8d76d> in <module>
      1 import torch
----> 2 from transformers import Speech2TextProcessor, Speech2TextTransformerForConditionalGeneration
      3 from datasets import load_dataset
      4 import soundfile as sf

ImportError: cannot import name 'Speech2TextProcessor' from 'transformers' (unknown location)
```
## Expected behavior
able to run the code from _https://huggingface.co/facebook/s2t-large-librispeech-asr_ from docs

one suggestion, if any fine-tuning script for s2t-large-librispeech-asr model it would be a great help

thankyou


Advice on creating/wrapping `PreTrainedModel` to be compatible with the codebase?

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: NA
- Platform: NA
- Python version: NA
- PyTorch version (GPU?): NA
- Tensorflow version (GPU?): NA
- Using GPU in script?: NA
- Using distributed or parallel set-up in script?: NA

### Who can help
@patrickvonplaten

<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @jplu

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [X] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce
Thanks for the amazing library!

I'm curious if there are instructions on creating a `PreTrainedModel` subclass or creating an `nn.Module` that behaves like a `PreTrainedModel`? Suppose I want to wrap the existing model with some simple additional capabilities inside an `nn.Module`, what are some of the methods that I need to implement/override -- so that they can work well with existing examples?

I'm aware of some tutorials on creating a new model, but that seems pretty complicated and involved -- whereas I'm interested in just adding a couple of simple features.

For example, in the Seq2Seq example, I have noticed that the function signature of `model.forward` determines what data will (not) be passed to the model (as in [`trainer._remove_unused_columns`](https://github.com/huggingface/transformers/blob/master/src/transformers/trainer.py#L458)), and the existence of `model.prepare_decoder_input_ids_from_labels` also influences the input data (as in [`DataCollatorForSeq2Seq .__call__`](https://github.com/huggingface/transformers/blob/master/src/transformers/data/data_collator.py#L292)).

It'd be great if someone could point me to some guidance on tweaking the model to be compatible with the rest of the codebase. Thanks in advance for your time!

Steps to reproduce the behavior:

1.
2.
3.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->


fix tf doc bug

I found there are a different between tfBertForPretraining and BertForPretraining.
I Have create a forum at `https://discuss.huggingface.co/t/different-doc-with-bertforpretraining-and-tfbertforpretraining/4167` and get a response.


offline mode for firewalled envs (part 2)

In https://github.com/huggingface/transformers/pull/10407 I noticed I missed a few places where `local_files_only` should be overridden for the offline mode, so this PR completes the process.

Also rewrote the test to be more readable. Could test TF/Flax too but I don't have tiny models to run quick tests on.

@LysandreJik, @sgugger


Ner label re alignment

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #10263

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case. [link](https://github.com/huggingface/transformers/issues/10263#issuecomment-781648059)
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?
- pipelines: @LysandreJik, @Narsil, @joshdevins


XLSR-53

# 🚀 Feature request

Is it possible to use XLSR-53 with transformers in the near future?


from_pretrained() - some model weights not initialized message

## Environment info
- `transformers` version: 4.0.1
- Platform: Linux-4.15.0-132-generic-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyTorch version (GPU?): 1.7.1+cu110 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: no

Note also: cookiecutter dependency is not included in pip install transformers so transformers-cli env initially fails

### Who can help
(T5) @patrickvonplaten, @patil-suraj, @sshleifer

When using `T5ForConditionalGeneration.from_pretrained('t5-base')`, I get the following warning at load:
```
Some weights of the model checkpoint at t5-large were not used when initializing T5ForConditionalGeneration: ['decoder.block.0.layer.1.EncDecAttention.relative_attention_bias.weight']
- This IS expected if you are initializing T5ForConditionalGeneration from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining m
odel).
- This IS NOT expected if you are initializing T5ForConditionalGeneration from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification mode
l).
```

If I load from a checkpoint that I create (i.e. local file), I get the same message.

But I think that all weights are, in fact, identical:
- evaluation code on the model I finetune before saving AND
- evaluation code on the model I finetune, save, and then reload
are identical.

This suggests that *all* weights are identical, since performance is identical. This contradicts the warning message.

Questions:
1) Are some weights actually not being loaded? If so, how could I observe identical behavior on metrics. Or is this warning wrong?
2) If this warning is correct, how can I force the model to fully load the model exactly as I saved it.
3) Is there any other difference (randomly initialized head, randomly initialized weights) between the t5 that is pretrained and the T5ForConditionalGeneration?


Mismatch between input and target batch_sizes while training FSMT model

Code to reproduce

```python
 tokenizer = get_fsmt_tokenizer()
    tokenizer.model_max_length=100
    model = get_fsmt_model()
    freeze_embeds(model)
    freeze_encoder(model)
    train_dataset = YandexRuEnDataset("data", split="train")
    val_dataset = YandexRuEnDataset("data", split="valid")

    training_args = TrainingArguments(
        output_dir='./results',  # output directory
        num_train_epochs=3,  # total # of training epochs
        per_device_train_batch_size=16,  # batch size per device during training
        per_device_eval_batch_size=64,  # batch size for evaluation
        warmup_steps=5000,  # number of warmup steps for learning rate scheduler
        weight_decay=0.01,  # strength of weight decay
        logging_dir='./logs',  # directory for storing logs
        fp16=True,
        fp16_opt_level='O2',
        save_steps=20000
    )

    trainer = Seq2SeqTrainer(
        model=model,  # the instantiated 🤗 Transformers model to be trained
        args=training_args,  # training arguments, defined above
        train_dataset=train_dataset,  # training dataset
        eval_dataset=val_dataset,  # evaluation dataset
        tokenizer=tokenizer,
        data_collator=collate_sentences(tokenizer)
    )

    trainer.train()
```
Dataset class

```python
class YandexRuEnDataset(Dataset):

    def __init__(self, root_data, split):
        src = open(f"{root_data}/corpus.en_ru.1m.ru", "r").readlines()
        tgt = open(f"{root_data}/corpus.en_ru.1m.en", "r").readlines()
        # src = src[:int(0.1*len(src))]
        # tgt = tgt[:int(0.1 * len(tgt))]
        X_train, X_test, y_train, y_test = train_test_split(src, tgt, test_size=0.33, random_state=228)
        if split == "train":
            self.src = X_train
            self.trg = y_train
        elif split == "valid":
            self.src = X_test
            self.trg = y_test

    def __len__(self):
        return len(self.src)

    def __getitem__(self, idx):
        src = self.src[idx]
        trg = self.trg[idx]

        return src, trg

def collate_sentences(tokenizer: Tokenizer):
    def collate_fn(batch):
        batch = list(zip(*batch))
        X_batch = list(batch[0])
        y_batch = list(batch[1])
        batch = tokenizer.prepare_seq2seq_batch(
            src_texts=X_batch, tgt_texts=y_batch,
            padding=True, truncation=True,
            return_tensors='pt'
        )
        return batch

    return collate_fn
```

Exception

```
  File "/home/farit/PycharmProjects/NMT/train.py", line 40, in <module>
    trainer.train()
  File "/home/farit/anaconda3/envs/NMT/lib/python3.8/site-packages/transformers/trainer.py", line 940, in train
    tr_loss += self.training_step(model, inputs)
  File "/home/farit/anaconda3/envs/NMT/lib/python3.8/site-packages/transformers/trainer.py", line 1302, in training_step
    loss = self.compute_loss(model, inputs)
  File "/home/farit/anaconda3/envs/NMT/lib/python3.8/site-packages/transformers/trainer.py", line 1334, in compute_loss
    outputs = model(**inputs)
  File "/home/farit/anaconda3/envs/NMT/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/farit/anaconda3/envs/NMT/lib/python3.8/site-packages/transformers/models/fsmt/modeling_fsmt.py", line 1180, in forward
    masked_lm_loss = loss_fct(lm_logits.view(-1, self.config.tgt_vocab_size), labels.view(-1))
  File "/home/farit/anaconda3/envs/NMT/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/farit/anaconda3/envs/NMT/lib/python3.8/site-packages/torch/nn/modules/loss.py", line 961, in forward
    return F.cross_entropy(input, target, weight=self.weight,
  File "/home/farit/anaconda3/envs/NMT/lib/python3.8/site-packages/torch/nn/functional.py", line 2468, in cross_entropy
    return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
  File "/home/farit/anaconda3/envs/NMT/lib/python3.8/site-packages/torch/nn/functional.py", line 2261, in nll_loss
    raise ValueError('Expected input batch_size ({}) to match target batch_size ({}).'
ValueError: Expected input batch_size (1440) to match target batch_size (1600).
````


[Causal Language Modeling] seems not as expected

# Problem
Causal Models is only attended to the left context. Therefore causal models should not depend on the right tokens.
For example, The word embedding of "I" will be unchanged no matter what is in the right In GPT2. Since Causal Language Model are uni-directional self-attention.

```
from transformers import AutoModel,AutoTokenizer, AutoConfig
import torch

# gpt
gpt_model = AutoModel.from_pretrained('gpt2')
gpt_tokenizer = AutoTokenizer.from_pretrained('gpt2')
embeddings = gpt_model.get_input_embeddings()

# create ids of encoded input vectors
decoder_input_ids = gpt_tokenizer("<pad> Ich will ein", return_tensors="pt", add_special_tokens=False).input_ids

# pass decoder input_ids and encoded input vectors to decoder
lm_logits = gpt_model(decoder_input_ids).last_hidden_state

# change the decoder input slightly
decoder_input_ids_perturbed = gpt_tokenizer("<pad> Ich will das", return_tensors="pt", add_special_tokens=False).input_ids
lm_logits_perturbed = gpt_model(decoder_input_ids_perturbed).last_hidden_state

# compare values of word embedding of "I" for input_ids and perturbed input_ids
print("Is encoding for `Ich` equal to its perturbed version?: ", torch.allclose(lm_logits[0, 0], lm_logits_perturbed[0, 0], atol=1e-3))
```
Result
```
Is encoding for `Ich` equal to its perturbed version?:  True
```

However, when it comes to other models, the result is not following the assumption, the logits will be changed when changing the right side input? What is the reason? Is it a bug? I really want to know the answer, thank you!

BERT
```
Is encoding for `Ich` equal to its perturbed version?:  False
```

BART
```
Is encoding for `Ich` equal to its perturbed version?:  False
```

Roberta
```
Is encoding for `Ich` equal to its perturbed version?:  False
```

Experiment notebook [colab](https://colab.research.google.com/drive/15V37RWAL40vhrk-uBIh9m99j1gZMLjUy?usp=sharing)

## Environment info

- `transformers` version: 4.3.3
- Platform: Linux-4.19.112+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.10
- PyTorch version (GPU?): 1.7.1+cu101 (False)
- Tensorflow version (GPU?): 2.4.1 (False)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help

- @patrickvonplaten
- @LysandreJik
-  @patil-suraj

<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @jplu

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

## Information

Model I am using (GPT, Bert, RoBerta, BART ForCausalLM):

The problem arises when using:
* [ x] the official example scripts: https://huggingface.co/blog/encoder-decoder#decoder

## To reproduce

Experiment notebook [colab](https://colab.research.google.com/drive/15V37RWAL40vhrk-uBIh9m99j1gZMLjUy?usp=sharing)

## Expected behavior

Causal Models should not be affected by the right context?


I have trained Bert on my own data which has been converted to IDs by using BertForMaskedLM, but when I use the model for the further fine-tuned, I found this error

@LysandreJik
## code info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
here's my model:

![image](https://user-images.githubusercontent.com/56223656/110211963-61276580-7e99-11eb-8508-42591b5138e6.png)

![image](https://user-images.githubusercontent.com/56223656/110212033-b2cff000-7e99-11eb-8378-116dafe78f8e.png)

## Information

Model I am using my own pre-tained Bert model which is stored in the path "../bert":

The problem arises when using:

![image](https://user-images.githubusercontent.com/56223656/110212083-fb87a900-7e99-11eb-9457-f4b94da1234b.png)

here are the files in the path "../bert":

![image](https://user-images.githubusercontent.com/56223656/110212157-55886e80-7e9a-11eb-8b7f-dce93af1b72e.png)

The tasks I am working on is: text matching:
* [ ] my own dataset is like:

![屏幕快照 2021-03-06 16 29 16](https://user-images.githubusercontent.com/56223656/110211923-2a514f80-7e99-11eb-86ae-c403a7e7001e.png)

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
what exactly is the problem?


Stale bot updated

This is an updated version of the stale bot.

**It is easier to review the file than the diff, you can find the file [here](https://github.com/huggingface/transformers/blob/d1e516ea0fe9e641a75a89e5a7522392f7dbd59d/scripts/stale.py).**

It sends a warning message after 23 days of inactivity, and closes the issue/PR if no activity is detected in the following 7 days.

It ignores the following labels (case insensitive):
- `Good First Issue`
- `Good Second Issue`
- `Feature Request`
- `New Model`
- `WIP`

If there are assignees on the issue/PR, then it puts the following comment: `f"This issue has been stale for a while, ping @{assignee.login}"`

I propose to leave the PR like it is, and I'll push an empty commit daily to check the result of the stale bot test (I'll remove other tests to ensure that we don't spend unnecessary CI credits). Once we verify that it works as expected for a few days, I propose to merge it.

You may check the results of the first run here: https://github.com/huggingface/transformers/runs/2045189559?check_suite_focus=true (Second commit was rate limited)


[examples tests on multigpu] resolving require_torch_non_multi_gpu_but_fix_me

A while ago I added `@require_torch_non_multi_gpu_but_fix_me` to quickly allow us to start running example tests on multigpu, so this PR resolves that temporary band-aid.

This PR:
* fixes a few tests to make them run on multi-gpu
* removes the decorator where it's not needed after testing that it works
* leaves the dropped legacy tests untouched - since they don't run on CI
* eliminates  `@require_torch_non_multi_gpu_but_fix_me` from existence since it's no longer needed
* the only test I couldn't figure out is https://github.com/huggingface/transformers/issues/10560 but it's not worse off than it was - added some refactoring to it and prepared it for multi-gpu if someone knows how to fix it

Note: 2 slow tests in `examples/test_examples.py` currently fail because of yet another missed thing in ported `run_seq2seq.py` - but these should be resolved by https://github.com/huggingface/transformers/pull/10551 once that one is merged, so we can merge this PR after it.

@LysandreJik


[examples] run_glue_deebert.py distrbuted fails

I'm working on making the tests work under multiple gpus and run into and this one that proved to be stubborn, for some reason it doesn't work under any DP scheme. I don't know anything about this script, To reproduce:

Note - you need at least 2 gpus:

Actually it fails with 1 gpu too (just change to --nproc_per_node=1)
```
python -m torch.distributed.launch --nproc_per_node=2 examples/research_projects/deebert/run_glue_deebert.py --model_type roberta --model_name_or_path roberta-base --task_name MRPC --do_train --do_eval --do_lower_case --data_dir ./tests/fixtures/tests_samples/MRPC/ --max_seq_length 128 --per_gpu_eval_batch_size=1 --per_gpu_train_batch_size=8 --learning_rate 2e-4 --num_train_epochs 3 --overwrite_output_dir --seed 42 --output_dir ./examples/deebert/saved_models/roberta-base/MRPC/two_stage --plot_data_dir ./examples/deebert/results/ --save_steps 0 --overwrite_cache --eval_after_first_stage

W reducer.cpp:1084] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
Traceback (most recent call last):
  File "examples/research_projects/deebert/run_glue_deebert.py", line 730, in <module>
    main()
  File "examples/research_projects/deebert/run_glue_deebert.py", line 645, in main
    global_step, tr_loss = train(args, train_dataset, model, tokenizer)
  File "examples/research_projects/deebert/run_glue_deebert.py", line 176, in train
    outputs = model(**inputs)
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 872, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 705, in forward
    if self.reducer._rebuild_buckets():
RuntimeError: Expected to have finished reduction in the prior iteration before starting a new one. This error indicates that your module has parameters that were not used in producing loss. Since `find_unused_parameters=True` is enabled, this likely  means that not all `forward` outputs participate in computing loss. You can fix this by making sure all `forward` function outputs participate in calculating loss.
If you already have done the above, then the distributed data parallel module wasn't able to locate the output tensors in the return value of your module's `forward` function. Please include the loss function and the structure of the return value of `forward` of your module when reporting this issue (e.g. list, dict, iterable).
[W reducer.cpp:1084] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
Iteration: 100%1/1 [00:00<00:00,  1.83it/s]
Iteration:   0%|                                                                                | 0/1 [00:00<?, ?it/s]
Epoch:  33                                                                      | 1/3 [00:00<00:01,  1.82it/s]
Traceback (most recent call last):
  File "examples/research_projects/deebert/run_glue_deebert.py", line 730, in <module>
    main()
  File "examples/research_projects/deebert/run_glue_deebert.py", line 645, in main
    global_step, tr_loss = train(args, train_dataset, model, tokenizer)
  File "examples/research_projects/deebert/run_glue_deebert.py", line 176, in train
    outputs = model(**inputs)
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 872, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 705, in forward
    if self.reducer._rebuild_buckets():
RuntimeError: Expected to have finished reduction in the prior iteration before starting a new one. This error indicates that your module has parameters that were not used in producing loss. Since `find_unused_parameters=True` is enabled, this likely  means that not all `forward` outputs participate in computing loss. You can fix this by making sure all `forward` function outputs participate in calculating loss.
If you already have done the above, then the distributed data parallel module wasn't able to locate the output tensors in the return value of your module's `forward` function. Please include the loss function and the structure of the return value of `forward` of your module when reporting this issue (e.g. list, dict, iterable).
Killing subprocess 2242528
Killing subprocess 2242529
Traceback (most recent call last):
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/distributed/launch.py", line 340, in <module>
    main()
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/distributed/launch.py", line 326, in main
    sigkill_handler(signal.SIGTERM, None)  # not coming back
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/distributed/launch.py", line 301, in sigkill_handler
    raise subprocess.CalledProcessError(returncode=last_return_code, cmd=cmd)
subprocess.CalledProcessError: Command '['/home/stas/anaconda3/envs/main-38/bin/python', '-u', 'examples/research_projects/deebert/run_glue_deebert.py', '--local_rank=1', '--model_type', 'roberta', '--model_name_or_path', 'roberta-base', '--task_name', 'MRPC', '--do_train', '--do_eval', '--do_lower_case', '--data_dir', './tests/fixtures/tests_samples/MRPC/', '--max_seq_length', '128', '--per_gpu_eval_batch_size=1', '--per_gpu_train_batch_size=8', '--learning_rate', '2e-4', '--num_train_epochs', '3', '--overwrite_output_dir', '--seed', '42', '--output_dir', './examples/deebert/saved_models/roberta-base/MRPC/two_stage', '--plot_data_dir', './examples/deebert/results/', '--save_steps', '0', '--overwrite_cache', '--eval_after_first_stage']' returned non-zero exit status 1.
```

@LysandreJik


[website] installation doc blues

Switching docs to a different branch seems to be broken from at least one doc, e.g. going from
https://huggingface.co/transformers/master/installation.html#caching-models
upper left corner - if you want to switch to a different branch it sends you to a 404 page on all of them:

it has an issue with the base url, note how it links to: https://huggingface.co/transformers/master/master
master twice, so now it prefixes all branches with master/branch

ok narrowed it down - it happens specifically on the installation page: https://huggingface.co/transformers/master/installation.html
I tried a bunch of other pages and it seems to be ok there.

oddly I don't see anything unusual about installation.md in toctree or its content.

if you look at the snapshot - instead of showing `master` for the currently selected version it shows a chunk of the url instead.

![snapshot_8](https://user-images.githubusercontent.com/10676103/110193531-41d0ff80-7de9-11eb-9697-293daa527983.png)

@sgugger


Dear developer, does transformers have the support to translate Chinese text into English?

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

## Motivation

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->

## Your contribution

<!-- Is there any way that you could help, e.g. by submitting a PR?
     Make sure to read the CONTRIBUTING.MD readme:
     https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md -->


[RAG] Expected RAG output after fine tuning

Hi there.

Perhaps the following isn’t even a real issue, but I’m a bit confused with the current outputs I got.

I’m trying to fine tune RAG on a bunch of question-answer pairs I have (for while, not that much, < 1k ones). I have splitted them as suggested (train.source, train.target, val.source…). After running the ```finetune_rag.py```, the outputs generated were **only two files (~2 kB)**:

- git_log.json
- hparams.pkl

Is that right? Because I was expecting *a big binary file or something like that containing the weight matrices*, so I could use them afterwards in a new trial.

Could you please tell me what’s the point I’m missing here?

----------------------

I provide more details below. Btw, I have two NVIDIA RTX 3090, 24GB each, but they were barely used in the whole process (which took ~3 hours).

**Command:**
```
python finetune_rag.py \
    --data_dir rag_manual_qa_finetuning \
    --output_dir output_ft \
    --model_name_or_path rag-sequence-base \
    --model_type rag_sequence \
    --gpus 2 \
    --distributed_retriever pytorch
```

**Logs** (in fact, it’s strange but the logs even seem to be generated in duplicate - I don’t know why):
```
loading configuration file rag-sequence-base/config.json
Model config RagConfig {
  "architectures": [
    "RagSequenceForGeneration"
  ],
  "dataset": "wiki_dpr",
  "dataset_split": "train",
  "do_deduplication": true,
  "do_marginalize": false,
  "doc_sep": " // ",
  "exclude_bos_score": false,
  "forced_eos_token_id": 2,
  "generator": {
    "_name_or_path": "",
    "_num_labels": 3,
    "activation_dropout": 0.0,
    "activation_function": "gelu",
    "add_bias_logits": false,
    "add_cross_attention": false,
    "add_final_layer_norm": false,
    "architectures": [
      "BartModel",
      "BartForMaskedLM",
      "BartForSequenceClassification"
    ],
    "attention_dropout": 0.0,
    "bad_words_ids": null,
    "bos_token_id": 0,
    "chunk_size_feed_forward": 0,
    "classif_dropout": 0.0,
    "classifier_dropout": 0.0,
    "d_model": 1024,
    "decoder_attention_heads": 16,
    "decoder_ffn_dim": 4096,
    "decoder_layerdrop": 0.0,
    "decoder_layers": 12,
    "decoder_start_token_id": 2,
    "diversity_penalty": 0.0,
    "do_sample": false,
    "dropout": 0.1,
    "early_stopping": false,
    "encoder_attention_heads": 16,
    "encoder_ffn_dim": 4096,
    "encoder_layerdrop": 0.0,
    "encoder_layers": 12,
    "encoder_no_repeat_ngram_size": 0,
    "eos_token_id": 2,
    "extra_pos_embeddings": 2,
    "finetuning_task": null,
    "force_bos_token_to_be_generated": false,
    "forced_bos_token_id": null,
    "forced_eos_token_id": 2,
    "gradient_checkpointing": false,
    "id2label": {
      "0": "LABEL_0",
      "1": "LABEL_1",
      "2": "LABEL_2"
    },
    "init_std": 0.02,
    "is_decoder": false,
    "is_encoder_decoder": true,
    "label2id": {
      "LABEL_0": 0,
      "LABEL_1": 1,
      "LABEL_2": 2
    },
    "length_penalty": 1.0,
    "max_length": 20,
    "max_position_embeddings": 1024,
    "min_length": 0,
    "model_type": "bart",
    "no_repeat_ngram_size": 0,
    "normalize_before": false,
    "normalize_embedding": true,
    "num_beam_groups": 1,
    "num_beams": 1,
    "num_hidden_layers": 12,
    "num_return_sequences": 1,
    "output_attentions": false,
    "output_hidden_states": false,
    "output_past": false,
    "output_scores": false,
    "pad_token_id": 1,
    "prefix": " ",
    "pruned_heads": {},
    "repetition_penalty": 1.0,
    "return_dict": false,
    "return_dict_in_generate": false,
    "scale_embedding": false,
    "sep_token_id": null,
    "static_position_embeddings": false,
    "task_specific_params": {
      "summarization": {
        "early_stopping": true,
        "length_penalty": 2.0,
        "max_length": 142,
        "min_length": 56,
        "no_repeat_ngram_size": 3,
        "num_beams": 4
      }
    },
    "temperature": 1.0,
    "tie_encoder_decoder": false,
    "tie_word_embeddings": true,
    "tokenizer_class": null,
    "top_k": 50,
    "top_p": 1.0,
    "torchscript": false,
    "transformers_version": "4.4.0.dev0",
    "use_bfloat16": false,
    "use_cache": true,
    "vocab_size": 50265
  },
  "index_name": "exact",
  "index_path": null,
  "is_encoder_decoder": true,
  "label_smoothing": 0.0,
  "max_combined_length": 300,
  "model_type": "rag",
  "n_docs": 5,
  "output_retrieved": false,
  "passages_path": null,
  "question_encoder": {
    "_name_or_path": "",
    "add_cross_attention": false,
    "architectures": [
      "DPRQuestionEncoder"
    ],
    "attention_probs_dropout_prob": 0.1,
    "bad_words_ids": null,
    "bos_token_id": null,
    "chunk_size_feed_forward": 0,
    "decoder_start_token_id": null,
    "diversity_penalty": 0.0,
    "do_sample": false,
    "early_stopping": false,
    "encoder_no_repeat_ngram_size": 0,
    "eos_token_id": null,
    "finetuning_task": null,
    "forced_bos_token_id": null,
    "forced_eos_token_id": null,
    "gradient_checkpointing": false,
    "hidden_act": "gelu",
    "hidden_dropout_prob": 0.1,
    "hidden_size": 768,
    "id2label": {
      "0": "LABEL_0",
      "1": "LABEL_1"
    },
    "initializer_range": 0.02,
    "intermediate_size": 3072,
    "is_decoder": false,
    "is_encoder_decoder": false,
    "label2id": {
      "LABEL_0": 0,
      "LABEL_1": 1
    },
    "layer_norm_eps": 1e-12,
    "length_penalty": 1.0,
    "max_length": 20,
    "max_position_embeddings": 512,
    "min_length": 0,
    "model_type": "dpr",
    "no_repeat_ngram_size": 0,
    "num_attention_heads": 12,
    "num_beam_groups": 1,
    "num_beams": 1,
    "num_hidden_layers": 12,
    "num_return_sequences": 1,
    "output_attentions": false,
    "output_hidden_states": false,
    "output_scores": false,
    "pad_token_id": 0,
    "position_embedding_type": "absolute",
    "prefix": null,
    "projection_dim": 0,
    "pruned_heads": {},
    "repetition_penalty": 1.0,
    "return_dict": false,
    "return_dict_in_generate": false,
    "sep_token_id": null,
    "task_specific_params": null,
    "temperature": 1.0,
    "tie_encoder_decoder": false,
    "tie_word_embeddings": true,
    "tokenizer_class": null,
    "top_k": 50,
    "top_p": 1.0,
    "torchscript": false,
    "transformers_version": "4.4.0.dev0",
    "type_vocab_size": 2,
    "use_bfloat16": false,
    "use_cache": true,
    "vocab_size": 30522
  },
  "reduce_loss": false,
  "retrieval_batch_size": 8,
  "retrieval_vector_size": 768,
  "title_sep": " / ",
  "use_cache": true,
  "use_dummy_dataset": false,
  "vocab_size": null
}

Model name 'rag-sequence-base' not found in model shortcut name list (facebook/dpr-question_encoder-single-nq-base, facebook/dpr-question_encoder-multiset-base). Assuming 'rag-sequence-base' is a path, a model identifier, or url to a directory containing tokenizer files.
Didn't find file rag-sequence-base/question_encoder_tokenizer/tokenizer.json. We won't load it.
Didn't find file rag-sequence-base/question_encoder_tokenizer/added_tokens.json. We won't load it.
loading file rag-sequence-base/question_encoder_tokenizer/vocab.txt
loading file None
loading file None
loading file rag-sequence-base/question_encoder_tokenizer/special_tokens_map.json
loading file rag-sequence-base/question_encoder_tokenizer/tokenizer_config.json
Model name 'rag-sequence-base' not found in model shortcut name list (facebook/bart-base, facebook/bart-large, facebook/bart-large-mnli, facebook/bart-large-cnn, facebook/bart-large-xsum, yjernite/bart_eli5). Assuming 'rag-sequence-base' is a path, a model identifier, or url to a directory containing tokenizer files.
Didn't find file rag-sequence-base/generator_tokenizer/tokenizer.json. We won't load it.
Didn't find file rag-sequence-base/generator_tokenizer/added_tokens.json. We won't load it.
loading file rag-sequence-base/generator_tokenizer/vocab.json
loading file rag-sequence-base/generator_tokenizer/merges.txt
loading file None
loading file None
loading file rag-sequence-base/generator_tokenizer/special_tokens_map.json
loading file rag-sequence-base/generator_tokenizer/tokenizer_config.json
Loading passages from wiki_dpr
Downloading: 9.64kB [00:00, 10.8MB/s]
Downloading: 67.5kB [00:00, 59.5MB/s]
WARNING:datasets.builder:Using custom data configuration psgs_w100.nq.no_index-dummy=False,with_index=False
Downloading and preparing dataset wiki_dpr/psgs_w100.nq.no_index (download: 66.09 GiB, generated: 73.03 GiB, post-processed: Unknown size, total: 139.13 GiB) to /home/usp/.cache/huggingface/datasets/wiki_dpr/psgs_w100.nq.no_index-dummy=False,with_index=False/0.0.0/91b145e64f5bc8b55a7b3e9f730786ad6eb19cd5bc020e2e02cdf7d0cb9db9c1...
Downloading: 100%|█████████████████████████| 4.69G/4.69G [07:11<00:00, 10.9MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [02:27<00:00, 9.00MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [02:36<00:00, 8.47MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [02:37<00:00, 8.41MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [02:38<00:00, 8.36MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [02:40<00:00, 8.25MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [02:58<00:00, 7.45MB/s]

Downloading: 100%|█████████████████████████| 1.33G/1.33G [02:58<00:00, 7.43MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [03:00<00:00, 7.34MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [03:04<00:00, 7.17MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [03:05<00:00, 7.13MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [03:07<00:00, 7.06MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [03:10<00:00, 6.94MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [03:24<00:00, 6.48MB/s]
Downloading: 100%|█████████████████████████| 1.32G/1.32G [03:27<00:00, 6.38MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [03:33<00:00, 6.21MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [04:57<00:00, 4.45MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [02:36<00:00, 8.47MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [02:28<00:00, 8.94MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [02:44<00:00, 8.03MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [02:55<00:00, 7.54MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [02:28<00:00, 8.92MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [02:28<00:00, 8.90MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [02:56<00:00, 7.49MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [03:19<00:00, 6.63MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [02:53<00:00, 7.63MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [03:00<00:00, 7.33MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [03:11<00:00, 6.92MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [03:14<00:00, 6.80MB/s]
Downloading: 100%|█████████████████████████| 1.32G/1.32G [03:06<00:00, 7.10MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [03:35<00:00, 6.16MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [03:50<00:00, 5.76MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [02:28<00:00, 8.93MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [02:32<00:00, 8.67MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [03:07<00:00, 7.05MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [02:53<00:00, 7.62MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [03:22<00:00, 6.56MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [02:47<00:00, 7.93MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [02:26<00:00, 9.06MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [02:40<00:00, 8.25MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [02:42<00:00, 8.17MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [02:54<00:00, 7.59MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [02:41<00:00, 8.22MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [03:18<00:00, 6.69MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [02:30<00:00, 8.83MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [03:00<00:00, 7.34MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [02:20<00:00, 9.44MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [02:24<00:00, 9.19MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [02:21<00:00, 9.38MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [02:18<00:00, 9.59MB/s]
Downloading: 100%|█████████████████████████| 1.33G/1.33G [02:19<00:00, 9.53MB/s]
0 examples [00:00, ? examples/s]2021-03-05 12:11:39.666323: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
Dataset wiki_dpr downloaded and prepared to /home/usp/.cache/huggingface/datasets/wiki_dpr/psgs_w100.nq.no_index-dummy=False,with_index=False/0.0.0/91b145e64f5bc8b55a7b3e9f730786ad6eb19cd5bc020e2e02cdf7d0cb9db9c1. Subsequent calls will reuse this data.
loading weights file rag-sequence-base/pytorch_model.bin
All model checkpoint weights were used when initializing RagSequenceForGeneration.

Some weights of RagSequenceForGeneration were not initialized from the model checkpoint at rag-sequence-base and are newly initialized: ['rag.generator.lm_head.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
loading configuration file rag-sequence-base/config.json
Model config RagConfig {
  "architectures": [
    "RagSequenceForGeneration"
  ],
  "dataset": "wiki_dpr",
  "dataset_split": "train",
  "do_deduplication": true,
  "do_marginalize": false,
  "doc_sep": " // ",
  "exclude_bos_score": false,
  "forced_eos_token_id": 2,
  "generator": {
    "_name_or_path": "",
    "_num_labels": 3,
    "activation_dropout": 0.0,
    "activation_function": "gelu",
    "add_bias_logits": false,
    "add_cross_attention": false,
    "add_final_layer_norm": false,
    "architectures": [
      "BartModel",
      "BartForMaskedLM",
      "BartForSequenceClassification"
    ],
    "attention_dropout": 0.0,
    "bad_words_ids": null,
    "bos_token_id": 0,
    "chunk_size_feed_forward": 0,
    "classif_dropout": 0.0,
    "classifier_dropout": 0.0,
    "d_model": 1024,
    "decoder_attention_heads": 16,
    "decoder_ffn_dim": 4096,
    "decoder_layerdrop": 0.0,
    "decoder_layers": 12,
    "decoder_start_token_id": 2,
    "diversity_penalty": 0.0,
    "do_sample": false,
    "dropout": 0.1,
    "early_stopping": false,
    "encoder_attention_heads": 16,
    "encoder_ffn_dim": 4096,
    "encoder_layerdrop": 0.0,
    "encoder_layers": 12,
    "encoder_no_repeat_ngram_size": 0,
    "eos_token_id": 2,
    "extra_pos_embeddings": 2,
    "finetuning_task": null,
    "force_bos_token_to_be_generated": false,
    "forced_bos_token_id": null,
    "forced_eos_token_id": 2,
    "gradient_checkpointing": false,
    "id2label": {
      "0": "LABEL_0",
      "1": "LABEL_1",
      "2": "LABEL_2"
    },
    "init_std": 0.02,
    "is_decoder": false,
    "is_encoder_decoder": true,
    "label2id": {
      "LABEL_0": 0,
      "LABEL_1": 1,
      "LABEL_2": 2
    },
    "length_penalty": 1.0,
    "max_length": 20,
    "max_position_embeddings": 1024,
    "min_length": 0,
    "model_type": "bart",
    "no_repeat_ngram_size": 0,
    "normalize_before": false,
    "normalize_embedding": true,
    "num_beam_groups": 1,
    "num_beams": 1,
    "num_hidden_layers": 12,
    "num_return_sequences": 1,
    "output_attentions": false,
    "output_hidden_states": false,
    "output_past": false,
    "output_scores": false,
    "pad_token_id": 1,
    "prefix": " ",
    "pruned_heads": {},
    "repetition_penalty": 1.0,
    "return_dict": false,
    "return_dict_in_generate": false,
    "scale_embedding": false,
    "sep_token_id": null,
    "static_position_embeddings": false,
    "task_specific_params": {
      "summarization": {
        "early_stopping": true,
        "length_penalty": 2.0,
        "max_length": 142,
        "min_length": 56,
        "no_repeat_ngram_size": 3,
        "num_beams": 4
      }
    },
    "temperature": 1.0,
    "tie_encoder_decoder": false,
    "tie_word_embeddings": true,
    "tokenizer_class": null,
    "top_k": 50,
    "top_p": 1.0,
    "torchscript": false,
    "transformers_version": "4.4.0.dev0",
    "use_bfloat16": false,
    "use_cache": true,
    "vocab_size": 50265
  },
  "index_name": "exact",
  "index_path": null,
  "is_encoder_decoder": true,
  "label_smoothing": 0.0,
  "max_combined_length": 300,
  "model_type": "rag",
  "n_docs": 5,
  "output_retrieved": false,
  "passages_path": null,
  "question_encoder": {
    "_name_or_path": "",
    "add_cross_attention": false,
    "architectures": [
      "DPRQuestionEncoder"
    ],
    "attention_probs_dropout_prob": 0.1,
    "bad_words_ids": null,
    "bos_token_id": null,
    "chunk_size_feed_forward": 0,
    "decoder_start_token_id": null,
    "diversity_penalty": 0.0,
    "do_sample": false,
    "early_stopping": false,
    "encoder_no_repeat_ngram_size": 0,
    "eos_token_id": null,
    "finetuning_task": null,
    "forced_bos_token_id": null,
    "forced_eos_token_id": null,
    "gradient_checkpointing": false,
    "hidden_act": "gelu",
    "hidden_dropout_prob": 0.1,
    "hidden_size": 768,
    "id2label": {
      "0": "LABEL_0",
      "1": "LABEL_1"
    },
    "initializer_range": 0.02,
    "intermediate_size": 3072,
    "is_decoder": false,
    "is_encoder_decoder": false,
    "label2id": {
      "LABEL_0": 0,
      "LABEL_1": 1
    },
    "layer_norm_eps": 1e-12,
    "length_penalty": 1.0,
    "max_length": 20,
    "max_position_embeddings": 512,
    "min_length": 0,
    "model_type": "dpr",
    "no_repeat_ngram_size": 0,
    "num_attention_heads": 12,
    "num_beam_groups": 1,
    "num_beams": 1,
    "num_hidden_layers": 12,
    "num_return_sequences": 1,
    "output_attentions": false,
    "output_hidden_states": false,
    "output_scores": false,
    "pad_token_id": 0,
    "position_embedding_type": "absolute",
    "prefix": null,
    "projection_dim": 0,
    "pruned_heads": {},
    "repetition_penalty": 1.0,
    "return_dict": false,
    "return_dict_in_generate": false,
    "sep_token_id": null,
    "task_specific_params": null,
    "temperature": 1.0,
    "tie_encoder_decoder": false,
    "tie_word_embeddings": true,
    "tokenizer_class": null,
    "top_k": 50,
    "top_p": 1.0,
    "torchscript": false,
    "transformers_version": "4.4.0.dev0",
    "type_vocab_size": 2,
    "use_bfloat16": false,
    "use_cache": true,
    "vocab_size": 30522
  },
  "reduce_loss": false,
  "retrieval_batch_size": 8,
  "retrieval_vector_size": 768,
  "title_sep": " / ",
  "use_cache": true,
  "use_dummy_dataset": false,
  "vocab_size": null
}

Model name 'rag-sequence-base' not found in model shortcut name list (facebook/dpr-question_encoder-single-nq-base, facebook/dpr-question_encoder-multiset-base). Assuming 'rag-sequence-base' is a path, a model identifier, or url to a directory containing tokenizer files.
Didn't find file rag-sequence-base/question_encoder_tokenizer/tokenizer.json. We won't load it.
Didn't find file rag-sequence-base/question_encoder_tokenizer/added_tokens.json. We won't load it.
loading file rag-sequence-base/question_encoder_tokenizer/vocab.txt
loading file None
loading file None
loading file rag-sequence-base/question_encoder_tokenizer/special_tokens_map.json
loading file rag-sequence-base/question_encoder_tokenizer/tokenizer_config.json

Model name 'rag-sequence-base' not found in model shortcut name list (facebook/bart-base, facebook/bart-large, facebook/bart-large-mnli, facebook/bart-large-cnn, facebook/bart-large-xsum, yjernite/bart_eli5). Assuming 'rag-sequence-base' is a path, a model identifier, or url to a directory containing tokenizer files.
Didn't find file rag-sequence-base/generator_tokenizer/tokenizer.json. We won't load it.
Didn't find file rag-sequence-base/generator_tokenizer/added_tokens.json. We won't load it.
loading file rag-sequence-base/generator_tokenizer/vocab.json
loading file rag-sequence-base/generator_tokenizer/merges.txt
loading file None
loading file None
loading file rag-sequence-base/generator_tokenizer/special_tokens_map.json
loading file rag-sequence-base/generator_tokenizer/tokenizer_config.json
GPU available: True, used: True
INFO:lightning:GPU available: True, used: True
TPU available: False, using: 0 TPU cores
INFO:lightning:TPU available: False, using: 0 TPU cores
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0,1]
INFO:lightning:LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0,1]
```


Layoutlm tf

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

This PR adds TF version of LayoutLM for issue [(10312)](https://github.com/huggingface/transformers/issues/10312)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [X] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [X] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @jplu

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Add new GLUE example with no Trainer.

# What does this PR do?

This PR adds a new GLUE example that does not use the `Trainer`, leveraging [accelerate](https://github.com/huggingface/accelerate) for the distributed training. The necessary instructions are added in the text-classification README.

cc @JetRunner as it should be of interest to you.


Fixed dead link in Trainer documentation

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #10548

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @jplu

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Transformers upgrade

Transformers upgrade - redoing all ec-ml related changes


Handle padding in decoder_inputs_id when using generate

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #10478 for pytorch version of `generate()` in generate_utils.py

As described in the issue, when `decoder_input_ids` have different lengths and require padding, generation continues after the padding tokens. This PR modifies that behavior so that tokens are generated before the padding for each element in the batch.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

I am adding those who I see more often in the git blame of the file:
@patrickvonplaten, @patil-suraj, @yjernite

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @jplu

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

This is my first PR, so let me use it to thank everyone involved in this library for all the cool work :)


Added max_sample_ arguments

Examples

Which is related to examples in general

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #10437 #10423

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)?  #10437
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Notes:
All the PyTorch-based examples except the below two files will have support for the arguments by adding these changes.
1. The same changes can be implemented for `run_mlm_flax.py` but since I couldn't test the changes I didn't make changes to that file.
2. `run_generation.py`
* I have reverted the code changes for three TF-based examples since it was giving an error and we want to keep it as it is.
* Test/Predict code addition is still pending. I will do it next.
## review:
@stas00  @sgugger

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @jplu

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


How to get best model from hyperparameter search easily

Hi,

after doing a hyperparameter search(by calling `hyperparameter_search`on the trainer object) , I asked myself how to easily get the best model out of it. Currently, I'm using Ray Tune as a backend.
Given [the code in integrations.py](https://github.com/huggingface/transformers/blob/54e55b52d4886d4c63e592310b4253e01c606285/src/transformers/integrations.py#L238) the trial id, objective and chosen hyperparameters are stored in class [BestRun](https://github.com/huggingface/transformers/blob/54e55b52d4886d4c63e592310b4253e01c606285/src/transformers/trainer_utils.py#L116) which is then returned by the hyperparameter_search function. But here the model is somehow missing or am I wrong? One option would be to retrained from the given hyperparameters but this is not possible in PBT because the perturbation is applied during hyperparameter search (and cannot be repeated).

The only thing I currently see is to load the model based on the run_id and compose the corresponding file path.
But maybe there is an easier way to do it (or is this the expected way?). I also tried out the parameter `load_best_model_at_end=True` in [TrainingArguments](https://huggingface.co/transformers/main_classes/trainer.html#trainingarguments) which is used in [this example](https://docs.ray.io/en/master/tune/examples/pbt_transformers.html) but it does not help.
One proposal would be to add a parameter in BestRun which would contain the model or even better load it directly into the trainer (such that the `predict` and `save_model` functions also work).
Is this a reasonable feature request? In case yes, I'd be happy to create a pull request.

In other documentations they show how they extract the checkpoint directly:
- [Ray Docs](https://docs.ray.io/en/master/tune/tutorials/tune-serve-integration-mnist.html#configuring-the-search-space-and-starting-ray-tune)
```
best_trial = analysis.get_best_trial("mean_accuracy", "max", "last")
best_accuracy = best_trial.metric_analysis["mean_accuracy"]["last"]
best_trial_config = best_trial.config
best_checkpoint = best_trial.checkpoint.value
```
- [Colab example for PBT](https://colab.research.google.com/drive/1tQgAKgcKQzheoh503OzhS4N9NtfFgmjF?usp=sharing#scrollTo=TxKyvQ6WNlvG)
```
best_config = analysis.get_best_config(metric="eval_acc", mode="max")
print(best_config)
best_checkpoint = recover_checkpoint(
    analysis.get_best_trial(metric="eval_acc",
                            mode="max").checkpoint.value)
```

Best regards
Sven


Fix embeddings for PyTorch 1.8

# What does this PR do?

This PR fixes several embeddings layer with the recent breaking change introduced in PyTorch 1.8. Up until PyTorch 1.7, the `padding_idx` passed to an embedding layer was used to initialize the corresponding row in the weights to 0 but ignored afterwards.

Now, this `padding_idx` is used at every forward pass and ignores the potential weights of the padding index (spoiler alert, all pretrained models I checked have a nonzero one).

To solve this, this PR removes all `padding_idx` passed to embeddings layer. As we were in any case re-initializing them in the `_init_weights` function, the zero weight for that index was ignored in any case. This PR thus introduces no breaking change on our side while dealing with the breaking change in PyTorch.


Dead link to optuna.create_study under hyperparamter_search in Trainer

I noticed the hyperlink to the documentation of optuna's create_study under ```kwargs``` in the ```hyperparameter_search``` method of Trainer is outdated.

https://huggingface.co/transformers/main_classes/trainer.html

### Who can help
Documentation: @sgugger

New URL (I'm guessing): https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.create_study.html
Old (currently used): https://optuna.readthedocs.io/en/stable/reference/alias_generated/optuna.create_study.html#optuna.create_study


[Wav2Vec2 Example Script] Typo

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fix a typo. Script should be as generic as possible

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @jplu

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->