RuntimeError with DistributedDataParallel

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.3.1
- Platform: Ubuntu 18.04
- Python version: 3.7.7.final.0
- PyTorch version (GPU?): 1.6.0 gpu
- Tensorflow version (GPU?): X
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: distributed set-up
- Conda : 4.8.3
- cuda =11.0

### Who can help
 @LysandreJik  @sgugger

## Information

Model I am using (Bert, XLNet ...): bert mulitilingual cased

The problem arises when using:
* [x] my own modified scripts : Triplet Loss applied to sentence similarity, to practice writing efficient distributed learning code.

The tasks I am working on is:
* [x] my own task or dataset: Triplet Loss applied to sentence similarity, to practice writing efficient distributed learning code.

## To reproduce

I have not been able to generte a minimal version yet.

Stack trace

```
Traceback (most recent call last):
[...]
  File "my_program.py", line 246, in epoch
    loss.backward()
  File "conda/lib/python3.7/site-packages/torch/tensor.py", line 185, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "conda/lib/python3.7/site-packages/torch/autograd/__init__.py", line 127, in backward
    allow_unreachable=True)  # allow_unreachable flag

RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.LongTensor [1, 200]] is at version 9; expected version 7 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).
```

My model :

```python

def __init__(self, trainfile, devfile, outdir, rank=0):
         # default config
          self.max_sent_len = 200
          self.batchsize = 10
          self.lr = 2e-5
          self.accumulated_batches = 6
          self.max_epochs = 10
          self.seed = 42
          self.embs = "bert-base-multilingual-cased"
          self.tokenizer, self.model = self.transformers(self.embs)
          self.tripletloss = lossless_triplet_loss.LosslessTripletLoss()

          print("use %s gpus" % torch.cuda.device_count())
          self.model = self.model.to(rank)
          self.model = torch.nn.parallel.DistributedDataParallel(self.model, device_ids=[rank], output_device=rank, find_unused_para  meters=True)
          self.rank = rank

          half = False
          self.model = self.model.to(self.rank)

          self.optimizer = optim.Adam(filter(lambda param: param.requires_grad, self.model.parameters()), lr=self.lr)

          self.traindataset = data.TripleDataset2(trainfile, self.tokenizer, self.max_sent_len)
          self.devdataset = data.TripleDataset2(devfile, self.tokenizer, self.max_sent_len)
          self.traindataloader = DataLoader(self.traindataset,
                                            collate_fn=self.collate_fn,
                                            batch_size = self.batchsize,
                                            num_workers = 0)
          self.devdataloader = DataLoader(self.devdataset,
                                          collate_fn=self.collate_fn,
                                          batch_size = self.batchsize,
                                         num_workers = 0)

 def train(self, epoch):
          self.model.train()

          iterations = int(math.ceil(len(self.traindataset)/self.batchsize))
          iterator = tqdm.tqdm(enumerate(self.traindataloader),
                               ncols=100,
                               ascii=True,
                               desc="epoch %d" % (epoch),
                               mininterval=1.0,
                               total = iterations)
          lastloss = None
          for batch_idx, anchor_pos_neg in iterator:
              (anchor, amask), (pos, pmask), (neg, nmask) = anchor_pos_neg

              anchor = anchor.to(self.rank)
              pos = pos.to(self.rank)
              neg = neg.to(self.rank)

              amask = amask.to(self.rank)
              pmask = pmask.to(self.rank)
              nmask = nmask.to(self.rank)

              anchor_vec = self.tokens2vec(anchor, attention_mask=amask)
              pos_vec = self.tokens2vec(pos, attention_mask=pmask)
              neg_vec = self.tokens2vec(neg, attention_mask=nmask)
              loss, distp, distn = self.calc_triplet_loss(anchor_vec, pos_vec, neg_vec)

              loss.backward()
              if (batch_idx+1) % self.accumulated_batches == 0:
                  self.optimizer.step()
                  self.optimizer.zero_grad()

      def tokens2vec(self, tokens_tensor, attention_mask=None):
          hidden_states = self.model(tokens_tensor, attention_mask=attention_mask)[0]
          token_vecs = torch.squeeze(hidden_states, dim=0)

          a = torch.sum(torch.mul(token_vecs, torch.unsqueeze(attention_mask, 2)), dim=1)
          m = torch.unsqueeze(torch.sum(attention_mask, dim=1), 1)
          sentence_embedding = torch.sigmoid(torch.div(a, m))
          return sentence_embedding

      def transformers(self, name):
          if name.startswith("bert"):
              from transformers import BertTokenizer, BertModel
              tokenizer = BertTokenizer.from_pretrained(name, do_lower_case=False)
              model = BertModel.from_pretrained(name)
          else:
              tokenizer = None
              model = None
              print("ERROR: invalid embeddings name <%s>" % name, file=sys.stderr)
          return tokenizer, model

```

The error happens with batch_idx = 0.

200 is my maximum sequence length. I only use Longtensor as input to BertModel (for anchor, pos and neg).
What I find strange is that I am sending samples in batches of size 10, so I am wondering if my problem is really with the embedding layer, or rather caused by my custom collate_fn  :

```python
  def collate_fn(self, batch):
          anchor = []
          pos = []
          neg = []
          anchor_mask = []
          pos_mask = []
          neg_mask = []
          for a,p,n in batch:
              anchor.append(a[0])
              pos.append(p[0])
              neg.append(n[0])
              anchor_mask.append(a[1])
              pos_mask.append(p[1])
              neg_mask.append(n[1])
          return (torch.stack(anchor, dim=0),torch.stack(anchor_mask, dim=0)), \
               (torch.stack(pos, dim=0),torch.stack(pos_mask, dim=0)), \
               (torch.stack(neg, dim=0),torch.stack(neg_mask, dim=0))
```

My model is working well with only one GPU, or when using DataParallel. My issue arises only with DistributedDataParallel.

## Expected behavior
A running training.


Where do the models go in colab?

## Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 documentation: @sgugger
 -->

## Information

Model I am using (Bert):

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [.] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [.] my own task or dataset: (give details below)
Semeval 2017 task 10 keyword boundary detection

## To reproduce

Steps to reproduce the behavior:

1. Run `train.py` from [this repo](https://github.com/pranav-ust/BERT-keyphrase-extraction)
2. Load the model as `model = BertForTokenClassification.from_pretrained('bert-base-uncased', num_labels=len(params.tag2idx))`
3. Run `eval.py` as `python evaluate.py --data_dir data/task1/ --bert_model_dir model/ --model_dir experiments/base_model --restore_file best`

This gives the following error:
`Traceback (most recent call last):
  File "evaluate.py", line 117, in <module>
    config = BertConfig.from_json_file(config_path)
  File "/usr/local/lib/python3.6/dist-packages/transformers/configuration_utils.py", line 436, in from_json_file
    config_dict = cls._dict_from_json_file(json_file)
  File "/usr/local/lib/python3.6/dist-packages/transformers/configuration_utils.py", line 441, in _dict_from_json_file
    with open(json_file, "r", encoding="utf-8") as reader:
FileNotFoundError: [Errno 2] No such file or directory: 'model/bert_config.json'
2020-10-16 07:41:50.582535: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
Loading the dataset...
   done.
Traceback (most recent call last):
  File "evaluate.py", line 117, in <module>
    config = BertConfig.from_json_file(config_path)
  File "/usr/local/lib/python3.6/dist-packages/transformers/configuration_utils.py", line 436, in from_json_file
    config_dict = cls._dict_from_json_file(json_file)
  File "/usr/local/lib/python3.6/dist-packages/transformers/configuration_utils.py", line 441, in _dict_from_json_file
    with open(json_file, "r", encoding="utf-8") as reader:
FileNotFoundError: [Errno 2] No such file or directory: 'model/bert_config.json'`

Which I believe is caused by me passing `model/` as `--bert-model-dir` argument.
Where do the downloaded models go on colab? Or more importantly, can I pass a path argument to download them in a specific folder?

### Edit
If I download a model as `!wget https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased.tar.gz` and extract it in the model/ folder, then `python eval.py` gives the following error:
`Loading the dataset...
    - done.
 Traceback (most recent call last):
       File "evaluate.py", line 118, in <module>
       model = BertForTokenClassification(config, num_labels=len(params.tag2idx))
TypeError: __init__() got an unexpected keyword argument 'num_labels'`


BartTokenizer prepare_seq2seq_batch() does not return decoder_input_ids,  decoder_attention_mask as per document after passing tgt_texts

I am trying to train a seq2seq model using BartModel. As per BartTokenizer documentation if I pass tgt_texts then it should return decoder_attention_mask and decoder_input_ids please check the attachment for clarity.
![image](https://user-images.githubusercontent.com/47693507/96223213-5d10af00-0faf-11eb-9253-5985990f15f9.png)
But I am only getting input_ids, attention_mask and labels.
![image](https://user-images.githubusercontent.com/47693507/96223257-71ed4280-0faf-11eb-9c8f-b26b7af8dd9a.png)


[seq2seq testing] improve readability

improve readability

@sshleifer


[seq2seq distributed] child process stuck on error

Filing an issue for myself to help move along https://github.com/huggingface/transformers/pull/7281
```
pytest -sv examples/seq2seq/test_seq2seq_examples_multi_gpu.py
```
gets stuck if one of the child subprocesses throws an error.

Remove `skip_output_dir_check=True,` to reproduce the problem.

It could be related to realtime async read/write pipes getting into a deadlock - see the XXX warning in that test, need to first try whether the problem goes away if I switch to non-real time pipes.

I will get back to it later.


[seq2seq] get_git_info fails gracefully

It looks like gitpython is broken when one tries to use it from `git checkout hash`, see: https://github.com/gitpython-developers/GitPython/issues/633

While debugging/bisecting we need to be able to step through different commits and for the code to still work.

Currently if I do:
```
git checkout fb94b8f1e16eb
```
and run an examples test, like `examples/seq2seq/test_seq2seq_examples_multi_gpu.py` (soon to be merged) it fails with:
```
ERR:   File "./examples/seq2seq/distillation.py", line 504, in <module>
ERR:     distill_main(args)
ERR:   File "./examples/seq2seq/distillation.py", line 494, in distill_main
ERR:     model = create_module(args)
ERR:   File "./examples/seq2seq/distillation.py", line 411, in create_module
ERR:     model = module_cls(args)
ERR:   File "/mnt/nvme1/code/huggingface/transformers-multigpu/examples/seq2seq/finetune.py", line 58, in __init__
ERR:     save_git_info(self.hparams.output_dir)
ERR:   File "/mnt/nvme1/code/huggingface/transformers-multigpu/examples/seq2seq/utils.py", line 355, in save_git_info
ERR:     repo_infos = get_git_info()
ERR:   File "/mnt/nvme1/code/huggingface/transformers-multigpu/examples/seq2seq/utils.py", line 374, in get_git_info
ERR:     "repo_branch": str(repo.active_branch),
ERR:   File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/git/repo/base.py", line 705, in active_branch
ERR:     return self.head.reference
ERR:   File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/git/refs/symbolic.py", line 272, in _get_reference
ERR:     raise TypeError("%s is a detached symbolic reference as it points to %r" % (self, sha))
ERR: TypeError: HEAD is a detached symbolic reference as it points to 'fb94b8f1e16eb21d166174f52e3e49e669ef0ac4'
```

The odd leading `ERR` string is just a replay of stderr from the sub-process - please ignore this nuance.

This PR provides a workaround.

@sshleifer


[testing] disable FutureWarning in examples tests

same as tests/conftest.py, we can't resolve those warning, so turn the noise off.

same as PR: https://github.com/huggingface/transformers/pull/7079


[DOC] Typo and fix the input of labels to `cross_entropy`

# What does this PR do?
The current version caused some errors. The proposed changes fixed it for me. Hope this is helpful!

- [ x] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).

@sgugger


Token Type IDs returned from the tokenizer for T5 don't work with special tokens

With `transformers-3.3.1`:

```
import transformers
t = transformers.AutoTokenizer.from_pretrained('t5-small')
t.encode_plus(["a"], ["b"], add_special_tokens=True, return_token_type_ids=True)
```

This results in
```
{'input_ids': [9, 1, 115, 1], 'token_type_ids': [0, 1], 'attention_mask': [1, 1, 1, 1]}
```

As you can see, the token type IDs don't align with the other outputs.


Small fixes to HP search

# What does this PR do?

HP search has two small bugs in `Trainer` right now:
- it pops things form the metrics dict, which are then improperly logged
- it doesn't pop out the `total_flos`, so the hp search tries to maximize the flos (which I guess is @TevenLeScao master plan to conquer the universe ;-) )


State of ONNX

Hi, love the work that's going on with ONNX. I wanted to share the current state of ONNX support in case others were wondering about it (sorry for another table).

Pipeline | Supported
--- | ---
feature-extraction | ✓
sentiment-analysis | ✓
ner | ✓
question-answering | ✓
fill-mask | ✓
text-generation | ✓
translation | Broken https://github.com/huggingface/transformers/issues/5948#issuecomment-701699251
summarization |
zero-shot-classification |
conversational |
text2text-generation |

I was able to export models for both summarization and zero-shot-classification, but they both error without a specific token length due to a reshape inside the ONNX model (code in https://github.com/huggingface/transformers/issues/7404#issuecomment-703966076). If you have any ideas for how to prevent this, I'm happy to try and put together a PR.

---

A note for Mac Catalina users: exporting models may error with:

```text
[libprotobuf ERROR google/protobuf/descriptor_database.cc:394] Invalid file descriptor data passed to EncodedDescriptorDatabase::Add().
[libprotobuf FATAL google/protobuf/descriptor.cc:1356] CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size):
libc++abi.dylib: terminating with uncaught exception of type google::protobuf::FatalException: CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size):
```

Use an older version of `protobuf` to avoid this (https://github.com/onnx/onnx/issues/2940#issuecomment-669979419):

```sh
pip3 uninstall protobuf
pip3 install protobuf==3.11.3
```


[testing] fix/hide warnings

in `test_trainer_callback.py`:

- fixes PytestCollectionWarning - pytest doesn't like non-unittest classes starting with `Test`
- hides scatter_gather warnings - as it was suggested they are just so

@sgugger

Fixes: https://github.com/huggingface/transformers/issues/7832


model card for arabic-ner model

README file for the Arabic NER model

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


[cleanup] assign todos, faster bart-cnn test

I went through each TODO(SS), and either did it, or made a self-assigned github issue with it.
I also deleted some commented out code blocks.


[utils/check_copies.py] fix DeprecationWarning

in `tests/test_utils_check_copies.py` I was getting intermittently:
```
utils/check_copies.py:52
  /mnt/nvme1/code/transformers-comet/utils/check_copies.py:52: DeprecationWarning: invalid escape sequence \s
    while line_index < len(lines) and re.search(f"^{indent}(class|def)\s+{name}", lines[line_index]) is None:
```
So this should fix it. Not sure why it wasn't showing up all the time.

@sgugger


[s2s trainer] tests fail on multi-gpu machine

#### Command
```bash
RUN_SLOW=1 USE_CUDA=1 pytest examples/seq2seq/test_finetune_trainer.py
```

#### Traceback
```python
=========================================================== test session starts ===========================================================
platform linux -- Python 3.7.4, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /home/shleifer/transformers_fork, inifile: pytest.ini
plugins: forked-1.1.3, hydra-core-1.0.0, xdist-1.31.0, requests-mock-1.8.0
collected 2 items

examples/seq2seq/test_finetune_trainer.py /home/shleifer/transformers_fork/src/transformers/training_args.py:339: FutureWarning: The `evaluate_during_training` argument is deprecated in favor of `evaluation_strategy` (which has more options)
  FutureWarning,
F/home/shleifer/transformers_fork/src/transformers/training_args.py:339: FutureWarning: The `evaluate_during_training` argument is deprecated in favor of `evaluation_strategy` (which has more options)
  FutureWarning,
F

================================================================ FAILURES =================================================================
__________________________________________________________ test_finetune_trainer __________________________________________________________

    def test_finetune_trainer():
>       output_dir = run_trainer(1, "12", MBART_TINY, 1)

examples/seq2seq/test_finetune_trainer.py:19:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
examples/seq2seq/test_finetune_trainer.py:105: in run_trainer
    main()
examples/seq2seq/finetune_trainer.py:294: in main
    model_path=model_args.model_name_or_path if os.path.isdir(model_args.model_name_or_path) else None
src/transformers/trainer.py:583: in train
    train_dataloader = self.get_train_dataloader()
src/transformers/trainer.py:386: in get_train_dataloader
    train_sampler = self._get_train_sampler()
examples/seq2seq/seq2seq_trainer.py:108: in _get_train_sampler
    self.args.per_device_train_batch_size, distributed=self.args.n_gpu > 1
examples/seq2seq/utils.py:156: in make_sortish_sampler
    return DistributedSortishSampler(self, batch_size, shuffle=shuffle, **kwargs)
examples/seq2seq/utils.py:368: in __init__
    num_replicas = dist.get_world_size()
../miniconda3/envs/nb/lib/python3.7/site-packages/torch/distributed/distributed_c10d.py:582: in get_world_size
    return _get_group_size(group)
../miniconda3/envs/nb/lib/python3.7/site-packages/torch/distributed/distributed_c10d.py:196: in _get_group_size
    _check_default_pg()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def _check_default_pg():
        """
        Helper that checks if the default ProcessGroup has been initialized, with
        assertion

        """
        assert _default_pg is not None, \
>           "Default process group is not initialized"
E       AssertionError: Default process group is not initialized

../miniconda3/envs/nb/lib/python3.7/site-packages/torch/distributed/distributed_c10d.py:187: AssertionError
_______________________________________________________ test_finetune_trainer_slow ________________________________________________________

    @slow
    def test_finetune_trainer_slow():
        # TODO(SS): This will fail on devices with more than 1 GPU.
        # There is a missing call to __init__process_group somewhere
>       output_dir = run_trainer(eval_steps=2, max_len="128", model_name=MARIAN_MODEL, num_train_epochs=3)

examples/seq2seq/test_finetune_trainer.py:30:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
examples/seq2seq/test_finetune_trainer.py:105: in run_trainer
    main()
examples/seq2seq/finetune_trainer.py:294: in main
    model_path=model_args.model_name_or_path if os.path.isdir(model_args.model_name_or_path) else None
src/transformers/trainer.py:583: in train
    train_dataloader = self.get_train_dataloader()
src/transformers/trainer.py:386: in get_train_dataloader
    train_sampler = self._get_train_sampler()
examples/seq2seq/seq2seq_trainer.py:108: in _get_train_sampler
    self.args.per_device_train_batch_size, distributed=self.args.n_gpu > 1
examples/seq2seq/utils.py:156: in make_sortish_sampler
    return DistributedSortishSampler(self, batch_size, shuffle=shuffle, **kwargs)
examples/seq2seq/utils.py:368: in __init__
    num_replicas = dist.get_world_size()
../miniconda3/envs/nb/lib/python3.7/site-packages/torch/distributed/distributed_c10d.py:582: in get_world_size
    return _get_group_size(group)
../miniconda3/envs/nb/lib/python3.7/site-packages/torch/distributed/distributed_c10d.py:196: in _get_group_size
    _check_default_pg()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def _check_default_pg():
        """
        Helper that checks if the default ProcessGroup has been initialized, with
        assertion

        """
        assert _default_pg is not None, \
>           "Default process group is not initialized"
E       AssertionError: Default process group is not initialized

../miniconda3/envs/nb/lib/python3.7/site-packages/torch/distributed/distributed_c10d.py:187: AssertionError
========================================================= short test summary info =========================================================
FAILED examples/seq2seq/test_finetune_trainer.py::test_finetune_trainer - AssertionError: Default process group is not initialized
FAILED examples/seq2seq/test_finetune_trainer.py::test_finetune_trainer_slow - AssertionError: Default process group is not initialized
=========================================================== 2 failed in 11.51s ============================================================
```


[testing] test_trainer_callback.py cannot collect test class 'TestTrainerCallback'

something is wrong here:
```
$ pytest tests/test_trainer_callback.py
tests/test_trainer_callback.py:24
  /mnt/nvme1/code/transformers-comet/tests/test_trainer_callback.py:24: PytestCollectionWarning: cannot collect test class 'TestTrainerCallback' because it has a __init__ constructor (from: tests/test_trainer_callback.py)
    class TestTrainerCallback(TrainerCallback):
```

another one in the same test:
```
  /home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/parallel/_functions.py:64: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
    warnings.warn('Was asked to gather along dimension 0, but all '

tests/test_trainer_callback.py::TrainerCallbackTest::test_event_flow
```

hint: these are in the warnings section of the test.

@sgugger


[RAG] RagSequenceForGeneration should not load "facebook/rag-token-nq" and RagTokenForGeneration also should not load "facebook/rag-sequence-nq"

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.3.1
- Platform: Linux-4.19.112+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.6.0+cu101 (False)
- Tensorflow version (GPU?): 2.3.0 (False)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 blenderbot: @mariamabarham
 Bart: @sshleifer
 Marian: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->
@patrickvonplaten
## Information

Model I am using (Bert, XLNet ...): RAG

The problem arises when using:
* [X] the official example scripts: (give details below)

The tasks I am working on is:
* [X] an official GLUE/SQUaD task: (give the name)

## To reproduce

Following usage of token and sequence models should not be allowed, it may give unintended result in forward pass-
```
# RagSequenceForGeneration with "facebook/rag-token-nq"
model = RagSequenceForGeneration.from_pretrained("facebook/rag-token-nq", retriever=retriever)

# RagTokenForGeneration with "facebook/rag-sequence-nq"
model = RagTokenForGeneration.from_pretrained("facebook/rag-sequence-nq", retriever=retriever)
```
Also please correct example at https://huggingface.co/transformers/master/model_doc/rag.html#ragsequenceforgeneration

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
Above usage should throw exception because both the models are incompatible with the each other.


fix wandb/comet problems

As discussed in https://github.com/huggingface/transformers/issues/7821

* handle the case where `comet_ml` is installed but not configured
* fix error in wandb code:
```
>               combined_dict = {**model.config.to_dict(), **combined_dict}
E               AttributeError: 'NoneType' object has no attribute 'to_dict'
```

Fixes: #7821

@sgugger


[RAG] RagSequenceForGeneration: Running "retriever separately example" giving error

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.3.1
- Platform: Linux-4.19.112+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.6.0+cu101 (False)
- Tensorflow version (GPU?): 2.3.0 (False)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 blenderbot: @mariamabarham
 Bart: @sshleifer
 Marian: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->
@patrickvonplaten @LysandreJik

## Information

Model I am using (Bert, XLNet ...): RAG

The problem arises when using:
* [X] the official example scripts: (give details below)

The tasks I am working on is:
* [X] an official GLUE/SQUaD task: `dummy_dataset`

## To reproduce

Steps to reproduce the behavior:

1. Execute code snippets provided (Partially modified example script from https://huggingface.co/transformers/master/model_doc/rag.html)

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->
Code snippets:
```
!pip install git+https://github.com/huggingface/transformers.git
!pip install datasets
!pip install faiss-cpu

from transformers import RagTokenizer, RagRetriever, RagTokenForGeneration, RagSequenceForGeneration
import torch
import faiss

tokenizer = RagTokenizer.from_pretrained("facebook/rag-sequence-nq")
model = RagSequenceForGeneration.from_pretrained("facebook/rag-sequence-nq", use_dummy_dataset=True)
retriever = RagRetriever.from_pretrained("facebook/rag-sequence-nq", index_name="exact", use_dummy_dataset=True)

input_dict = tokenizer.prepare_seq2seq_batch("How many people live in Paris?", "In Paris, there are 10 million people.", return_tensors="pt")
input_ids = input_dict["input_ids"]

# Caling retriever seperately

question_hidden_states = model.question_encoder(input_ids)[0]
# 2. Retrieve
docs_dict = retriever(input_ids.numpy(), question_hidden_states.detach().numpy(), return_tensors="pt")
print(docs_dict)
doc_scores = torch.bmm(question_hidden_states.unsqueeze(1), docs_dict["retrieved_doc_embeds"].float().transpose(1, 2)).squeeze(1)
# 3. Forward to generator
outputs = model.generate(input_ids=input_ids, context_input_ids=docs_dict["context_input_ids"], context_attention_mask=docs_dict["context_attention_mask"], doc_scores=doc_scores)

generated_string = tokenizer.batch_decode(outputs, skip_special_tokens=True)
print(generated_string)

```
Stacktrace:
```
AssertionError                            Traceback (most recent call last)
<ipython-input-5-9f622b1f6353> in <module>()
      7 doc_scores = torch.bmm(question_hidden_states.unsqueeze(1), docs_dict["retrieved_doc_embeds"].float().transpose(1, 2)).squeeze(1)
      8 # 3. Forward to generator
----> 9 outputs = model.generate(input_ids=input_ids, context_input_ids=docs_dict["context_input_ids"], context_attention_mask=docs_dict["context_attention_mask"], doc_scores=doc_scores)
     10 generated_string = tokenizer.batch_decode(outputs, skip_special_tokens=True)
     11 print(generated_string)

5 frames
/usr/local/lib/python3.6/dist-packages/torch/autograd/grad_mode.py in decorate_context(*args, **kwargs)
     13         def decorate_context(*args, **kwargs):
     14             with self:
---> 15                 return func(*args, **kwargs)
     16         return decorate_context
     17

/usr/local/lib/python3.6/dist-packages/transformers/modeling_rag.py in generate(self, input_ids, attention_mask, context_input_ids, do_deduplication, num_return_sequences, num_beams, **kwargs)
    902             # then, run model forwards to get nll scores:
    903             new_input_ids = input_ids[index : index + 1].repeat(len(output_sequences), 1)
--> 904             outputs = self(new_input_ids, labels=output_sequences, exclude_bos_score=True)
    905             top_cand_inds = (-outputs["loss"]).topk(num_doc_return_sequences)[1]
    906

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    720             result = self._slow_forward(*input, **kwargs)
    721         else:
--> 722             result = self.forward(*input, **kwargs)
    723         for hook in itertools.chain(
    724                 _global_forward_hooks.values(),

/usr/local/lib/python3.6/dist-packages/transformers/modeling_rag.py in forward(self, input_ids, attention_mask, encoder_outputs, decoder_input_ids, decoder_attention_mask, past_key_values, context_input_ids, context_attention_mask, doc_scores, use_cache, output_attentions, output_hidden_states, output_retrieved, exclude_bos_score, reduce_loss, labels, **kwargs)
    767             output_attentions=output_attentions,
    768             output_hidden_states=output_hidden_states,
--> 769             output_retrieved=output_retrieved,
    770         )
    771

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    720             result = self._slow_forward(*input, **kwargs)
    721         else:
--> 722             result = self.forward(*input, **kwargs)
    723         for hook in itertools.chain(
    724                 _global_forward_hooks.values(),

/usr/local/lib/python3.6/dist-packages/transformers/modeling_rag.py in forward(self, input_ids, attention_mask, encoder_outputs, decoder_input_ids, decoder_attention_mask, past_key_values, doc_scores, context_input_ids, context_attention_mask, use_cache, output_attentions, output_hidden_states, output_retrieved)
    589                 assert (
    590                     context_input_ids is not None
--> 591                 ), "Make sure that `context_input_ids` are passed, if no `retriever` is set. Alternatively, you can set a retriever using the `set_retriever(...)` function."
    592                 assert (
    593                     context_attention_mask is not None

AssertionError: Make sure that `context_input_ids` are passed, if no `retriever` is set. Alternatively, you can set a retriever using the `set_retriever(...)` function.
```
I suspect `context_input_ids` is not passed to `forward` method. And if model is not initialised with retriever then `forward` function complain about missing `context_input_ids` or `retriever`. Referring to following piece of code in `RagSequenceForGeneration` class and `generator` function.
```
            # then, run model forwards to get nll scores:
            new_input_ids = input_ids[index : index + 1].repeat(len(output_sequences), 1)
            outputs = self(new_input_ids, labels=output_sequences, exclude_bos_score=True)
            top_cand_inds = (-outputs["loss"]).topk(num_doc_return_sequences)[1]
```

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
It should work as intended as `RagTokenForGeneration` do.


fix: ignore padding tokens in Bart loss

# What does this PR do?
There is a discrepancy between the [fine-tuning script](https://github.com/huggingface/transformers/blob/e7aa64838cc604abf7a49e69ca0ffe7af683d8ca/examples/seq2seq/finetune.py#L153) and the [BartForConditionalGeneration](https://github.com/huggingface/transformers/blob/e7aa64838cc604abf7a49e69ca0ffe7af683d8ca/src/transformers/modeling_bart.py#L1113) which is also noted in the comments.

From `examples/seq2seq/finetune.py`:
```python
# Same behavior as modeling_bart.py, besides ignoring pad_token_id
ce_loss_fct = torch.nn.CrossEntropyLoss(ignore_index=pad_token_id)
```

From `transformers/src/modeling_bart.py`:
```python
loss_fct = CrossEntropyLoss()
# TODO(SS): do we need to ignore pad tokens in labels?
```

Training with the `Trainer` and `BartForConditionalGeneration` results in a model that produces garbled text (lots of repetitions and no coherence). Adding the `ignore_index=self.config.pad_token_id` in the `CrossEntropyLoss` resolves the issue.

Besides a before and after run I did not study the behaviour in a systematic way since training the model requires a significant amount of time and compute. If you would like to see more testing let me know what you think is the best way to test this thoroughly.


Do I need to apply the softmax function to my logit before calculating the CrossEntropyLoss?

Hello, I am trying to compute the CrossEntropyLoss directly by using this code:
```
loss_fct = CrossEntropyLoss()
mc_loss = loss_fct(reshaped_logits, mc_labels)
```
If the reshaped_logits contain the logit values before softmax, should I apply `nn.softmax` function before I do `loss_fct(reshaped_logits, mc_labels)`? Thank you,


[Pipelines] Fix links to model lists


RFC: Move `_NoLayerEmbedTokens` to modeling_tf_utils.py

It has 0 computation and will be needed in many places. Here is the code:

```python
class _NoLayerEmbedTokens:
    """
    this class wraps a the TFSharedEmbeddingTokens layer into a python 'no-keras-layer'
    class to avoid problem with weight restoring. Also it makes sure that the layer is
    called from the correct scope to avoid problem with saving/storing the correct weights
    """

    def __init__(self, layer, abs_scope_name=None):
        self._layer = layer
        self._abs_scope_name = abs_scope_name

    def call(self, inputs, mode="embedding"):
        if self._abs_scope_name is None:
            return self._layer.call(inputs, mode)

        # if an abs scope name is given to the embedding variable, call variable from absolute scope
        with tf.compat.v1.variable_scope(self._abs_scope_name, auxiliary_name_scope=False) as abs_scope_name:
            with tf.name_scope(abs_scope_name.original_name_scope):
                return self._layer.call(inputs, mode)

    def __call__(self, inputs, mode="embedding"):
        if self._abs_scope_name is None:
            return self._layer(inputs, mode)

        # if an abs scope name is given to the embedding variable, call variable from absolute scope
        with tf.compat.v1.variable_scope(self._abs_scope_name, auxiliary_name_scope=False) as abs_scope_name:
            with tf.name_scope(abs_scope_name.original_name_scope):
                return self._layer(inputs, mode)
```

I will copy it into tfbart to avoid getting bogged down in debate, but wdyt @patrickvonplaten ?


Bart Caching: do we need encoder outputs after step 1?

since projected cross attention k,v are cached, this doesn't seem like it's actually needed.
Maybe we could save some mem if we deleted them after we were done.


Support for custom data_collator in Trainer.train() with datasets.Dataset

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->
Currently (transformers==3.3.1) Trainer removes unknown columns (not present in forward method of a model) from datasets.Dataset object. It prevents using custom DataCollator in `.train` method since it doesn't have columns that one would want to use.

I would be interested in an option to not remove unknown columns and allow user to handle them in DataCollator (or provide their own DataLoaders to the train method).

## Motivation
The example code I was trying to use for training looks like this:
```
from typing import List

from datasets import load_dataset
from transformers import RobertaForSequenceClassification, RobertaTokenizer, Trainer

tokenizer = RobertaTokenizer.from_pretrained('roberta-base')
model = RobertaForSequenceClassification.from_pretrained('roberta-base', return_dict=True)

train_dataset = load_dataset("yelp_polarity", split="train")
val_dataset = load_dataset("yelp_polarity", split="test")

class DataCollator:
    def __init__(self, tokenizer):
        self.tokenizer = tokenizer

    def __call__(self, examples: List[dict]):
        labels = [example['label'] for example in examples]
        texts = [example['text'] for example in examples]
        tokenizer_output = self.tokenizer(texts, truncation=True, padding=True)
        tokenizer_output['input_ids'] = torch.tensor(tokenizer_output['input_ids'])
        tokenizer_output['attention_mask'] = torch.tensor(tokenizer_output['attention_mask'])
        output_dict = dict(labels=labels, **tokenizer_output)
        return output_dict

data_collator = DataCollator(tokenizer)

trainer = Trainer(
    model=model,                   # the instantiated 🤗 Transformers model to be trained
    train_dataset=train_dataset,   # training dataset
    eval_dataset=val_dataset,      # evaluation dataset
    data_collator=data_collator    # problem with removing unwanted columns by transformers.Trainer code. Doesn't really work, is it a bug?
)

trainer.train()
```
(with `trainer.train()` not working)

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->

## Your contribution
Before trying to fix things, I would be interested to learn if there is a suggested alternative way of doing that?

<!-- Is there any way that you could help, e.g. by submitting a PR?
     Make sure to read the CONTRIBUTING.MD readme:
     https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md -->


[testing] test_modeling_deberta.py is failing

```
USE_CUDA=1 pytest tests/test_modeling_deberta.py
```

```
====================================================================== test session starts =======================================================================
platform linux -- Python 3.8.5, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
rootdir: /mnt/nvme1/code/huggingface/transformers-master
plugins: xdist-2.1.0, forked-1.3.0, typeguard-2.9.1, flake8-1.0.6, hydra-core-1.0.0, cov-2.10.1, instafail-0.4.2, flakefinder-1.0.0
collected 33 items

tests/test_modeling_deberta.py ...F.FssFs............s..F....sss                                                                                [100%]

============================================================================ FAILURES ============================================================================
______________________________________________________________ DebertaModelTest.test_deberta_model _______________________________________________________________

self = <tests.test_modeling_deberta.DebertaModelTest testMethod=test_deberta_model>

    def test_deberta_model(self):
        config_and_inputs = self.model_tester.prepare_config_and_inputs()
>       self.model_tester.create_and_check_deberta_model(*config_and_inputs)

tests/test_modeling_deberta.py:210:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/test_modeling_deberta.py:159: in create_and_check_deberta_model
    sequence_output = model(input_ids, attention_mask=input_mask, token_type_ids=token_type_ids)[0]
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/modules/module.py:727: in _call_impl
    result = self.forward(*input, **kwargs)
src/transformers/modeling_deberta.py:891: in forward
    encoder_outputs = self.encoder(
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/modules/module.py:727: in _call_impl
    result = self.forward(*input, **kwargs)
src/transformers/modeling_deberta.py:401: in forward
    hidden_states = layer_module(
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/modules/module.py:727: in _call_impl
    result = self.forward(*input, **kwargs)
src/transformers/modeling_deberta.py:324: in forward
    attention_output = self.attention(
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/modules/module.py:727: in _call_impl
    result = self.forward(*input, **kwargs)
src/transformers/modeling_deberta.py:257: in forward
    self_output = self.self(
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/modules/module.py:727: in _call_impl
    result = self.forward(*input, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = DisentangledSelfAttention(
  (in_proj): Linear(in_features=32, out_features=96, bias=False)
  (dropout): StableDropout()
)
hidden_states = tensor([[[-0.1865,  1.6041, -1.3552,  ..., -1.2947, -0.9667,  0.9630],
         [ 1.1352,  0.1829,  0.3669,  ..., -2.4...        [-0.3743, -0.0251, -0.8356,  ..., -0.9786, -1.3914, -1.9630]]],
       device='cuda:0', grad_fn=<MulBackward0>)
attention_mask = tensor([[[[1, 1, 1, 0, 1, 1, 0],
          [1, 1, 1, 0, 1, 1, 0],
          [1, 1, 1, 0, 1, 1, 0],
          [0, 0, 0,..., 0, 0, 0, 0],
          [0, 0, 0, 0, 0, 0, 0],
          [0, 1, 1, 1, 0, 0, 1]]]], device='cuda:0', dtype=torch.uint8)
return_att = False, query_states = None, relative_pos = None, rel_embeddings = None

    def forward(
        self,
        hidden_states,
        attention_mask,
        return_att=False,
        query_states=None,
        relative_pos=None,
        rel_embeddings=None,
    ):
        """Call the module

        Args:
            hidden_states (:obj:`torch.FloatTensor`):
                Input states to the module usally the output from previous layer, it will be the Q,K and V in `Attention(Q,K,V)`

            attention_mask (:obj:`torch.ByteTensor`):
                An attention mask matrix of shape [`B`, `N`, `N`] where `B` is the batch size, `N` is the maxium sequence length in which element [i,j] = `1` means the `i` th token in the input can attend to the `j` th token.

            return_att (:obj:`bool`, optional):
                Whether return the attention maxitrix.

            query_states (:obj:`torch.FloatTensor`, optional):
                The `Q` state in `Attention(Q,K,V)`.

            relative_pos (:obj:`torch.LongTensor`):
                The relative position encoding between the tokens in the sequence. It's of shape [`B`, `N`, `N`] with values ranging in [`-max_relative_positions`, `max_relative_positions`].

            rel_embeddings (:obj:`torch.FloatTensor`):
                The embedding of relative distances. It's a tensor of shape [:math:`2 \\times \\text{max_relative_positions}`, `hidden_size`].

        """
        if query_states is None:
            qp = self.in_proj(hidden_states)  # .split(self.all_head_size, dim=-1)
            query_layer, key_layer, value_layer = self.transpose_for_scores(qp).chunk(3, dim=-1)
        else:

            def linear(w, b, x):
                if b is not None:
                    return torch.matmul(x, w.t()) + b.t()
                else:
                    return torch.matmul(x, w.t())  # + b.t()

            ws = self.in_proj.weight.chunk(self.num_attention_heads * 3, dim=0)
            qkvw = [torch.cat([ws[i * 3 + k] for i in range(self.num_attention_heads)], dim=0) for k in range(3)]
            qkvb = [None] * 3

            q = linear(qkvw[0], qkvb[0], query_states)
            k, v = [linear(qkvw[i], qkvb[i], hidden_states) for i in range(1, 3)]
            query_layer, key_layer, value_layer = [self.transpose_for_scores(x) for x in [q, k, v]]

        query_layer += self.transpose_for_scores(self.q_bias[None, None, :])
>       value_layer += self.transpose_for_scores(self.v_bias[None, None, :])
E       RuntimeError: diff_view_meta->output_nr_ == 0 INTERNAL ASSERT FAILED at "/opt/conda/conda-bld/pytorch_1602659340682/work/torch/csrc/autograd/variable.cpp":352, please report a bug to PyTorch.

src/transformers/modeling_deberta.py:575: RuntimeError
__________________________________________________________ DebertaModelTest.test_feed_forward_chunking ___________________________________________________________

self = <tests.test_modeling_deberta.DebertaModelTest testMethod=test_feed_forward_chunking>

    def test_feed_forward_chunking(self):
        (
            original_config,
            inputs_dict,
        ) = self.model_tester.prepare_config_and_inputs_for_common()
        for model_class in self.all_model_classes:
            torch.manual_seed(0)
            config = copy.deepcopy(original_config)
            model = model_class(config)
            model.to(torch_device)
            model.eval()

>           hidden_states_no_chunk = model(**self._prepare_for_class(inputs_dict, model_class))[0]

tests/test_modeling_common.py:617:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/modules/module.py:727: in _call_impl
    result = self.forward(*input, **kwargs)
src/transformers/modeling_deberta.py:891: in forward
    encoder_outputs = self.encoder(
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/modules/module.py:727: in _call_impl
    result = self.forward(*input, **kwargs)
src/transformers/modeling_deberta.py:401: in forward
    hidden_states = layer_module(
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/modules/module.py:727: in _call_impl
    result = self.forward(*input, **kwargs)
src/transformers/modeling_deberta.py:324: in forward
    attention_output = self.attention(
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/modules/module.py:727: in _call_impl
    result = self.forward(*input, **kwargs)
src/transformers/modeling_deberta.py:257: in forward
    self_output = self.self(
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/modules/module.py:727: in _call_impl
    result = self.forward(*input, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = DisentangledSelfAttention(
  (in_proj): Linear(in_features=32, out_features=96, bias=False)
  (dropout): StableDropout()
)
hidden_states = tensor([[[-0.1510,  1.7101, -1.2962,  ...,  1.5861, -0.6766,  0.3315],
         [ 0.0000, -0.0000, -0.0000,  ...,  0.0...        [-0.0000, -0.0000, -0.0000,  ...,  0.0000,  0.0000,  0.0000]]],
       device='cuda:0', grad_fn=<MulBackward0>)
attention_mask = tensor([[[[1, 0, 0, 1, 1, 1, 0],
          [0, 0, 0, 0, 0, 0, 0],
          [0, 0, 0, 0, 0, 0, 0],
          [1, 0, 0,..., 0, 1, 1, 0],
          [1, 0, 0, 0, 1, 1, 0],
          [0, 0, 0, 0, 0, 0, 0]]]], device='cuda:0', dtype=torch.uint8)
return_att = False, query_states = None, relative_pos = None, rel_embeddings = None

    def forward(
        self,
        hidden_states,
        attention_mask,
        return_att=False,
        query_states=None,
        relative_pos=None,
        rel_embeddings=None,
    ):
        """Call the module

        Args:
            hidden_states (:obj:`torch.FloatTensor`):
                Input states to the module usally the output from previous layer, it will be the Q,K and V in `Attention(Q,K,V)`

            attention_mask (:obj:`torch.ByteTensor`):
                An attention mask matrix of shape [`B`, `N`, `N`] where `B` is the batch size, `N` is the maxium sequence length in which element [i,j] = `1` means the `i` th token in the input can attend to the `j` th token.

            return_att (:obj:`bool`, optional):
                Whether return the attention maxitrix.

            query_states (:obj:`torch.FloatTensor`, optional):
                The `Q` state in `Attention(Q,K,V)`.

            relative_pos (:obj:`torch.LongTensor`):
                The relative position encoding between the tokens in the sequence. It's of shape [`B`, `N`, `N`] with values ranging in [`-max_relative_positions`, `max_relative_positions`].

            rel_embeddings (:obj:`torch.FloatTensor`):
                The embedding of relative distances. It's a tensor of shape [:math:`2 \\times \\text{max_relative_positions}`, `hidden_size`].

        """
        if query_states is None:
            qp = self.in_proj(hidden_states)  # .split(self.all_head_size, dim=-1)
            query_layer, key_layer, value_layer = self.transpose_for_scores(qp).chunk(3, dim=-1)
        else:

            def linear(w, b, x):
                if b is not None:
                    return torch.matmul(x, w.t()) + b.t()
                else:
                    return torch.matmul(x, w.t())  # + b.t()

            ws = self.in_proj.weight.chunk(self.num_attention_heads * 3, dim=0)
            qkvw = [torch.cat([ws[i * 3 + k] for i in range(self.num_attention_heads)], dim=0) for k in range(3)]
            qkvb = [None] * 3

            q = linear(qkvw[0], qkvb[0], query_states)
            k, v = [linear(qkvw[i], qkvb[i], hidden_states) for i in range(1, 3)]
            query_layer, key_layer, value_layer = [self.transpose_for_scores(x) for x in [q, k, v]]

        query_layer += self.transpose_for_scores(self.q_bias[None, None, :])
>       value_layer += self.transpose_for_scores(self.v_bias[None, None, :])
E       RuntimeError: diff_view_meta->output_nr_ == 0 INTERNAL ASSERT FAILED at "/opt/conda/conda-bld/pytorch_1602659340682/work/torch/csrc/autograd/variable.cpp":352, please report a bug to PyTorch.

src/transformers/modeling_deberta.py:575: RuntimeError
_______________________________________________________ DebertaModelTest.test_for_sequence_classification ________________________________________________________

self = <tests.test_modeling_deberta.DebertaModelTest testMethod=test_for_sequence_classification>

    def test_for_sequence_classification(self):
        config_and_inputs = self.model_tester.prepare_config_and_inputs()
>       self.model_tester.create_and_check_deberta_for_sequence_classification(*config_and_inputs)

tests/test_modeling_deberta.py:214:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/test_modeling_deberta.py:177: in create_and_check_deberta_for_sequence_classification
    loss, logits = model(
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/modules/module.py:727: in _call_impl
    result = self.forward(*input, **kwargs)
src/transformers/modeling_deberta.py:987: in forward
    outputs = self.deberta(
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/modules/module.py:727: in _call_impl
    result = self.forward(*input, **kwargs)
src/transformers/modeling_deberta.py:891: in forward
    encoder_outputs = self.encoder(
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/modules/module.py:727: in _call_impl
    result = self.forward(*input, **kwargs)
src/transformers/modeling_deberta.py:401: in forward
    hidden_states = layer_module(
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/modules/module.py:727: in _call_impl
    result = self.forward(*input, **kwargs)
src/transformers/modeling_deberta.py:324: in forward
    attention_output = self.attention(
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/modules/module.py:727: in _call_impl
    result = self.forward(*input, **kwargs)
src/transformers/modeling_deberta.py:257: in forward
    self_output = self.self(
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/modules/module.py:727: in _call_impl
    result = self.forward(*input, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = DisentangledSelfAttention(
  (in_proj): Linear(in_features=32, out_features=96, bias=False)
  (dropout): StableDropout()
)
hidden_states = tensor([[[-6.2643e-01, -1.6159e+00,  1.4385e+00,  ...,  1.2083e+00,
          -7.2520e-02,  4.6405e-01],
         [ 7....62e-01,  6.8716e-01,  ..., -1.0658e+00,
          -9.1048e-01,  4.0098e-01]]], device='cuda:0', grad_fn=<MulBackward0>)
attention_mask = tensor([[[[1, 1, 1, 1, 1, 0, 0],
          [1, 1, 1, 1, 1, 0, 0],
          [1, 1, 1, 1, 1, 0, 0],
          [1, 1, 1,..., 0, 0, 0, 0],
          [1, 1, 1, 1, 0, 1, 1],
          [1, 1, 1, 1, 0, 1, 1]]]], device='cuda:0', dtype=torch.uint8)
return_att = False, query_states = None, relative_pos = None, rel_embeddings = None

    def forward(
        self,
        hidden_states,
        attention_mask,
        return_att=False,
        query_states=None,
        relative_pos=None,
        rel_embeddings=None,
    ):
        """Call the module

        Args:
            hidden_states (:obj:`torch.FloatTensor`):
                Input states to the module usally the output from previous layer, it will be the Q,K and V in `Attention(Q,K,V)`

            attention_mask (:obj:`torch.ByteTensor`):
                An attention mask matrix of shape [`B`, `N`, `N`] where `B` is the batch size, `N` is the maxium sequence length in which element [i,j] = `1` means the `i` th token in the input can attend to the `j` th token.

            return_att (:obj:`bool`, optional):
                Whether return the attention maxitrix.

            query_states (:obj:`torch.FloatTensor`, optional):
                The `Q` state in `Attention(Q,K,V)`.

            relative_pos (:obj:`torch.LongTensor`):
                The relative position encoding between the tokens in the sequence. It's of shape [`B`, `N`, `N`] with values ranging in [`-max_relative_positions`, `max_relative_positions`].

            rel_embeddings (:obj:`torch.FloatTensor`):
                The embedding of relative distances. It's a tensor of shape [:math:`2 \\times \\text{max_relative_positions}`, `hidden_size`].

        """
        if query_states is None:
            qp = self.in_proj(hidden_states)  # .split(self.all_head_size, dim=-1)
            query_layer, key_layer, value_layer = self.transpose_for_scores(qp).chunk(3, dim=-1)
        else:

            def linear(w, b, x):
                if b is not None:
                    return torch.matmul(x, w.t()) + b.t()
                else:
                    return torch.matmul(x, w.t())  # + b.t()

            ws = self.in_proj.weight.chunk(self.num_attention_heads * 3, dim=0)
            qkvw = [torch.cat([ws[i * 3 + k] for i in range(self.num_attention_heads)], dim=0) for k in range(3)]
            qkvb = [None] * 3

            q = linear(qkvw[0], qkvb[0], query_states)
            k, v = [linear(qkvw[i], qkvb[i], hidden_states) for i in range(1, 3)]
            query_layer, key_layer, value_layer = [self.transpose_for_scores(x) for x in [q, k, v]]

        query_layer += self.transpose_for_scores(self.q_bias[None, None, :])
>       value_layer += self.transpose_for_scores(self.v_bias[None, None, :])
E       RuntimeError: diff_view_meta->output_nr_ == 0 INTERNAL ASSERT FAILED at "/opt/conda/conda-bld/pytorch_1602659340682/work/torch/csrc/autograd/variable.cpp":352, please report a bug to PyTorch.

src/transformers/modeling_deberta.py:575: RuntimeError
_________________________________________________________ DebertaModelTest.test_resize_tokens_embeddings _________________________________________________________

self = <tests.test_modeling_deberta.DebertaModelTest testMethod=test_resize_tokens_embeddings>

    def test_resize_tokens_embeddings(self):
        (
            original_config,
            inputs_dict,
        ) = self.model_tester.prepare_config_and_inputs_for_common()
        if not self.test_resize_embeddings:
            return

        for model_class in self.all_model_classes:
            config = copy.deepcopy(original_config)
            model = model_class(config)
            model.to(torch_device)

            if self.model_tester.is_training is False:
                model.eval()

            model_vocab_size = config.vocab_size
            # Retrieve the embeddings and clone theme
            model_embed = model.resize_token_embeddings(model_vocab_size)
            cloned_embeddings = model_embed.weight.clone()

            # Check that resizing the token embeddings with a larger vocab size increases the model's vocab size
            model_embed = model.resize_token_embeddings(model_vocab_size + 10)
            self.assertEqual(model.config.vocab_size, model_vocab_size + 10)
            # Check that it actually resizes the embeddings matrix
            self.assertEqual(model_embed.weight.shape[0], cloned_embeddings.shape[0] + 10)
            # Check that the model can still do a forward pass successfully (every parameter should be resized)
>           model(**self._prepare_for_class(inputs_dict, model_class))

tests/test_modeling_common.py:655:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/modules/module.py:727: in _call_impl
    result = self.forward(*input, **kwargs)
src/transformers/modeling_deberta.py:891: in forward
    encoder_outputs = self.encoder(
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/modules/module.py:727: in _call_impl
    result = self.forward(*input, **kwargs)
src/transformers/modeling_deberta.py:401: in forward
    hidden_states = layer_module(
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/modules/module.py:727: in _call_impl
    result = self.forward(*input, **kwargs)
src/transformers/modeling_deberta.py:324: in forward
    attention_output = self.attention(
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/modules/module.py:727: in _call_impl
    result = self.forward(*input, **kwargs)
src/transformers/modeling_deberta.py:257: in forward
    self_output = self.self(
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/modules/module.py:727: in _call_impl
    result = self.forward(*input, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = DisentangledSelfAttention(
  (in_proj): Linear(in_features=32, out_features=96, bias=False)
  (dropout): StableDropout()
)
hidden_states = tensor([[[ 0.0000, -1.3935,  1.0241,  ...,  0.4936,  0.0103,  0.0000],
         [-0.9223,  0.6012, -0.7819,  ..., -0.8...    [ 0.0000, -0.4612,  1.3087,  ...,  1.2537,  0.3499, -0.9116]]],
       device='cuda:0', grad_fn=<XDropoutBackward>)
attention_mask = tensor([[[[1, 1, 0, 0, 1, 1, 0],
          [1, 1, 0, 0, 1, 1, 0],
          [0, 0, 0, 0, 0, 0, 0],
          [0, 0, 0,..., 0, 1, 0, 1],
          [0, 0, 0, 0, 0, 0, 0],
          [1, 0, 1, 0, 1, 0, 1]]]], device='cuda:0', dtype=torch.uint8)
return_att = False, query_states = None, relative_pos = None, rel_embeddings = None

    def forward(
        self,
        hidden_states,
        attention_mask,
        return_att=False,
        query_states=None,
        relative_pos=None,
        rel_embeddings=None,
    ):
        """Call the module

        Args:
            hidden_states (:obj:`torch.FloatTensor`):
                Input states to the module usally the output from previous layer, it will be the Q,K and V in `Attention(Q,K,V)`

            attention_mask (:obj:`torch.ByteTensor`):
                An attention mask matrix of shape [`B`, `N`, `N`] where `B` is the batch size, `N` is the maxium sequence length in which element [i,j] = `1` means the `i` th token in the input can attend to the `j` th token.

            return_att (:obj:`bool`, optional):
                Whether return the attention maxitrix.

            query_states (:obj:`torch.FloatTensor`, optional):
                The `Q` state in `Attention(Q,K,V)`.

            relative_pos (:obj:`torch.LongTensor`):
                The relative position encoding between the tokens in the sequence. It's of shape [`B`, `N`, `N`] with values ranging in [`-max_relative_positions`, `max_relative_positions`].

            rel_embeddings (:obj:`torch.FloatTensor`):
                The embedding of relative distances. It's a tensor of shape [:math:`2 \\times \\text{max_relative_positions}`, `hidden_size`].

        """
        if query_states is None:
            qp = self.in_proj(hidden_states)  # .split(self.all_head_size, dim=-1)
            query_layer, key_layer, value_layer = self.transpose_for_scores(qp).chunk(3, dim=-1)
        else:

            def linear(w, b, x):
                if b is not None:
                    return torch.matmul(x, w.t()) + b.t()
                else:
                    return torch.matmul(x, w.t())  # + b.t()

            ws = self.in_proj.weight.chunk(self.num_attention_heads * 3, dim=0)
            qkvw = [torch.cat([ws[i * 3 + k] for i in range(self.num_attention_heads)], dim=0) for k in range(3)]
            qkvb = [None] * 3

            q = linear(qkvw[0], qkvb[0], query_states)
            k, v = [linear(qkvw[i], qkvb[i], hidden_states) for i in range(1, 3)]
            query_layer, key_layer, value_layer = [self.transpose_for_scores(x) for x in [q, k, v]]

        query_layer += self.transpose_for_scores(self.q_bias[None, None, :])
>       value_layer += self.transpose_for_scores(self.v_bias[None, None, :])
E       RuntimeError: diff_view_meta->output_nr_ == 0 INTERNAL ASSERT FAILED at "/opt/conda/conda-bld/pytorch_1602659340682/work/torch/csrc/autograd/variable.cpp":352, please report a bug to PyTorch.

src/transformers/modeling_deberta.py:575: RuntimeError
======================================================================== warnings summary ========================================================================
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/comet_ml/utils.py:30
  /home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/comet_ml/utils.py:30: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working
    from collections import Mapping, defaultdict

/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/comet_ml/monkey_patching.py:19
  /home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/comet_ml/monkey_patching.py:19: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
    import imp

/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/graphql/type/directives.py:55
  /home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/graphql/type/directives.py:55: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working
    assert isinstance(locations, collections.Iterable), 'Must provide locations for directive.'

/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/graphql/type/typemap.py:1
  /home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/graphql/type/typemap.py:1: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working
    from collections import OrderedDict, Sequence, defaultdict

tests/test_modeling_deberta.py::DebertaModelTest::test_deberta_model
tests/test_modeling_deberta.py::DebertaModelTest::test_feed_forward_chunking
tests/test_modeling_deberta.py::DebertaModelTest::test_for_sequence_classification
tests/test_modeling_deberta.py::DebertaModelTest::test_resize_tokens_embeddings
  /mnt/nvme1/code/huggingface/transformers-master/src/transformers/modeling_deberta.py:574: UserWarning: Output 0 of SplitBackward is a view and is being modified inplace. This view is an output of a function that returns multiple views. Inplace operators on such views are being deprecated and will be forbidden starting from version 1.8. Consider using `unsafe_` version of the function that produced this view or don't modify this view inplace. (Triggered internally at  /opt/conda/conda-bld/pytorch_1602659340682/work/torch/csrc/autograd/variable.cpp:480.)
    query_layer += self.transpose_for_scores(self.q_bias[None, None, :])

tests/test_modeling_deberta.py::DebertaModelTest::test_deberta_model
tests/test_modeling_deberta.py::DebertaModelTest::test_feed_forward_chunking
tests/test_modeling_deberta.py::DebertaModelTest::test_for_sequence_classification
tests/test_modeling_deberta.py::DebertaModelTest::test_resize_tokens_embeddings
  /mnt/nvme1/code/huggingface/transformers-master/src/transformers/modeling_deberta.py:575: UserWarning: Output 2 of SplitBackward is a view and is being modified inplace. This view is an output of a function that returns multiple views. Inplace operators on such views are being deprecated and will be forbidden starting from version 1.8. Consider using `unsafe_` version of the function that produced this view or don't modify this view inplace. (Triggered internally at  /opt/conda/conda-bld/pytorch_1602659340682/work/torch/csrc/autograd/variable.cpp:480.)
    value_layer += self.transpose_for_scores(self.v_bias[None, None, :])

tests/test_modeling_deberta.py::DebertaModelTest::test_model_outputs_equivalence
  /mnt/nvme1/code/huggingface/transformers-master/src/transformers/modeling_deberta.py:1011: UserWarning: This overload of nonzero is deprecated:
        nonzero()
  Consider using one of the following signatures instead:
        nonzero(*, bool as_tuple) (Triggered internally at  /opt/conda/conda-bld/pytorch_1602659340682/work/torch/csrc/utils/python_arg_parser.cpp:945.)
    label_index = (labels >= 0).nonzero()

-- Docs: https://docs.pytest.org/en/stable/warnings.html
==================================================================== short test summary info =====================================================================
FAILED tests/test_modeling_deberta.py::DebertaModelTest::test_deberta_model - RuntimeError: diff_view_meta->output_nr_ == 0 INTERNAL ASSERT FAILED at "/opt/con...
FAILED tests/test_modeling_deberta.py::DebertaModelTest::test_feed_forward_chunking - RuntimeError: diff_view_meta->output_nr_ == 0 INTERNAL ASSERT FAILED at "...
FAILED tests/test_modeling_deberta.py::DebertaModelTest::test_for_sequence_classification - RuntimeError: diff_view_meta->output_nr_ == 0 INTERNAL ASSERT FAILE...
FAILED tests/test_modeling_deberta.py::DebertaModelTest::test_resize_tokens_embeddings - RuntimeError: diff_view_meta->output_nr_ == 0 INTERNAL ASSERT FAILED a...
====================================================== 4 failed, 22 passed, 7 skipped, 13 warnings in 7.85s ====================================================
```

Thank you!

## Environment info

```
- `transformers` version: master
- Platform: Linux-4.15.0-118-generic-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyTorch version (GPU?): 1.8.0.dev20201014 (True)
- Tensorflow version (GPU?): 2.3.1 (True)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
```

@sgugger, @LysandreJik


[testing] trainer tests fail - 2 issues

trainer tests seem to have several issues:

```
USE_CUDA=1 pytest tests/test_trainer.py::TrainerIntegrationTest::test_custom_optimizer
```

1.

```
        if self.api_key is None:
>           raise ValueError(
                "Comet.ml requires an API key. Please provide as the "
                "first argument to Experiment(api_key) or as an environment"
                " variable named COMET_API_KEY "
            )
```

the test should skip and not fail if the user doesn't have Comet.ml

2. wandb seems to still be problematic when there is an error in the test.
```
Traceback (most recent call last):
  File "/home/stas/anaconda3/envs/main-38/bin/pytest", line 8, in <module>
    sys.exit(console_main())
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/_pytest/config/__init__.py", line 180, in console_main
    code = main()
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/_pytest/config/__init__.py", line 157, in main
    ret = config.hook.pytest_cmdline_main(
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/_pytest/main.py", line 289, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/_pytest/main.py", line 284, in wrap_session
    config._ensure_unconfigure()
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/_pytest/config/__init__.py", line 920, in _ensure_unconfigure
    fin()
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/_pytest/capture.py", line 632, in stop_global_capturing
    self._global_capturing.pop_outerr_to_orig()
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/_pytest/capture.py", line 522, in pop_outerr_to_orig
    out, err = self.readouterr()
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/_pytest/capture.py", line 563, in readouterr
    out = self.out.snap()
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/_pytest/capture.py", line 481, in snap
    self.tmpfile.seek(0)
OSError: [Errno 29] Illegal seek
```
I have to add `WANDB_DISABLED=true` to overcome those.
There was an issue about it earlier, but I am not able to find it.

Full report:
```
FAILED tests/test_trainer.py::TrainerIntegrationTest::test_can_resume_training - ValueError: Comet.ml requires an API key. Please provide as the first argument...
ERROR tests/test_trainer.py::TrainerIntegrationTest::test_can_resume_training - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer.py::TrainerIntegrationTest::test_custom_optimizer - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer.py::TrainerIntegrationTest::test_custom_optimizer - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer.py::TrainerIntegrationTest::test_evaluate - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer.py::TrainerIntegrationTest::test_evaluate - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer.py::TrainerIntegrationTest::test_flos_extraction - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer.py::TrainerIntegrationTest::test_flos_extraction - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer.py::TrainerIntegrationTest::test_load_best_model_at_end - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer.py::TrainerIntegrationTest::test_load_best_model_at_end - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer.py::TrainerIntegrationTest::test_model_init - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer.py::TrainerIntegrationTest::test_model_init - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer.py::TrainerIntegrationTest::test_num_train_epochs_in_training - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer.py::TrainerIntegrationTest::test_num_train_epochs_in_training - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer.py::TrainerIntegrationTest::test_number_of_steps_in_training - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer.py::TrainerIntegrationTest::test_number_of_steps_in_training - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer.py::TrainerIntegrationTest::test_predict - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer.py::TrainerIntegrationTest::test_predict - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer.py::TrainerIntegrationTest::test_reproducible_training - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer.py::TrainerIntegrationTest::test_reproducible_training - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer.py::TrainerIntegrationTest::test_save_checkpoints - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer.py::TrainerIntegrationTest::test_save_checkpoints - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer.py::TrainerIntegrationTest::test_train_and_eval_dataloaders - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer.py::TrainerIntegrationTest::test_train_and_eval_dataloaders - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer.py::TrainerIntegrationTest::test_trainer_eval_lm - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer.py::TrainerIntegrationTest::test_trainer_eval_lm - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer.py::TrainerIntegrationTest::test_trainer_eval_mrpc - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer.py::TrainerIntegrationTest::test_trainer_eval_mrpc - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer.py::TrainerIntegrationTest::test_trainer_iterable_dataset - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer.py::TrainerIntegrationTest::test_trainer_iterable_dataset - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer.py::TrainerIntegrationTest::test_trainer_with_datasets - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer.py::TrainerIntegrationTest::test_trainer_with_datasets - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer.py::TrainerIntegrationTest::test_training_arguments_are_left_untouched - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer.py::TrainerIntegrationTest::test_training_arguments_are_left_untouched - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer_callback.py::TrainerCallbackTest::test_add_remove_callback - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer_callback.py::TrainerCallbackTest::test_add_remove_callback - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer_callback.py::TrainerCallbackTest::test_event_flow - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer_callback.py::TrainerCallbackTest::test_event_flow - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer_callback.py::TrainerCallbackTest::test_init_callback - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer_callback.py::TrainerCallbackTest::test_init_callback - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer_utils.py::TrainerUtilsTest::test_distributed_tensor_gatherer - OSError: [Errno 29] Illegal seek
ERROR tests/test_trainer_utils.py::TrainerUtilsTest::test_distributed_tensor_gatherer - OSError: [Errno 29] Illegal seek
ERROR tests/test_utils_check_copies.py::CopyCheckTester::test_find_code_in_transformers - OSError: [Errno 29] Illegal seek
ERROR tests/test_utils_check_copies.py::CopyCheckTester::test_find_code_in_transformers - OSError: [Errno 29] Illegal seek
ERROR tests/test_utils_check_copies.py::CopyCheckTester::test_is_copy_consistent - OSError: [Errno 29] Illegal seek
ERROR tests/test_utils_check_copies.py::CopyCheckTester::test_is_copy_consistent - OSError: [Errno 29] Illegal seek
```

Thank you!

## Environment info

```
- `transformers` version: master
- Platform: Linux-4.15.0-118-generic-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyTorch version (GPU?): 1.8.0.dev20201014 (True)
- Tensorflow version (GPU?): 2.3.1 (True)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
```

@sgugger


Fix small type hinting error

Fix small type hinting error which causes type warnings in some IDEs when using `torch.device` instead of a string


Create README.md

model card

Related to pretrained model cards

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


Remove masked_lm_labels from returned dictionary in DataCollatorForNextSentencePrediction

# What does this PR do?

This PR removes ```masked_lm_labels``` from dictionary returned in DataCollatorForNextSentencePrediction ```__call__``` method. I noticed the warning while BERT pre-training. I removed the lines (452,453) and the warning was finally gone. See discussion George (@gmihaila) and I already had in the [merged PR](https://github.com/huggingface/transformers/pull/7595)

<!-- Remove if not applicable -->

## Before submitting
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case. **Discussed on forum but on previously merged [PR](https://github.com/huggingface/transformers/pull/7595) **
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation). **Not needed**
- [x] Did you write any new necessary tests? **Not needed**

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

@LysandreJik, @sgugger, and @gmihaila can review


Fix missing reference titles in retrieval evaluation of RAG

There's currently a bug in the retrieval evaluation script of RAG where for each sample, the first reference title is discarded.
Moreover only `k` reference titles were taken into account while we need all of them to compute the Precision @ k.

Because of this bug the index provided by the RAG team (hnsw M=128 + SQ8 with the "inner product to L2" trick) only had 35% of Precision @ k. With this fix it is back to 70%, which is consistent with what they had internally as far as I know.

Apparently the original parsing script used to add the question before the titles, that's why it was discarding the first entry.

cc @ola13


RAG - MissingIndex: Index with index_name 'embeddings' not initialized yet

## Environment info
transformers version: 3.3.1
Platform: Ubuntu
Python version:3.6.12
PyTorch version (GPU: yes): 1.6.0
Using GPU in script?: 1 gpu
Using distributed or parallel set-up in script?: no

### Who can help
@patrickvonplaten @sgugger

## Information
model name: facebook/rag-sequence-base

The problem arises when using the official example scripts: (give details below)
The tasks I am working on is my own task or dataset: (give details below)

## To reproduce
1) Make directory at examples/rag/ioannis-data and add train eval and test files in the directory
2) Run transformers/examples/rag/finetune.sh with following changes:
    --data_dir examples/rag/ioannis-data \
    --output_dir examples/rag/ioannis-output  \
    --model_name_or_path facebook/rag-sequence-base

The script terminates with the following error:

`datasets.search.MissingIndex: Index with index_name 'embeddings' not initialized yet. Please make sure that you call 'add_faiss_index' or 'add_elasticsearch_index' first.`


"Cannot re-initialize CUDA in forked subprocess." error when running "transformers/notebooks/05-benchmark.ipynb" notebook

## Environment info
I am getting this error on a server, but also on Collab, so giving the Collab specs:

- `transformers` version: 3.3.1
- Platform: Linux-4.19.112+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.6.0+cu101 (True)
- Tensorflow version (GPU?): 2.3.0 (True)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No?

And on my server:
- `transformers` version: 3.3.1
- Platform: Linux-4.19.0-11-amd64-x86_64-with-debian-10.6
- Python version: 3.6.12
- PyTorch version (GPU?): 1.4.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No?

### Who can help
 @patrickvonplaten

## Information

Model I am using (Bert, XLNet ...): Any model.

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. [Run the colab on benchmarking provided on the transformers GitHub](https://github.com/huggingface/transformers/blob/master/notebooks/05-benchmark.ipynb).

```
2020-10-15 14:20:38.078717: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
1 / 5
Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method
Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method
Traceback (most recent call last):
  File "run_benchmark.py", line 47, in <module>
    main()
  File "run_benchmark.py", line 43, in main
    benchmark.run()
  File "/usr/local/lib/python3.6/dist-packages/transformers/benchmark/benchmark_utils.py", line 674, in run
    memory, inference_summary = self.inference_memory(model_name, batch_size, sequence_length)
ValueError: too many values to unpack (expected 2)
```

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->


BART/TFBart: allow decoder_input_ids.shape[-1] > 1 + use_cache = True

There is an edge case fixed in https://github.com/huggingface/transformers/pull/4581
Try to apply the same logic to bart/move input_id slicing to prepare_inputs_for_generation


Small fixes to NotebookProgressCallback

# What does this PR do?

Fixes a few things in the `NotebookProgressCallback`, mainly:
- triggering forced updates after an evaluation (as an evaluation is longer than just a training step)
- fixing the behavior when an evaluation strategy is not epochs
- removing empty except in the `is_in_notebook` test


the results of run_squad.py is terrible

# ❓ the results of run_squad.py is terrible
## I runned run_squad.py and the results is as folllows:
`Results: {'exact': 40.25098964036048, 'f1': 44.224848486777795, 'total': 11873, 'HasAns_exact': 80.61740890688259, 'HasAns_f1': 88.57652261867625, 'HasAns_total': 5928, 'NoAns_exact': 0.0, 'NoAns_f1': 0.0, 'NoAns_total': 5945, 'best_exact': 50.11370336056599, 'best_exact_thresh': 0.0, 'best_f1': 50.11370336056599, 'best_f1_thresh': 0.0}`

_**`exact`,`f1` and `HasAns_exact`,`HasAns_f1` are very different!**_

## The startup args are as follows:

export SQUAD_DIR=./datasets/squad

python run_squad.py \
  --squad_model bert_qa_model_squad \
  --model_type bert \
  --model_name_or_path bert-base-uncased \
  --do_train \
  --do_eval \
  --do_lower_case \
  --train_file $SQUAD_DIR/train-v2.0.json \
  --predict_file $SQUAD_DIR/dev-v2.0.json \
  --per_gpu_train_batch_size 12 \
  --learning_rate 3e-5 \
  --num_train_epochs 2.0 \
  --max_seq_length 384 \
  --doc_stride 128 \
  --output_dir ./models/debug_squad/


Fix issue #7781

`decoder_config` may not be defined at this line, use `kwargs_decoder["config"]` instead. This is also consistent with the rest of the function.

Fixes # 7781
https://github.com/huggingface/transformers/issues/7781

## Who can review?

Anybody can review, though @patrickvonplaten gave an informal approval in the issue discussion.


Metrics calculate error: can only calculate the mean of floating types. Got Bool instead

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 3.3.1
- Platform: Ubuntu 18.04
- Python version: 3.7.7
- PyTorch version (GPU?): 1.6.0 GPU
- Tensorflow version (GPU?): -
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Yes

### Who can help
Someone working on the examples/GLUE benchmarks

## Information

Model I am using (Bert, XLNet ...): BERT

The problem arises when using:
* [X] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [X] an official GLUE/SQUaD task: (GLUE
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Run QNLI experiment

Error:
```bash
File "/home/lucadiliello/anaconda3/envs/nlp/lib/python3.7/site-packages/transformers/data/metrics/__init__.py", line 73, in glue_compute_metrics
    return {"acc": simple_accuracy(preds, labels)}
  File "/home/lucadiliello/anaconda3/envs/nlp/lib/python3.7/site-packages/transformers/data/metrics/__init__.py", line 34, in simple_accuracy
    return (preds == labels).mean()
RuntimeError: Can only calculate the mean of floating types. Got Bool instead.
Exception ignored in: <function tqdm.__del__ at 0x7f8d1d09ff28>
Traceback (most recent call last):
  File "/home/lucadiliello/anaconda3/envs/nlp/lib/python3.7/site-packages/tqdm/std.py", line 1086, in __del__
  File "/home/lucadiliello/anaconda3/envs/nlp/lib/python3.7/site-packages/tqdm/std.py", line 1293, in close
  File "/home/lucadiliello/anaconda3/envs/nlp/lib/python3.7/site-packages/tqdm/std.py", line 1471, in display
  File "/home/lucadiliello/anaconda3/envs/nlp/lib/python3.7/site-packages/tqdm/std.py", line 1089, in __repr__
  File "/home/lucadiliello/anaconda3/envs/nlp/lib/python3.7/site-packages/tqdm/std.py", line 1433, in format_dict
TypeError: cannot unpack non-iterable NoneType object
```

A simple fix is to modify `transformers/data/metrics/__init__.py", line 34, in simple_accuracy` to:
```python
return (preds == labels).to(dtype=torch.float32).mean()
```

## Expected behavior

Metric should be computed without problems


[Seq2Seq] Allow EncoderDecoderModels to be trained with Seq2Seq

# What does this PR do?

This PR changes the Seq2Seq Trainer a bit to:
1) Make it work with `EncoderDecoder`
2) Align its API more with the general `Trainer`

@sshleifer @patil-suraj @sgugger - it would be great if you could take a look and give your general opinion on it :-)
If this would be ok for you, I will fix the examples test.