Switch `return_dict` to `True` by default.
Author: sgugger
Pull request: yes
Labels: name: model card

# What does this PR do?

As part of the main changes we want to bring to Transformers v4, this PR switches the `return_dict` argument default from `False` to `True`. Most of the examples and documentation were already using this value (the PR removes `return_dict=True` in all those instances since this is now the default).

**New model outputs**

The new model outputs are dictionaries (instead of tuples) with a bit of added functionality: you can access elements by their keys, as attributes or even by index (to keep most of the backward compatibility). Here is an example:

```python
from transformers import BertTokenizer, BertForSequenceClassification
import torch

tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertForSequenceClassification.from_pretrained('bert-base-uncased')

inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
labels = torch.tensor([1]).unsqueeze(0)  # Batch size 1
outputs = model(**inputs, labels=labels)
```

The `outputs` object contains loss and logits and you can access them as keys:
```python
loss = outputs["loss"]
logits = outputs["logits"]
```
or as attributes.
```python
loss = outputs.loss
logits = outputs.logits
```
In an IDE you can thus use autocomplete to help.

You can still index with integers or a slice:
```python
loss = outputs[0]
logits = outputs[1]
loss, logits = outputs[:2]
```
but you can't unpack the tuple directly as was done before:
```python
loss, logits = outputs
```
will return "loss" and "logits" in `loss` and `logits` (like dictionaries).

**Known caveats**

Torchscript is incompatible with dict outputs (until 1.7) so doesn't work with those new outputs. The flag `return_dict` will default to `False` if you use `torchscript=True` when building your model.

TF Saved models deal with dictionaries but not their subclasses, so if you save and load your model using `tf.keras.models.load_model`, it will output regular dictionaries. This means that you won't be able to index in the outputs with integers or attributes, only their string keys.

```python
tf.saved_model.save(model, tmpdirname)
model = tf.keras.models.load_model(tmpdirname)
outputs = model(inputs)
loss = outputs["loss"]
logits = outputs["logits"]
```

Apex used in optimization mode "O2" will also lose the output types of the model and return regular dictionaries. This means that you won't be able to index in the outputs with integers or attributes, only their string keys.

**Breaking change**

All code of the form
```python
loss, output = model(**inputs)
``` 
needs to be switched to
```python
loss, output = model(**inputs).to_tuple()
``` 
or use the key/attributes of the model output returned.

Alternatively, you can pass `return_dict=False` when creating your model to get regular tuples as outputs.

Comments:
> You can still index with integers or a slice:
> loss = outputs[0]

Doesn't seem to be the case if I use fp16/apex (possibly any fp16):

`finetune.py` currently fails with:

```
  File "finetune.py", line 170, in training_step
    loss_tensors = self._step(batch)
  File "finetune.py", line 151, in _step
    lm_logits = outputs[0]
KeyError: 0
```

Oddly enough all finetune tests pass, and it works with fp32, but if I pass apex (which tests don't test) it fails. Any ideas how fp16 could make a difference? I can't test native amp due to the leak.

This may have something to do with PL too, since the call stack goes through PL:

Full backtrace follows:

```
Traceback (most recent call last):
  File "finetune.py", line 444, in <module>
    main(args)
  File "finetune.py", line 411, in main
    trainer: pl.Trainer = generic_train(
  File "/mnt/nvme1/code/huggingface/transformers-finetune-fixes/examples/lightning_base.py", line 398, in generic_train
    trainer.fit(model)
  File "/mnt/nvme1/code/github/00pytorch/pytorch-lightning/pytorch_lightning/trainer/trainer.py", line 469, in fit
    results = self.accelerator_backend.train()
  File "/mnt/nvme1/code/github/00pytorch/pytorch-lightning/pytorch_lightning/accelerators/gpu_accelerator.py", line 64, in train
    results = self.train_or_test()
  File "/mnt/nvme1/code/github/00pytorch/pytorch-lightning/pytorch_lightning/accelerators/accelerator.py", line 66, in train_or_test
    results = self.trainer.train()
  File "/mnt/nvme1/code/github/00pytorch/pytorch-lightning/pytorch_lightning/trainer/trainer.py", line 521, in train
    self.train_loop.run_training_epoch()
  File "/mnt/nvme1/code/github/00pytorch/pytorch-lightning/pytorch_lightning/trainer/training_loop.py", line 539, in run_training_epoch
    batch_output = self.run_training_batch(batch, batch_idx, dataloader_idx)
  File "/mnt/nvme1/code/github/00pytorch/pytorch-lightning/pytorch_lightning/trainer/training_loop.py", line 691, in run_training_batch
    self.optimizer_step(optimizer, opt_idx, batch_idx, train_step_and_backward_closure)
  File "/mnt/nvme1/code/github/00pytorch/pytorch-lightning/pytorch_lightning/trainer/training_loop.py", line 477, in optimizer_step
    self.trainer.accelerator_backend.optimizer_step(
  File "/mnt/nvme1/code/github/00pytorch/pytorch-lightning/pytorch_lightning/accelerators/accelerator.py", line 114, in optimizer_step
    model_ref.optimizer_step(
  File "/mnt/nvme1/code/github/00pytorch/pytorch-lightning/pytorch_lightning/core/lightning.py", line 1406, in optimizer_step
    optimizer_closure()
  File "/mnt/nvme1/code/github/00pytorch/pytorch-lightning/pytorch_lightning/trainer/training_loop.py", line 681, in train_step_and_backward_closure
    result = self.training_step_and_backward(
  File "/mnt/nvme1/code/github/00pytorch/pytorch-lightning/pytorch_lightning/trainer/training_loop.py", line 770, in training_step_and_backward
    result = self.training_step(split_batch, batch_idx, opt_idx, hiddens)
  File "/mnt/nvme1/code/github/00pytorch/pytorch-lightning/pytorch_lightning/trainer/training_loop.py", line 324, in training_step
    training_step_output = self.trainer.accelerator_backend.training_step(args)
  File "/mnt/nvme1/code/github/00pytorch/pytorch-lightning/pytorch_lightning/accelerators/gpu_accelerator.py", line 72, in training_step
    output = self.__training_step(args)
  File "/mnt/nvme1/code/github/00pytorch/pytorch-lightning/pytorch_lightning/accelerators/gpu_accelerator.py", line 80, in __training_step
    output = self.trainer.model.training_step(*args)
  File "finetune.py", line 172, in training_step
    loss_tensors = self._step(batch)
  File "finetune.py", line 153, in _step
    lm_logits = outputs[0]
KeyError: 0
```

I will fix the program, but thought I'd share this strange case.

I really should get away from finetune for awhile, it seems to be the doorway to some incessant work...



Fixed with:
```
-        lm_logits = outputs[0]
+        lm_logits = outputs["logits"]
```
can't use attributes either:

`outputs.logits` fails:

`AttributeError: 'dict' object has no attribute 'logits'`