a transparent solution for  DataParallel.gather not supporting ModelOutput (dataclass)
Author: stas00
Pull request: yes

1. Modify torch/nn/parallel/scatter_gather.gather function to support ModelOutput (dataclass) outputs. We override the torch.nn.DataParallel.gather method with this custom method. 

2. remove previously committed workarounds

implementation: @sgugger 
integration/testing: me

This should transparently solve https://github.com/huggingface/transformers/issues/5693


Comments:
This solves the problems encountered after the model outputs PR and doesn't break anything in existing PyTorch behavior. I don't know how the rest of the team feels about patching over libraries methods though.

Other possible fixes are:
- breaking changes and just use dicts everywhere (supported by DataParallel and supported by JIT in PyTorch 1.6.0)
- adding a hook at init of `PretrainedModel` to check at the first forward pass if the model has been wrapped in a DataParallel container and setting return_tuple in that case (but that's kind of ugly).
- manually testing in every model during the forward pass if it has been wrapped in a DataParallel container and setting return_tuple in that case (by changing the test that sets `return_tuple`).

One extra note to @sgugger's comments is that even if pytorch fixes `gather` to support `dataclasses` - converting them to dicts, it still won't be sufficient, since we have models where the output dataclass has optional members (w/o defaults) followed by required members, e.g. `MaskedLMOutput`, so such possibly modified by pytorch `gather` would fail here, unless the optional members are looked up and `None` is passed to the constructor - I guess this could be done in pytorch's `gather` as well.

Perhaps coming up with a pure `dataclasses` and not `ModelOutput`-specific implementation, that can be given to the pytorch team? and then this workaround will be needed only for older pytorch versions. 

# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5941?src=pr&el=h1) Report
> Merging [#5941](https://codecov.io/gh/huggingface/transformers/pull/5941?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/32883b310ba30d72e67bb2ebb5847888f03a90a8&el=desc) will **decrease** coverage by `0.05%`.
> The diff coverage is `19.23%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5941/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5941?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #5941      +/-   ##
==========================================
- Coverage   78.51%   78.46%   -0.06%     
==========================================
  Files         146      146              
  Lines       26214    26236      +22     
==========================================
+ Hits        20583    20585       +2     
- Misses       5631     5651      +20     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5941?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/trainer.py](https://codecov.io/gh/huggingface/transformers/pull/5941/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90cmFpbmVyLnB5) | `38.58% <ø> (+0.12%)` | :arrow_up: |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5941/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `76.73% <19.23%> (-4.77%)` | :arrow_down: |
| [src/transformers/generation\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5941/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9nZW5lcmF0aW9uX3RmX3V0aWxzLnB5) | `86.21% <0.00%> (-0.26%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5941?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5941?src=pr&el=footer). Last update [32883b3...528758e](https://codecov.io/gh/huggingface/transformers/pull/5941?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Thinking of it a little bit more, if PyTorch starts to support dataclasses in `gather`, we can then drop the patching by having `= None` for every attribute of every type of `ModelOuput` (as is done in #5740).

Since the model outputs had the other problem of not working with TensorFlow, we are going on a different road (see [the forum](https://discuss.huggingface.co/t/new-model-output-types/195/8)).