[test] tests/test_modeling_deberta.py breaks on pytorch-nightly
Author: stas00
Pull request: no

pytorch-nightly and pytorch-1.7-candidate break these:
```
FAILED tests/test_modeling_deberta.py::DebertaModelTest::test_deberta_model - RuntimeError: diff_view_meta->output_nr_ == 0 INTERNAL ASSERT FAILED...
FAILED tests/test_modeling_deberta.py::DebertaModelTest::test_feed_forward_chunking - RuntimeError: diff_view_meta->output_nr_ == 0 INTERNAL ASSER...
FAILED tests/test_modeling_deberta.py::DebertaModelTest::test_for_sequence_classification - RuntimeError: diff_view_meta->output_nr_ == 0 INTERNAL...
FAILED tests/test_modeling_deberta.py::DebertaModelTest::test_resize_tokens_embeddings - RuntimeError: diff_view_meta->output_nr_ == 0 INTERNAL AS...
```

looks like a bug in pytorch, right?

```
RuntimeError: diff_view_meta->output_nr_ == 0 INTERNAL ASSERT FAILED at "/opt/conda/conda- \ 
bld/pytorch_1603436966316/work/torch/csrc/autograd/variable.cpp":363, please report a bug to PyTorch.
```

log:
```
================================================================ test session starts ================================================================
platform linux -- Python 3.8.5, pytest-6.1.1, py-1.9.0, pluggy-0.13.1 -- /home/stas/anaconda3/envs/main-38/bin/python
cachedir: .pytest_cache
rootdir: /mnt/nvme1/code/huggingface/transformers-master
plugins: typeguard-2.10.0, forked-1.3.0, xdist-2.1.0, instafail-0.4.2
collecting ... 2020-10-24 09:21:07.169605: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
collected 1 item                                                                                                                                    

tests/test_modeling_deberta.py::DebertaModelTest::test_deberta_model FAILED
________________________________________________________ DebertaModelTest.test_deberta_model ________________________________________________________

self = <tests.test_modeling_deberta.DebertaModelTest testMethod=test_deberta_model>

    def test_deberta_model(self):
        config_and_inputs = self.model_tester.prepare_config_and_inputs()
>       self.model_tester.create_and_check_deberta_model(*config_and_inputs)

tests/test_modeling_deberta.py:210: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/test_modeling_deberta.py:159: in create_and_check_deberta_model
    sequence_output = model(input_ids, attention_mask=input_mask, token_type_ids=token_type_ids)[0]
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/modules/module.py:744: in _call_impl
    result = self.forward(*input, **kwargs)
src/transformers/modeling_deberta.py:891: in forward
    encoder_outputs = self.encoder(
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/modules/module.py:744: in _call_impl
    result = self.forward(*input, **kwargs)
src/transformers/modeling_deberta.py:401: in forward
    hidden_states = layer_module(
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/modules/module.py:744: in _call_impl
    result = self.forward(*input, **kwargs)
src/transformers/modeling_deberta.py:324: in forward
    attention_output = self.attention(
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/modules/module.py:744: in _call_impl
    result = self.forward(*input, **kwargs)
src/transformers/modeling_deberta.py:257: in forward
    self_output = self.self(
/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/torch/nn/modules/module.py:744: in _call_impl
    result = self.forward(*input, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = DisentangledSelfAttention(
  (in_proj): Linear(in_features=32, out_features=96, bias=False)
  (dropout): StableDropout()
)
hidden_states = tensor([[[-1.5951, -1.0046,  0.5641,  ..., -0.4472,  0.0159, -0.1435],
         [-1.3476, -0.1559, -1.3866,  ...,  1.2...        [ 0.5269, -0.0601, -0.4018,  ..., -0.1616,  0.5335, -0.8894]]],
       device='cuda:0', grad_fn=<MulBackward0>)
attention_mask = tensor([[[[1, 1, 1, 1, 0, 0, 0],
          [1, 1, 1, 1, 0, 0, 0],
          [1, 1, 1, 1, 0, 0, 0],
          [1, 1, 1,..., 1, 1, 1, 1],
          [0, 0, 0, 1, 1, 1, 1],
          [0, 0, 0, 1, 1, 1, 1]]]], device='cuda:0', dtype=torch.uint8)
return_att = False, query_states = None, relative_pos = None, rel_embeddings = None

    def forward(
        self,
        hidden_states,
        attention_mask,
        return_att=False,
        query_states=None,
        relative_pos=None,
        rel_embeddings=None,
    ):
        """Call the module
    
        Args:
            hidden_states (:obj:`torch.FloatTensor`):
                Input states to the module usally the output from previous layer, it will be the Q,K and V in `Attention(Q,K,V)`
    
            attention_mask (:obj:`torch.ByteTensor`):
                An attention mask matrix of shape [`B`, `N`, `N`] where `B` is the batch size, `N` is the maxium sequence length in which element [i,j] = `1` means the `i` th token in the input can attend to the `j` th token.                                                                         
    
            return_att (:obj:`bool`, optional):
                Whether return the attention maxitrix.
    
            query_states (:obj:`torch.FloatTensor`, optional):
                The `Q` state in `Attention(Q,K,V)`.
    
            relative_pos (:obj:`torch.LongTensor`):
                The relative position encoding between the tokens in the sequence. It's of shape [`B`, `N`, `N`] with values ranging in [`-max_relative_positions`, `max_relative_positions`].

            rel_embeddings (:obj:`torch.FloatTensor`):
                The embedding of relative distances. It's a tensor of shape [:math:`2 \\times \\text{max_relative_positions}`, `hidden_size`].


        """
        if query_states is None:
            qp = self.in_proj(hidden_states)  # .split(self.all_head_size, dim=-1)
            query_layer, key_layer, value_layer = self.transpose_for_scores(qp).chunk(3, dim=-1)
        else:

            def linear(w, b, x):
                if b is not None:
                    return torch.matmul(x, w.t()) + b.t()
                else:
                    return torch.matmul(x, w.t())  # + b.t()

            ws = self.in_proj.weight.chunk(self.num_attention_heads * 3, dim=0)
            qkvw = [torch.cat([ws[i * 3 + k] for i in range(self.num_attention_heads)], dim=0) for k in range(3)]
            qkvb = [None] * 3

            q = linear(qkvw[0], qkvb[0], query_states)
            k, v = [linear(qkvw[i], qkvb[i], hidden_states) for i in range(1, 3)]
            query_layer, key_layer, value_layer = [self.transpose_for_scores(x) for x in [q, k, v]]

        query_layer += self.transpose_for_scores(self.q_bias[None, None, :])
>       value_layer += self.transpose_for_scores(self.v_bias[None, None, :])
E       RuntimeError: diff_view_meta->output_nr_ == 0 INTERNAL ASSERT FAILED at "/opt/conda/conda-bld/pytorch_1603436966316/work/torch/csrc/autograd/variable.cpp":363, please report a bug to PyTorch.

src/transformers/modeling_deberta.py:575: RuntimeError

============================================================== short test summary info ==============================================================
FAILED tests/test_modeling_deberta.py::DebertaModelTest::test_deberta_model - RuntimeError: diff_view_meta->output_nr_ == 0 INTERNAL ASSERT FAILED...
=========================================================== 1 failed, 5 warnings in 4.22s ===========================================================
```


## Environment info
```
- `transformers` version: 3.4.0
- Platform: Linux-5.4.0-52-generic-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyTorch version (GPU?): 1.8.0.dev20201023 (True)
- Tensorflow version (GPU?): 2.3.1 (True)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
```

for pytorch-1.7 candidate I used:

https://download.pytorch.org/whl/test/cu102/torch-1.7.0-cp38-cp38-linux_x86_64.whl 

----------------------

**update**: The release team https://github.com/pytorch/pytorch/issues/45592 has been notified of this issue on pytorch slack. Waiting to hear back from them.

Comments:
I couldn't reproduce this against 1.6.  When I run it against 1.7, I get:

  /data/users/gchanan/transformers/src/transformers/modeling_deberta.py:574: UserWarning: Output 0 of SplitBackward is a view and is being modified inplace. This view is an output of a function that returns multiple views. Inplace operators on such views are being deprecated and will be forbidden starting from version 1.8. Consider using `unsafe_` version of the function that produced this view or don't modify this view inplace. (Triggered internally at  /opt/conda/conda-bld/pytorch_1601363278767/work/torch/csrc/autograd/variable.cpp:480.)

Here's a workaround: change
https://github.com/huggingface/transformers/blob/5148f433097915f30864bf0ca6090656fecefbb8/src/transformers/modeling_deberta.py#L574

to:
`query_layer = query_layer + self.transpose_for_scores(self.q_bias[None, None, :])`

i.e. make the modification out-of-place.  It might be better to do what is in the warning and change the `split` to `unsafe_split`, but I haven't tested that.

Thank you very much, @gchanan! That solved the problem.

Also pasting a more details explanation from slack by @albanD:

> The warning that is raised just before tells you what the issue is and tells you that this won't be allowed soon as it can crash (as you have seen). Removing the inplace is the right fix here.
The reason here is that split and chunk were fixed to properly return view (and avoid silently wrong gradients). But Inplace on the result is not handled by the autograd and will soon raise an error.
We have left unsafe_split and unsafe_chunk (both in python and c++) if you need the old behavior while you fix your code to avoid the inplace.