[bart] SinusoidalPositionalEmbedding breaks under pytorch-nightly
Author: stas00
Pull request: no

pytorch-nightly breaks these:
```
FAILED tests/test_modeling_bart.py::TestSinusoidalPositionalEmbeddings::test_odd_embed_dim - RuntimeError: a view of a leaf Variable that requires...
FAILED tests/test_modeling_bart.py::TestSinusoidalPositionalEmbeddings::test_positional_emb_cache_logic - RuntimeError: a view of a leaf Variable ...
FAILED tests/test_modeling_bart.py::TestSinusoidalPositionalEmbeddings::test_positional_emb_weights_against_marian - RuntimeError: a view of a lea...
F
```




```
================================================================ test session starts ================================================================
platform linux -- Python 3.8.5, pytest-6.1.1, py-1.9.0, pluggy-0.13.1 -- /home/stas/anaconda3/envs/main-38/bin/python
cachedir: .pytest_cache
rootdir: /mnt/nvme1/code/huggingface/transformers-master
plugins: typeguard-2.10.0, forked-1.3.0, xdist-2.1.0, instafail-0.4.2
collecting ... 2020-10-24 09:14:35.276431: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
collected 1 item                                                                                                                                    

tests/test_modeling_bart.py::TestSinusoidalPositionalEmbeddings::test_odd_embed_dim FAILED
_______________________________________________ TestSinusoidalPositionalEmbeddings.test_odd_embed_dim _______________________________________________

self = <tests.test_modeling_bart.TestSinusoidalPositionalEmbeddings testMethod=test_odd_embed_dim>

    def test_odd_embed_dim(self):
        with self.assertRaises(NotImplementedError):
            SinusoidalPositionalEmbedding(num_positions=4, embedding_dim=5, padding_idx=0).to(torch_device)

        # odd num_positions is allowed
>       SinusoidalPositionalEmbedding(num_positions=5, embedding_dim=4, padding_idx=0).to(torch_device)

tests/test_modeling_bart.py:627:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
src/transformers/modeling_bart.py:1331: in __init__
    self.weight = self._init_weight(self.weight)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

out = Parameter containing:
tensor([[ 0.1368,  2.6925,  1.3918,  0.7332],
        [-1.2813, -0.3071,  1.0553, -0.4325],
    ...6445],
        [-1.6619, -0.2872,  0.6869,  0.6489],
        [-1.5226,  0.1161, -0.2026,  0.1853]], requires_grad=True)

    @staticmethod
    def _init_weight(out: nn.Parameter):
        """Identical to the XLM create_sinusoidal_embeddings except features are not interleaved.
        The cos features are in the 2nd half of the vector. [dim // 2:]
        """
        n_pos, dim = out.shape
        position_enc = np.array(
            [[pos / np.power(10000, 2 * (j // 2) / dim) for j in range(dim)] for pos in range(n_pos)]
        )
>       out[:, 0 : dim // 2] = torch.FloatTensor(np.sin(position_enc[:, 0::2]))  # This line breaks for odd n_pos
E       RuntimeError: a view of a leaf Variable that requires grad is being used in an in-place operation.

src/transformers/modeling_bart.py:1342: RuntimeError

============================================================== short test summary info ==============================================================
FAILED tests/test_modeling_bart.py::TestSinusoidalPositionalEmbeddings::test_odd_embed_dim - RuntimeError: a view of a leaf Variable that requires...
=========================================================== 1 failed, 3 warnings in 3.01s ===========================================================

```

## Environment info
```
- `transformers` version: 3.4.0
- Platform: Linux-5.4.0-52-generic-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyTorch version (GPU?): 1.8.0.dev20201023 (True)
- Tensorflow version (GPU?): 2.3.1 (True)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
```

Comments: