Model2Model: RuntimeError: expected device cpu and dtype Float but got device cpu and dtype Bool
Author: amirj
Pull request: no

## ❓ Questions & Help

I'm going to try the new Model2Model feature:

```
import torch
import numpy as np
from transformers import Model2Model, BertTokenizer, BertModel

# device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
device = torch.device("cpu")

tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = Model2Model.from_pretrained('bert-base-uncased').to(device)

source_ids = torch.tensor([tokenizer.encode("this is source sentence", add_special_tokens=True)], 
                          dtype=torch.long).to(device)
target_ids = torch.tensor([tokenizer.encode("this is target sentence", add_special_tokens=True)], 
                          dtype=torch.long).to(device)

model(source_ids, target_ids)
```

This is the output:
```
RuntimeError                              Traceback (most recent call last)
<ipython-input-10-885d1d4b847f> in <module>
----> 1 model(source_ids, target_ids)

/users/tr.amirhj/anaconda3/envs/genv/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    545             result = self._slow_forward(*input, **kwargs)
    546         else:
--> 547             result = self.forward(*input, **kwargs)
    548         for hook in self._forward_hooks.values():
    549             hook_result = hook(self, input, result)

/users/tr.amirhj/anaconda3/envs/genv/lib/python3.6/site-packages/transformers/modeling_encoder_decoder.py in forward(self, encoder_input_ids, decoder_input_ids, **kwargs)
    229             "attention_mask", None
    230         )
--> 231         decoder_outputs = self.decoder(decoder_input_ids, **kwargs_decoder)
    232 
    233         return decoder_outputs + encoder_outputs

/users/tr.amirhj/anaconda3/envs/genv/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    545             result = self._slow_forward(*input, **kwargs)
    546         else:
--> 547             result = self.forward(*input, **kwargs)
    548         for hook in self._forward_hooks.values():
    549             hook_result = hook(self, input, result)

/users/tr.amirhj/anaconda3/envs/genv/lib/python3.6/site-packages/transformers/modeling_bert.py in forward(self, input_ids, attention_mask, token_type_ids, position_ids, head_mask, inputs_embeds, masked_lm_labels, encoder_hidden_states, encoder_attention_mask, lm_labels)
    871                             inputs_embeds=inputs_embeds,
    872                             encoder_hidden_states=encoder_hidden_states,
--> 873                             encoder_attention_mask=encoder_attention_mask)
    874 
    875         sequence_output = outputs[0]

/users/tr.amirhj/anaconda3/envs/genv/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    545             result = self._slow_forward(*input, **kwargs)
    546         else:
--> 547             result = self.forward(*input, **kwargs)
    548         for hook in self._forward_hooks.values():
    549             hook_result = hook(self, input, result)

/users/tr.amirhj/anaconda3/envs/genv/lib/python3.6/site-packages/transformers/modeling_bert.py in forward(self, input_ids, attention_mask, token_type_ids, position_ids, head_mask, inputs_embeds, encoder_hidden_states, encoder_attention_mask)
    677                 seq_ids = torch.arange(seq_length, device=device)
    678                 causal_mask = seq_ids[None, None, :].repeat(batch_size, seq_length, 1) <= seq_ids[None, :, None]
--> 679                 extended_attention_mask = causal_mask[:, None, :, :] * attention_mask[:, None, None, :]
    680             else:
    681                 extended_attention_mask = attention_mask[:, None, None, :]

RuntimeError: expected device cpu and dtype Float but got device cpu and dtype Bool
```

If I missed something?

Comments:
In my environment, the code you've posted **works as expected**.
- Python: 3.6.9
- Transformers: 2.2.1 (installed from PyPi with pip install transformers)
- PyTorch: 1.3.1
- TensorFlow: 2.0
- OS: Ubuntu 16.04

Here the stack trace:
```
Python 3.6.9 |Anaconda, Inc.| (default, Jul 30 2019, 19:07:31) 
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> import numpy as np
>>> from transformers import Model2Model, BertTokenizer, BertModel
/home/<user>/anaconda3/envs/huggingface/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/<user>/anaconda3/envs/huggingface/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/<user>/anaconda3/envs/huggingface/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/<user>/anaconda3/envs/huggingface/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/<user>/anaconda3/envs/huggingface/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/<user>/anaconda3/envs/huggingface/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
2019-12-10 17:40:40.255384: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-12-10 17:40:40.277896: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3408000000 Hz
2019-12-10 17:40:40.279096: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x558b757804f0 executing computations on platform Host. Devices:
2019-12-10 17:40:40.279146: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): Host, Default Version
>>> device = torch.device("cpu")
>>> device
device(type='cpu')
>>> tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
>>> model = Model2Model.from_pretrained('bert-base-uncased').to(device)
>>> source_ids = torch.tensor([tokenizer.encode("this is source sentence", add_special_tokens=True)], 
...                           dtype=torch.long).to(device)
>>> target_ids = torch.tensor([tokenizer.encode("this is target sentence", add_special_tokens=True)], 
...                           dtype=torch.long).to(device)
>>> 
>>> source_ids
tensor([[ 101, 2023, 2003, 3120, 6251,  102]])
>>> target_ids
tensor([[ 101, 2023, 2003, 4539, 6251,  102]])
>>> model(source_ids, target_ids)
(tensor([[[ -6.3390,  -6.3664,  -6.4600,  ...,  -5.5354,  -4.1787,  -5.8384],
         [ -7.9225,  -7.7588,  -7.9552,  ...,  -6.6068,  -5.5835,  -6.9365],
         [-10.4273, -10.3139, -10.5899,  ...,  -9.5835,  -7.8032,  -9.9118],
         [ -8.8252,  -8.6229,  -8.8085,  ...,  -8.0037,  -6.6364,  -8.5376],
         [ -8.6978,  -8.4959,  -8.5453,  ...,  -7.9320,  -6.6115,  -8.7994],
         [-13.0414, -12.5687, -12.3714,  ..., -10.1630, -11.1963,  -9.3892]]],
       grad_fn=<AddBackward0>), tensor([[[-3.0067e-01,  1.5002e-01,  1.7042e-02,  ..., -3.6836e-01,
           2.2961e-01,  8.0086e-01],
         [-1.0987e+00, -2.3812e-01,  1.9028e-01,  ..., -6.9975e-01,
           6.7476e-01,  2.9067e-01],
         [-1.7711e-01, -3.5428e-01,  3.6858e-01,  ..., -1.1280e-01,
           1.6458e-01,  1.1558e+00],
         [-5.6245e-01, -1.9310e-01,  1.7546e-01,  ..., -1.8610e-02,
          -1.1314e-03,  3.2184e-01],
         [-3.8065e-01, -1.8030e-01, -1.2957e-01,  ...,  4.6774e-01,
           1.4298e-01, -1.8563e-01],
         [ 7.8768e-01,  1.0423e-01, -4.0617e-01,  ...,  2.6467e-01,
          -7.9018e-01, -1.9337e-01]]], grad_fn=<NativeLayerNormBackward>), tensor([[-9.0084e-01, -3.3628e-01,  2.9453e-01,  7.1089e-01, -1.0436e-02,
         -2.5144e-01,  9.0506e-01,  2.9434e-01,  1.6485e-01, -9.9996e-01,
          2.0915e-01,  2.8445e-01,  9.8403e-01, -3.0770e-01,  9.2687e-01,
         -6.1045e-01, -1.2372e-01, -5.7582e-01,  3.9420e-01, -7.7367e-01,
          6.4602e-01,  9.8964e-01,  6.7300e-01,  2.6016e-01,  4.0054e-01,
          4.2634e-01, -6.1309e-01,  9.4336e-01,  9.6244e-01,  7.9033e-01,
         -7.7723e-01,  2.5581e-01, -9.9027e-01, -2.3506e-01, -1.6533e-01,
         -9.8790e-01,  2.4701e-01, -7.8211e-01, -9.2877e-02, -4.5130e-02,
         -9.2165e-01,  3.7376e-01,  9.9949e-01, -2.2205e-01,  1.6105e-01,
         -3.5879e-01, -9.9999e-01,  3.1183e-01, -9.0365e-01, -1.2721e-01,
         -5.7083e-02, -3.8538e-01,  2.2891e-01,  4.1976e-01,  4.4054e-01,
          2.7219e-01, -1.6016e-02,  2.7714e-01, -1.6180e-01, -5.8537e-01,
         -6.2011e-01,  3.2424e-01, -1.1204e-01, -9.2093e-01, -1.9166e-01,
         -3.7498e-01, -1.5816e-01, -2.6796e-01, -1.0934e-01, -3.2014e-02,
          8.7326e-01,  2.5321e-01,  3.1921e-01, -8.0303e-01, -3.4841e-01,
          2.4700e-01, -4.6604e-01,  1.0000e+00, -4.6661e-01, -9.8111e-01,
         -1.2605e-01, -1.8299e-01,  4.1548e-01,  6.1520e-01, -3.6703e-01,
         -1.0000e+00,  3.6013e-01, -2.1875e-01, -9.9034e-01,  1.5795e-01,
          4.0751e-01, -2.1697e-01, -3.0685e-01,  3.8455e-01, -1.3388e-01,
         -1.6273e-01, -3.3509e-01,  7.5851e-03, -2.6005e-01, -1.5252e-01,
          1.6267e-01, -2.9343e-01, -1.8843e-01, -2.8192e-01,  1.9310e-01,
         -3.3882e-01, -4.8637e-01,  3.5417e-01, -4.0395e-01,  7.1347e-01,
          3.5647e-01, -3.2761e-01,  3.3358e-01, -9.4952e-01,  5.4614e-01,
         -2.8969e-01, -9.8452e-01, -4.2365e-01, -9.8693e-01,  7.5074e-01,
         -3.5488e-02, -2.6717e-01,  9.6647e-01,  5.1186e-01,  2.8068e-01,
         -1.0258e-01, -2.3203e-03, -1.0000e+00, -9.8173e-02, -3.1035e-01,
          2.0420e-01, -1.8622e-01, -9.8229e-01, -9.5138e-01,  6.5169e-01,
          9.6339e-01,  2.2344e-01,  9.9859e-01, -2.5536e-01,  9.4590e-01,
          3.1677e-01, -1.7800e-01, -5.1792e-01, -4.0876e-01,  5.2822e-01,
          5.4555e-01, -8.1303e-01,  2.1158e-01,  9.4905e-02, -8.9056e-02,
         -2.3806e-01, -3.3301e-01,  1.6834e-01, -9.2588e-01, -4.2112e-01,
          9.3633e-01,  2.8537e-01,  7.7606e-02,  7.2043e-01, -1.9238e-01,
         -3.9200e-01,  8.6078e-01,  3.3558e-01,  3.0295e-01,  6.4802e-02,
          4.6284e-01, -8.7253e-02,  4.8427e-01, -9.0531e-01,  3.4391e-01,
          4.1636e-01, -1.6641e-01,  1.7450e-01, -9.7965e-01, -3.0878e-01,
          4.5623e-01,  9.8710e-01,  8.1641e-01,  2.8662e-01,  5.9909e-02,
         -3.3217e-01,  2.3228e-01, -9.5294e-01,  9.7835e-01, -1.7293e-01,
          2.3846e-01,  4.8146e-01,  6.5912e-03, -8.8724e-01, -3.5229e-01,
          8.4911e-01, -3.5286e-02, -8.8944e-01, -5.5141e-02, -4.7656e-01,
         -4.7363e-01, -3.5688e-02,  6.3608e-01, -3.2397e-01, -4.2425e-01,
         -5.4916e-02,  9.3040e-01,  9.7627e-01,  7.4838e-01, -5.1590e-01,
          4.6674e-01, -9.0206e-01, -5.0592e-01,  1.5316e-01,  2.7624e-01,
          1.7898e-01,  9.9323e-01, -1.4045e-01, -1.6275e-01, -9.1684e-01,
         -9.8267e-01,  3.2413e-02, -8.8971e-01, -3.2410e-02, -7.1453e-01,
          4.0365e-01,  5.0860e-01, -2.6739e-01,  3.7175e-01, -9.8981e-01,
         -8.5210e-01,  3.3096e-01, -3.1729e-01,  4.9861e-01, -2.0997e-01,
          5.6376e-01,  1.7651e-01, -6.6355e-01,  7.7454e-01,  9.3114e-01,
          2.3015e-01, -7.5848e-01,  8.5644e-01, -2.3493e-01,  9.0546e-01,
         -6.1747e-01,  9.8845e-01,  2.5930e-01,  3.8508e-01, -9.3526e-01,
          1.6509e-01, -9.2224e-01,  1.8666e-01, -1.8823e-01, -6.0511e-01,
         -1.4290e-01,  4.5802e-01,  2.9694e-01,  7.0364e-01, -5.6475e-01,
          9.9713e-01, -4.6605e-01, -9.5852e-01,  3.6494e-01, -6.1851e-02,
         -9.8850e-01,  1.2088e-01,  1.8488e-01, -4.5003e-01, -4.3713e-01,
         -4.3971e-01, -9.6328e-01,  9.0248e-01,  1.4709e-01,  9.9092e-01,
          5.7188e-02, -9.3378e-01, -3.1652e-01, -9.2534e-01, -8.0443e-02,
         -2.1560e-01,  6.4397e-01, -9.1586e-02, -9.4833e-01,  4.7442e-01,
          5.7476e-01,  3.3297e-01,  3.8941e-01,  9.9658e-01,  9.9985e-01,
          9.7776e-01,  8.7411e-01,  8.7804e-01, -9.6168e-01, -1.2054e-01,
          9.9997e-01, -6.6824e-01, -1.0000e+00, -9.5125e-01, -5.6642e-01,
          4.1273e-01, -1.0000e+00, -1.6136e-01, -3.4676e-02, -9.1901e-01,
         -3.1622e-01,  9.8318e-01,  9.9124e-01, -1.0000e+00,  8.9389e-01,
          9.4346e-01, -5.0858e-01,  2.4580e-01, -2.3135e-01,  9.7547e-01,
          4.2250e-01,  3.7753e-01, -2.2546e-01,  3.7723e-01, -1.3091e-01,
         -8.7157e-01,  2.3319e-01,  2.5093e-01,  8.2724e-01,  1.5588e-01,
         -7.3930e-01, -9.3200e-01, -1.2279e-01, -6.6587e-02, -2.5732e-01,
         -9.6035e-01, -1.6951e-01, -3.5703e-01,  6.1311e-01,  2.4599e-01,
          2.3456e-01, -7.9384e-01,  2.7844e-01, -5.1939e-01,  4.3604e-01,
          5.1201e-01, -9.2245e-01, -6.2274e-01, -2.1160e-01, -4.7518e-01,
          2.7232e-01, -9.6657e-01,  9.7142e-01, -2.9870e-01,  2.4310e-01,
          1.0000e+00, -9.2202e-02, -8.8537e-01,  3.1929e-01,  1.6034e-01,
         -3.4469e-01,  1.0000e+00,  3.9171e-01, -9.8495e-01, -3.9130e-01,
          2.0869e-01, -3.9736e-01, -4.2046e-01,  9.9881e-01, -2.3887e-01,
          2.8045e-01,  2.6567e-01,  9.7683e-01, -9.9247e-01,  6.3824e-01,
         -9.0147e-01, -9.5820e-01,  9.5663e-01,  9.3855e-01, -1.4730e-01,
         -7.2889e-01,  1.4520e-01, -1.8675e-01,  2.6300e-01, -9.6400e-01,
          5.8518e-01,  4.4442e-01, -9.6464e-02,  8.8574e-01, -8.8098e-01,
         -3.9014e-01,  4.1658e-01,  9.9770e-02,  4.1451e-01,  2.6072e-01,
          4.5863e-01, -3.4371e-01,  1.0964e-01, -2.7387e-01, -8.9248e-02,
         -9.6777e-01,  4.3397e-02,  1.0000e+00,  1.2981e-01, -3.4366e-01,
         -4.5056e-02, -9.4596e-02, -2.1016e-01,  3.5447e-01,  5.0661e-01,
         -3.0578e-01, -8.1335e-01,  6.9142e-02, -9.1946e-01, -9.8745e-01,
          7.4571e-01,  1.8653e-01, -3.5182e-01,  9.9974e-01,  2.4423e-01,
          1.8763e-01, -7.2386e-02,  3.4985e-01,  1.0746e-01,  5.1677e-01,
         -4.9051e-01,  9.7835e-01, -3.0722e-01,  3.8846e-01,  8.6099e-01,
          1.8453e-01, -3.9804e-01, -6.3625e-01,  9.1733e-03, -9.4351e-01,
         -5.8535e-02, -9.6325e-01,  9.6869e-01, -1.2770e-01,  3.2308e-01,
          2.0592e-01,  1.4773e-01,  1.0000e+00,  2.8664e-01,  6.8401e-01,
         -6.8457e-01,  8.7746e-01, -9.4684e-01, -7.9937e-01, -3.8151e-01,
         -4.8727e-02,  4.4213e-01, -2.3993e-01,  2.1252e-01, -9.7509e-01,
         -1.9764e-01, -6.7608e-02, -9.7805e-01, -9.8934e-01,  4.5225e-01,
          7.6899e-01,  1.1139e-01, -6.8287e-01, -5.6328e-01, -5.9391e-01,
          2.5473e-01, -2.1508e-01, -9.2927e-01,  6.3278e-01, -3.2913e-01,
          4.2842e-01, -3.1567e-01,  4.6466e-01, -2.1445e-01,  7.9070e-01,
          1.9876e-01,  1.7233e-01, -1.2041e-01, -8.2787e-01,  7.1979e-01,
         -8.0239e-01,  1.0820e-01, -1.7385e-01,  1.0000e+00, -4.9901e-01,
         -3.6784e-02,  7.7607e-01,  7.4679e-01, -1.9120e-01,  1.9722e-01,
          1.9967e-01,  2.1493e-01,  3.5653e-01,  2.5057e-01, -8.0337e-01,
         -2.9930e-01,  5.6660e-01, -4.0009e-01, -2.1291e-01,  8.1289e-01,
          2.1814e-01,  8.7318e-02, -9.9111e-02,  1.8116e-01,  9.9893e-01,
         -1.7561e-01, -1.1083e-01, -5.9985e-01, -1.1718e-01, -3.0548e-01,
         -5.8867e-01,  1.0000e+00,  3.5209e-01, -1.9990e-01, -9.9180e-01,
         -1.8034e-02, -9.2345e-01,  9.9952e-01,  8.0807e-01, -8.5855e-01,
          5.7186e-01,  2.8361e-01, -1.6332e-01,  7.0452e-01, -2.8947e-01,
         -3.2616e-01,  1.7375e-01,  1.6440e-01,  9.5412e-01, -4.7137e-01,
         -9.6928e-01, -6.5504e-01,  3.6296e-01, -9.5361e-01,  9.7173e-01,
         -5.8073e-01, -1.9150e-01, -3.3605e-01,  3.5247e-01,  8.7008e-01,
         -2.1333e-03, -9.7685e-01, -1.8092e-01,  6.1657e-02,  9.7678e-01,
          2.7418e-01, -4.2944e-01, -9.5711e-01, -1.8267e-01,  6.5512e-02,
          3.0961e-01, -9.1480e-01,  9.7564e-01, -9.7270e-01,  2.6567e-01,
          1.0000e+00,  3.8786e-01, -6.4924e-01,  1.9543e-01, -4.9142e-01,
          2.3787e-01,  8.9131e-02,  5.6665e-01, -9.4914e-01, -2.9186e-01,
         -2.7113e-01,  2.6839e-01, -1.8699e-01,  3.7806e-01,  6.6034e-01,
          2.5334e-01, -3.5623e-01, -5.3300e-01, -2.1946e-01,  3.8268e-01,
          7.2743e-01, -2.6907e-01, -1.9909e-01,  1.3403e-01, -2.0919e-01,
         -9.0321e-01, -2.7320e-01, -3.6081e-01, -9.9466e-01,  6.8170e-01,
         -1.0000e+00, -1.3583e-01, -5.5586e-01, -2.3915e-01,  8.6088e-01,
          3.7196e-02,  5.7585e-02, -7.7021e-01,  3.6318e-01,  8.1365e-01,
          7.2954e-01, -2.8529e-01,  1.7030e-01, -7.6105e-01,  1.7249e-01,
         -1.9593e-01,  2.6639e-01,  1.1146e-01,  7.0965e-01, -1.9811e-01,
          1.0000e+00,  1.0188e-01, -5.4220e-01, -9.7256e-01,  3.0447e-01,
         -2.6452e-01,  9.9995e-01, -9.3875e-01, -9.5903e-01,  2.8526e-01,
         -6.0464e-01, -8.2965e-01,  2.4211e-01,  1.2796e-01, -6.8806e-01,
         -3.7915e-01,  9.5529e-01,  8.6700e-01, -3.4978e-01,  2.9469e-01,
         -3.8873e-01, -4.3963e-01,  8.3753e-02, -2.6750e-01,  9.8786e-01,
          2.4844e-01,  9.2651e-01,  6.6386e-01, -2.5228e-02,  9.6676e-01,
          3.2368e-01,  6.5488e-01,  1.0813e-01,  1.0000e+00,  3.6666e-01,
         -9.5177e-01,  2.3646e-01, -9.8821e-01, -2.6993e-01, -9.5921e-01,
          2.2923e-01,  1.7226e-01,  9.1098e-01, -2.9949e-01,  9.6250e-01,
          2.4218e-01,  1.3680e-01,  1.6822e-01,  5.4578e-01,  3.2755e-01,
         -9.3052e-01, -9.8844e-01, -9.8757e-01,  3.3784e-01, -4.5782e-01,
         -6.9121e-02,  3.6121e-01,  1.9176e-01,  3.9072e-01,  3.6573e-01,
         -1.0000e+00,  9.3469e-01,  4.4213e-01, -1.4691e-01,  9.6524e-01,
          1.3485e-02,  2.9751e-01,  2.4334e-01, -9.8886e-01, -9.6670e-01,
         -3.9043e-01, -3.6449e-01,  8.1793e-01,  6.7040e-01,  8.5270e-01,
          3.2083e-01, -4.9832e-01, -2.7567e-01,  4.0584e-01, -2.4154e-01,
         -9.9213e-01,  4.2448e-01,  2.6829e-01, -9.7171e-01,  9.6098e-01,
         -4.4311e-01, -2.2641e-01,  6.6900e-01,  6.8819e-02,  9.4393e-01,
          7.6540e-01,  5.9071e-01,  1.1600e-01,  6.0003e-01,  8.7642e-01,
          9.5714e-01,  9.8856e-01,  1.3229e-01,  7.8398e-01,  3.2535e-01,
          4.0681e-01,  3.5011e-01, -9.3994e-01,  2.2100e-01,  1.2674e-01,
         -1.6419e-01,  2.9378e-01, -2.3917e-01, -9.7171e-01,  3.4781e-01,
         -2.8501e-01,  5.4948e-01, -4.0438e-01,  7.0494e-02, -4.3903e-01,
         -2.3478e-01, -7.8532e-01, -5.0934e-01,  5.0192e-01,  4.1413e-01,
          9.2632e-01,  3.0985e-01, -1.4323e-01, -6.4190e-01, -1.6080e-01,
          3.1866e-01, -9.2836e-01,  9.2523e-01, -1.3718e-02,  4.0830e-01,
         -1.7649e-02, -5.0922e-04,  5.2501e-01, -3.0525e-01, -3.6783e-01,
         -2.6349e-01, -8.0582e-01,  8.1521e-01, -7.9358e-02, -4.9387e-01,
         -4.9402e-01,  5.9927e-01,  3.1836e-01,  9.9176e-01,  2.0806e-01,
          2.6599e-01, -1.2025e-01, -2.1694e-01,  3.5750e-01, -2.4472e-01,
         -1.0000e+00,  4.2622e-01,  2.0912e-01, -8.1241e-02,  7.5354e-02,
         -2.0835e-01,  1.8585e-01, -9.7545e-01, -1.5719e-01,  1.4028e-01,
         -1.8040e-01, -5.1406e-01, -3.6387e-01,  3.6267e-01,  5.9451e-01,
          3.2176e-01,  9.0730e-01, -4.6973e-02,  5.9712e-01,  4.5915e-01,
         -5.8261e-02, -6.2097e-01,  9.1518e-01]], grad_fn=<TanhBackward>))
>>> 
```

It seems that it was a problem with pytorch. Upgrading to 1.3 solve the problem. Thanks @TheEdoardo93