XLnet output_attentions=True raises an exception
Author: roskoN
Pull request: no

## 🐛 Bug

I am working on conditional sentences probabilities based on [this code](https://github.com/huggingface/transformers/issues/917#issuecomment-525297746) and whenever `output_attentions=True` and `target_mapping` is provided, there is an exception thrown.

Model I am using (Bert, XLNet....):  XLNet ('xlnet-base-cased')

Language I am using the model on (English, Chinese....): English

The problem arise when using:
* [x] my own modified scripts: Setting `output_attentions=True`, throws an exception: `AttributeError: 'tuple' object has no attribute 'permute'`.

The tasks I am working on is:
* [x] my own task or dataset: Using just some sample text

## To Reproduce

Here is a [Google Colab notebook](https://colab.research.google.com/drive/1fkNB0Aqlhtvo3CcHWQ6IqxmCm2Qt9Etn) where the issue can be reproduced as well. Just run all cells.

**Code:**
```python
# https://github.com/huggingface/transformers/issues/917#issuecomment-525297746

import torch
from transformers import XLNetTokenizer, XLNetLMHeadModel
import numpy as np
from scipy.special import softmax

PADDING_TEXT = """In 1991, the remains of Russian Tsar Nicholas II and his family
(except for Alexei and Maria) are discovered.
The voice of Nicholas's young son, Tsarevich Alexei Nikolaevich, narrates the
remainder of the story. 1883 Western Siberia,
a young Grigori Rasputin is asked by his father and a group of men to perform magic.
Rasputin has a vision and denounces one of the men as a horse thief. Although his
father initially slaps him for making such an accusation, Rasputin watches as the
man is chased outside and beaten. Twenty years later, Rasputin sees a vision of
the Virgin Mary, prompting him to become a priest. Rasputin quickly becomes famous,
with people, even a bishop, begging for his blessing. <eod> """

text = "The dog is very cute."

tokenizer = XLNetTokenizer.from_pretrained('xlnet-base-cased')
model = XLNetLMHeadModel.from_pretrained('xlnet-base-cased', output_attentions=True)

tokenize_input = tokenizer.tokenize(PADDING_TEXT + text)
tokenize_text = tokenizer.tokenize(text)

sum_lp = 0.0
for max_word_id in range((len(tokenize_input)-len(tokenize_text)), (len(tokenize_input))):

    sent = tokenize_input[:]

    input_ids = torch.tensor([tokenizer.convert_tokens_to_ids(sent)])

    perm_mask = torch.zeros((1, input_ids.shape[1], input_ids.shape[1]), dtype=torch.float)
    perm_mask[:, :, max_word_id:] = 1.0 

    target_mapping = torch.zeros((1, 1, input_ids.shape[1]), dtype=torch.float)
    target_mapping[0, 0, max_word_id] = 1.0

    with torch.no_grad():
        next_token_logits, attentions = model(input_ids, perm_mask=perm_mask, target_mapping=target_mapping)

    word_id = tokenizer.convert_tokens_to_ids([tokenize_input[max_word_id]])[0]
    predicted_prob = softmax(np.array(next_token_logits[0][-1]))
    lp = np.log(predicted_prob[word_id])

    sum_lp += lp

print("sentence logprob =", sum_lp)
```

**Stacktrace:**
```shell
AttributeError                            Traceback (most recent call last)
<ipython-input-5-6490f5f4333c> in <module>()
     38 
     39     with torch.no_grad():
---> 40         next_token_logits, attentions = model(input_ids, perm_mask=perm_mask, target_mapping=target_mapping)
     41 
     42     word_id = tokenizer.convert_tokens_to_ids([tokenize_input[max_word_id]])[0]

4 frames
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    539             result = self._slow_forward(*input, **kwargs)
    540         else:
--> 541             result = self.forward(*input, **kwargs)
    542         for hook in self._forward_hooks.values():
    543             hook_result = hook(self, input, result)

/usr/local/lib/python3.6/dist-packages/transformers/modeling_xlnet.py in forward(self, input_ids, attention_mask, mems, perm_mask, target_mapping, token_type_ids, input_mask, head_mask, inputs_embeds, labels)
    952                                                input_mask=input_mask,
    953                                                head_mask=head_mask,
--> 954                                                inputs_embeds=inputs_embeds)
    955 
    956         logits = self.lm_loss(transformer_outputs[0])

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    539             result = self._slow_forward(*input, **kwargs)
    540         else:
--> 541             result = self.forward(*input, **kwargs)
    542         for hook in self._forward_hooks.values():
    543             hook_result = hook(self, input, result)

/usr/local/lib/python3.6/dist-packages/transformers/modeling_xlnet.py in forward(self, input_ids, attention_mask, mems, perm_mask, target_mapping, token_type_ids, input_mask, head_mask, inputs_embeds)
    879             outputs = outputs + (hidden_states,)
    880         if self.output_attentions:
--> 881             attentions = tuple(t.permute(2, 3, 0, 1).contiguous() for t in attentions)
    882             outputs = outputs + (attentions,)
    883 

/usr/local/lib/python3.6/dist-packages/transformers/modeling_xlnet.py in <genexpr>(.0)
    879             outputs = outputs + (hidden_states,)
    880         if self.output_attentions:
--> 881             attentions = tuple(t.permute(2, 3, 0, 1).contiguous() for t in attentions)
    882             outputs = outputs + (attentions,)
    883 

AttributeError: 'tuple' object has no attribute 'permute'
```

## Expected behavior

The model should output the logits for each token and the attention values across layers, heads, and tokens.

## Environment

* OS: 18.04.3 LTS (Bionic Beaver)
* Python version: 3.6.8
* PyTorch version: 1.3.1
* PyTorch Transformers version (or branch): 2.2.0
* Using GPU ? No
* Distributed of parallel setup ? N/A
* Any other relevant information:

## Additional context

<!-- Add any other context about the problem here. -->


Comments:
The issue is fixed in #2007 .