Scrambled dimensions on output of forward pass
Author: amin3141
Pull request: no
Labels: name: wontfix

## 🐛 Bug

<!-- Important information -->

Model I am using (Bert, XLNet....): XLNet

Language I am using the model on (English, Chinese....): English

The problem arise when using:
* [ ] the official example scripts: (give details)
* [X] my own modified scripts: see attached minimum working example.

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [X] my own task or dataset: see attached minimum working example.

## To Reproduce

Steps to reproduce the behavior:

1. Run the minimal working example (see below) with the command: `python xl_mwe.py`
2. Observe the following output:
```
Embedded batch: torch.Size([3, 13, 300])
XLNet output  : torch.Size([13, 3, 300])
```
3. Per the documentation, the correct dimensions for the output should have been [3, 13, 300]. From the documentation of `last_hidden_state` in `XLNetModel.forward`: last_hidden_state (torch.FloatTensor of shape (batch_size, sequence_length, hidden_size)):

4. While constructing the minimal working example, I also observed another bug. If I change d_model to 25 (d_model = 300 in the code below) and n_heads to 5 (default is 10 in the code below), I get an error from einsum:

```
Traceback (most recent call last):
  File "xl_mwe.py", line 43, in <module>
    main()
  File "xl_mwe.py", line 37, in main
    xlnet_output = xlnet(inputs_embeds=embedded_batch)[0]
  File "C:\ProgramData\Anaconda3\envs\pytorch\lib\site-packages\torch\nn\modules\module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "C:\ProgramData\Anaconda3\envs\pytorch\lib\site-packages\transformers\modeling_xlnet.py", line 858, in forward
    head_mask=head_mask[i])
  File "C:\ProgramData\Anaconda3\envs\pytorch\lib\site-packages\torch\nn\modules\module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "C:\ProgramData\Anaconda3\envs\pytorch\lib\site-packages\transformers\modeling_xlnet.py", line 436, in forward
    head_mask=head_mask)
  File "C:\ProgramData\Anaconda3\envs\pytorch\lib\site-packages\torch\nn\modules\module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "C:\ProgramData\Anaconda3\envs\pytorch\lib\site-packages\transformers\modeling_xlnet.py", line 383, in forward
    k_head_r = torch.einsum('ibh,hnd->ibnd', r, self.r)
  File "C:\ProgramData\Anaconda3\envs\pytorch\lib\site-packages\torch\functional.py", line 202, in einsum
    return torch._C._VariableFunctions.einsum(equation, operands)
RuntimeError: size of dimension does not match previous size, operand 1, dim 0
```

## Minimal working example

```
import numpy as np
import torch
from transformers import XLNetConfig, XLNetModel, XLNetTokenizer

def embed(input_str, dims=25, fix_len=-1):
  result = []
  for word in input_str.split():
    result.append(np.random.rand(dims))
  if fix_len > -1:
    result = result[0: fix_len]
    if len(result) < fix_len:
      result = result + [np.zeros(dims)] * (fix_len - len(result))
  return result


def embed_batch(batch, dims=25, fix_len=-1):
  return np.stack([embed(x, dims, fix_len) for x in batch], axis=0)


def main():
  batch = [
      "Hello, how are you doing?",
      "Please go to the store and buy some bread.",
      "Trump was not exonerated by the Mueller report."
  ]


  d_model = 300
  config = XLNetConfig(d_model=d_model, n_head=10)
  xlnet = XLNetModel(config)

  embedded_batch = embed_batch(batch, dims=d_model, fix_len=13)
  embedded_batch = torch.from_numpy(embedded_batch).float()
  print(f"Embedded batch: {embedded_batch.shape}")

  xlnet_output = xlnet(inputs_embeds=embedded_batch)[0]
  print(f"XLNet output  : {xlnet_output.shape}")

  

if __name__ == "__main__":
  main()
```

## Environment

* OS: Windows 10
* Python version: 3.7.4
* PyTorch version:  1.2.0
* PyTorch Transformers version (or branch):  2.3.0
* Using GPU ? Yes
* Distributed or parallel setup ? No

## Additional context

<!-- Add any other context about the problem here. -->


Comments:
Hi! There was a mistake with the re-arrangement of the input embeddings inside the forward method of XLNet. I've fixed it with f09f42d.

Concerning the issue with `d_model=25` and `n_heads=5`, this is due to the model dimension being  an odd number which doesn't fare well with [`torch.arange` leveraging the model dimension to build relative positional embeddings](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_xlnet.py#L665). We should probably update this to allow for odd dimension XLNet architectures cc @thomwolf @julien-c.

Thanks for the quick fix on the the re-arrangement issue. I don't know how difficult the odd model dimension fix is. At the least, the model could throw a `ValueError` in the constructor if the dimension is odd. That would, at least, give users clear guidance.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Any updates, or do you want to close this one?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Any updates here? It's an easy fix to add a more informative error message.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.