ProjectedAdaptiveLogSoftmax log_prob computation dimensions error
Author: tonyhqanguyen
Pull request: no
Labels: name: wontfix

## 🐛 Bug

<!-- Important information -->

Model I am using (Bert, XLNet....): TransformerXL

The problem arise when using:
* [x] the official example scripts: (give details)
* [ ] my own modified scripts: (give details)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details)

## To Reproduce

Steps to reproduce the behavior:
```python
from pytorch_transformers.modeling_transfo_xl_utilities import ProjectedAdaptiveLogSoftmax
import torch
s = ProjectedAdaptiveLogSoftmax(10000, 8, 8, [1000, 2000, 8000])
outputs = torch.randn(5, 3, 8)
outputs = outputs.view(-1, outputs.size(-1))
log_prob = s.log_prob(outputs)
```

Error:

> Traceback (most recent call last):
  File "<input>", line 5, in <module>
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pytorch_transformers/modeling_transfo_xl_utilities.py", line 254, in log_prob
    logprob_i = head_logprob[:, -i] + tail_logprob_i
RuntimeError: The size of tensor a (15) must match the size of tensor b (1000) at non-singleton dimension 1

I think the code should be:

```python
    def log_prob(self, hidden):
        r""" Computes log probabilities for all :math:`n\_classes`
        From: https://github.com/pytorch/pytorch/blob/master/torch/nn/modules/adaptive.py
        Args:
            hidden (Tensor): a minibatch of examples
        Returns:
            log-probabilities of for each class :math:`c`
            in range :math:`0 <= c <= n\_classes`, where :math:`n\_classes` is a
            parameter passed to ``AdaptiveLogSoftmaxWithLoss`` constructor.
        Shape:
            - Input: :math:`(N, in\_features)`
            - Output: :math:`(N, n\_classes)`
        """
        if self.n_clusters == 0:
            logit = self._compute_logit(hidden, self.out_layers[0].weight,
                                        self.out_layers[0].bias, self.out_projs[0])
            return F.log_softmax(logit, dim=-1)
        else:
            # construct weights and biases
            weights, biases = [], []
            for i in range(len(self.cutoffs)):
                if self.div_val == 1:
                    l_idx, r_idx = self.cutoff_ends[i], self.cutoff_ends[i + 1]
                    weight_i = self.out_layers[0].weight[l_idx:r_idx]
                    bias_i = self.out_layers[0].bias[l_idx:r_idx]
                else:
                    weight_i = self.out_layers[i].weight
                    bias_i = self.out_layers[i].bias

                if i == 0:
                    weight_i = torch.cat(
                        [weight_i, self.cluster_weight], dim=0)
                    bias_i = torch.cat(
                        [bias_i, self.cluster_bias], dim=0)

                weights.append(weight_i)
                biases.append(bias_i)

            head_weight, head_bias, head_proj = weights[0], biases[0], self.out_projs[0]
            head_logit = self._compute_logit(hidden, head_weight, head_bias, head_proj)

            out = hidden.new_empty((head_logit.size(0), self.n_token))
            head_logprob = F.log_softmax(head_logit, dim=1)

            cutoff_values = [0] + self.cutoffs
            for i in range(len(cutoff_values) - 1):
                start_idx, stop_idx = cutoff_values[i], cutoff_values[i + 1]

                if i == 0:
                    out[:, :self.cutoffs[0]] = head_logprob[:, :self.cutoffs[0]]
                else:
                    weight_i, bias_i, proj_i = weights[i], biases[i], self.out_projs[i]

                    tail_logit_i = self._compute_logit(hidden, weight_i, bias_i, proj_i)
                    tail_logprob_i = F.log_softmax(tail_logit_i, dim=1)
                    logprob_i = head_logprob[:, -1].unsqueeze(1) + tail_logprob_i
                    out[:, start_idx:stop_idx] = logprob_i

            return out
```

The change here is on the third to last line, you guys did `logprob_i = head_logprob[:, -1] + tail_logprob_i`. This isn't fitting in dimensions, so I think unsqueezing it will fix the problem, the class [AdaptiveLogSoftmaxWithLoss](https://pytorch.org/docs/stable/_modules/torch/nn/modules/adaptive.html) had to unsqueeze the `head_logprob`. 

Another problem that I ran into is that, with the original code, your second to last line is 
`out[:, start_idx, stop_idx] = logprob_i`, but `out` only has 2 dimensions, so I think you meant `start_idx:stop_idx` instead. Let me know if I'm wrong.

## Environment

* OS: OSX Mojave
* Python version: 3.7
* PyTorch version: 1.1.0
* PyTorch Transformers version (or branch): Master
* Using GPU ? No
* Distributed of parallel setup ? None

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.