RuntimeError: Trying to create tensor with negative dimension
Author: davidliujiafeng
Pull request: no

## Environment info
- `transformers` version: 3.4.0
- Platform: Linux-3.10.0-957.el7.x86_64-x86_64-with-debian-stretch-sid
- Python version: 3.6.9
- PyTorch version (GPU?): 1.6.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No
 @TevenLeScao 

## Information
I am using TransfoXLModel. The problem arises when running the code below (if I do not fill in vocab_size=256, it works fine):
* the example scripts: 
```python
from transformers import TransfoXLConfig, TransfoXLModel
configuration = TransfoXLConfig(vocab_size=256)
model = TransfoXLModel(configuration)
```

## Error I get:
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-323-7039580347ad> in <module>
      3 configuration = TransfoXLConfig(vocab_size=256)
      4 # Initializing a model from the configuration
----> 5 model = TransfoXLModel(configuration)

/opt/conda/lib/python3.6/site-packages/transformers/modeling_transfo_xl.py in __init__(self, config)
    736 
    737         self.word_emb = AdaptiveEmbedding(
--> 738             config.vocab_size, config.d_embed, config.d_model, config.cutoffs, div_val=config.div_val
    739         )
    740 

/opt/conda/lib/python3.6/site-packages/transformers/modeling_transfo_xl.py in __init__(self, n_token, d_embed, d_proj, cutoffs, div_val, sample_softmax)
    421                 l_idx, r_idx = self.cutoff_ends[i], self.cutoff_ends[i + 1]
    422                 d_emb_i = d_embed // (div_val ** i)
--> 423                 self.emb_layers.append(nn.Embedding(r_idx - l_idx, d_emb_i))
    424                 self.emb_projs.append(nn.Parameter(torch.FloatTensor(d_proj, d_emb_i)))
    425 

/opt/conda/lib/python3.6/site-packages/torch/nn/modules/sparse.py in __init__(self, num_embeddings, embedding_dim, padding_idx, max_norm, norm_type, scale_grad_by_freq, sparse, _weight)
    107         self.scale_grad_by_freq = scale_grad_by_freq
    108         if _weight is None:
--> 109             self.weight = Parameter(torch.Tensor(num_embeddings, embedding_dim))
    110             self.reset_parameters()
    111         else:

RuntimeError: Trying to create tensor with negative dimension -199744: [-199744, 16]


Comments:
I'm experiencing the same issue. Setting `vocab_size=tokenizer.vocab_size` does not help.

I've noticed if I artificially inflate the vocab_size (i.e. `vocab_size=tokenizer.vocab_size+1`), the negative dimension in the error will also be greater by 1 (ie `-199743` instead of `-199744`.

Hey! From your description it sounds like you haven't changed the cutoff points for adaptive embeddings. (the different sizes of the clusters for the hierarchical softmax generation). This causes an issue as the last cluster of embeddings, the one for the least frequent words, has size `vocab_size - cutoffs[-1]` so if the last cutoff is bigger than the vocab size, that's negative.

Now for only 256 vocab words, adaptive embeddings don't really matter anyway, so I'd recommend running

```
from transformers import TransfoXLConfig, TransfoXLModel
configuration = TransfoXLConfig(vocab_size=256, cutoffs=[])
model = TransfoXLModel(configuration)
```

This worked for me, thanks a lot @TevenLeScao ! If you had a larger vocab size would you just recommend setting the last cutoff to be `0 < cutoff < vocab_size`?


Ah actually I re-read the code and docs and the two ends of the cutoffs are already provided; they're appended later, so what you want is actually `cutoffs=[]`, even if `cutoffs=[0, 256]` seems to work anyway (I've edited my previous answer).

In any case, to answer your question, yes, for a larger vocab size it is actually quite helpful to have `0 < cutoff < vocab_size`! Personally I start considering doing that for a vocabulary on the order of a few tens of thousands - so like 40000 for example, but your mileage may vary, I recommend experimenting yourself and checking whether it makes a difference :) it should mostly help with memory use.

Awesome, thanks for the helpful advice! I'd posted about the same issue on https://discuss.huggingface.co/t/transfoxllmheadmodel-trying-to-create-tensor-with-negative-dimension-199500/1768/2 but it remained unanswered, so I linked your comment in that thread as a solution.

@TevenLeScao Thanks very much, it works great for me, close the issue now.

RuntimeError: Trying to create tensor with negative dimension -128: [128, -128]
I am using Pytorch