Better error message? `CUDA error: CUBLAS_STATUS_ALLOC_FAILED`
Author: bayartsogt-ya
Pull request: no

## Environment info
- `transformers` version: 4.9.1
- Platform: Linux-5.4.104+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.11
- PyTorch version (GPU?): 1.9.0+cu102 (True)
- Tensorflow version (GPU?): 2.5.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: yes/no
- Using distributed or parallel set-up in script?: no

### Who can help
Models:
- gpt2: @patrickvonplaten, @LysandreJik


## Information

I found that out of index in embedding is little weird when using cuda.
```
RuntimeError: CUDA error: CUBLAS_STATUS_ALLOC_FAILED when calling `cublasCreate(handle)`
```

However, in cpu, it is understandable.
```
IndexError: index out of range in self
```

I just wondered if it needs better error message, or just leave it?

## To reproduce

### To get weird CUDA error:
```python
from transformers import AutoConfig, AutoModel, AutoTokenizer
import torch
config = AutoConfig.from_pretrained("gpt2")
config.update({"output_hidden_states":True, 
                "hidden_dropout_prob": 0.0,
                "layer_norm_eps": 1e-7})    

gpt_model = AutoModel.from_pretrained('gpt2').cuda()
input_ids = torch.randint(0, 100_000, (4, 128)).cuda()
attention_mask = torch.randint(0, 1, (4, 128)).cuda()

outputs = gpt_model(input_ids=input_ids, attention_mask=attention_mask)
last_hidden_states = outputs.last_hidden_states
print(last_hidden_states.shape)
```

```
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-2-b5d926c8a3c3> in <module>()
     10 attention_mask = torch.randint(0, 1, (4, 248)).cuda()
     11 
---> 12 outputs = gpt_model(input_ids=input_ids, attention_mask=attention_mask)
     13 last_hidden_states = outputs.last_hidden_states
     14 print(last_hidden_states.shape)

7 frames
/usr/local/lib/python3.7/dist-packages/transformers/modeling_utils.py in forward(self, x)
   1585     def forward(self, x):
   1586         size_out = x.size()[:-1] + (self.nf,)
-> 1587         x = torch.addmm(self.bias, x.view(-1, x.size(-1)), self.weight)
   1588         x = x.view(*size_out)
   1589         return x

RuntimeError: CUDA error: CUBLAS_STATUS_ALLOC_FAILED when calling `cublasCreate(handle)`
```


### To get cpu error:
```python
from transformers import AutoConfig, AutoModel, AutoTokenizer
import torch
config = AutoConfig.from_pretrained("gpt2")
config.update({"output_hidden_states":True, 
                "hidden_dropout_prob": 0.0,
                "layer_norm_eps": 1e-7})    

gpt_model = AutoModel.from_pretrained('gpt2')
input_ids = torch.randint(0, 100_000, (4, 128))
attention_mask = torch.randint(0, 1, (4, 128))

outputs = gpt_model(input_ids=input_ids, attention_mask=attention_mask)
last_hidden_states = outputs.last_hidden_states
print(last_hidden_states.shape)
```

```
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-3-262727302e1e> in <module>()
      9 input_ids = torch.randint(0, 100_000, (4, 248))
     10 attention_mask = torch.randint(0, 1, (4, 248))
---> 11 outputs = gpt_model(input_ids=input_ids, attention_mask=attention_mask)
     12 last_hidden_states = outputs.hidden_states
     13 print(last_hidden_states)

4 frames
/usr/local/lib/python3.7/dist-packages/torch/nn/functional.py in embedding(input, weight, padding_idx, max_norm, norm_type, scale_grad_by_freq, sparse)
   2041         # remove once script supports set_grad_enabled
   2042         _no_grad_embedding_renorm_(weight, input, max_norm, norm_type)
-> 2043     return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
   2044 
   2045 

IndexError: index out of range in self
```

Comments:
That's something we can't solve I suppose, unfortunately. If you have a CUDA error like that, it's always advised to run your code on CPU as it provides a much more informative error message.

@NielsRogge 
Thanks for the answer.

> it's always advised to run your code on CPU as it provides a much more informative error message.

Definitely agree on this. Closing this issue