Cubla Error on DistilBert
Author: Ricocotam
Pull request: no
Labels: name: wontfix

# 🐛 Bug
When using DistilBert I get `CUBLAS_STATUS_ALLOC_FAILED` when trying to forward

## Information
Exact error is : `RuntimeError : Cuda Error: CUBLAS_STATUS_ALLOC_FAILED when calling cublasCreate(handle)` and traceback indicates i happens on `output = input.matmul(weight.t())` which is probably not informative but the whole stack is filled with forward on a transformer.

I'm using distilbert-base-multilingual-cased with French

## Environment info
- `transformers` version: 2.5.1
- Platform: Linux-5.4.0-3amd-64-x89_64-with-debian-bullseye-sid
- Python version: 3.7.0
- PyTorch version (GPU?): 1.4.0 (True)
- Tensorflow version (GPU?): 2.1.0 (False)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no


Comments:
After some investigation, I have really no clue on what's happening. This issue is only referred through Tensorflow questions such as [this](https://stackoverflow.com/questions/41117740/tensorflow-crashes-with-cublas-status-alloc-failed) or [this issue](https://github.com/tensorflow/tensorflow/issues/9489) on Tensorflow's Github

I've got the same issue using the same environment as you. 
I'm using model `"mrm8488/distill-bert-base-spanish-wwm-cased-finetuned-spa-squad2-es"`
 My error though is at:
```
RuntimeError                              Traceback (most recent call last)
<ipython-input-17-90ab5d6d4393> in <module>
     13         outputs = model(**inputs, labels=labels)
     14         loss, logits = outputs[:2]
---> 15         loss.backward()
     16         
     17 
/opt/conda/lib/python3.7/site-packages/torch/tensor.py in backward(self, gradient, retain_graph, create_graph)
    193                 products. Defaults to ``False``.
    194         """
--> 195         torch.autograd.backward(self, gradient, retain_graph, create_graph)
    196 
    197     def register_hook(self, hook):

/opt/conda/lib/python3.7/site-packages/torch/autograd/__init__.py in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables)
     97     Variable._execution_engine.run_backward(
     98         tensors, grad_tensors, retain_graph, create_graph,
---> 99         allow_unreachable=True)  # allow_unreachable flag
    100 
    101 
RuntimeError: CUDA error: CUBLAS_STATUS_ALLOC_FAILED when calling `cublasCreate(handle)`
```
I've tried most stuff I've found but really nothing seems to work.

Find similar issue when using regular BERTModel

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Hi, @Ricocotam and @j6e 
I got the same error with torch 1.6.0 GPU with a DistilBert Classification error. Would you advise if and how you fix that?

So, I encountered the same issue

This might sound stupid, but it happened because I was using the wrong tokenizer. So, for future readers.. check that you really use an appropriate tokenizer ?

However, I agree the error message is rather obscure. I'm not sure exactly *why* it is triggered (I guess the other tokenizer would produce some IDs that do not exist in the distilbert vocabulary ?) , so I don't know if there is an easy fix for that