CUDA error: device-side assert triggered(pretrained_model.cuda())
Author: lipingbj
Pull request: no

## 🐛 Bug

<!-- Important information -->

Model I am using (XLNet....):

Language I am using the model on (English):

The problem arise when using:
 config = XLNetConfig.from_json_file('/data2/liping/xlnet/xlnet_cased_L-12_H-768_A-12/xlnet_config.json')
   encoder_model = XLNetModel.from_pretrained("/data2/liping/xlnet/xlnet_cased_L-12_H-768_A-12/xlnet_model.ckpt.index", config=config, from_tf=True)
   encoder_model.cuda("cuda:0")

The problem:
~/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py in _apply(self, fn)
    228                 # `with torch.no_grad():`
    229                 with torch.no_grad():
--> 230                     param_applied = fn(param)
    231                 should_use_set_data = compute_should_use_set_data(param, param_applied)
    232                 if should_use_set_data:

~/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py in <lambda>(t)
    309             Module: self
    310         """
--> 311         return self._apply(lambda t: t.cuda(device))
    312 
    313     def cpu(self):

RuntimeError: CUDA error: device-side assert triggered

Comments:
Hello! Is your checkpoint the original one from the XLNet repository or one of our TensorFlow checkpoints hosted on S3?

> Hello! Is your checkpoint the original one from the XLNet repository or one of our TensorFlow checkpoints hosted on S3?

The checkpoint is from the XLNet repository.

Could you then convert it to one a checkpoint readable by our models by using the script [convert_xlnet_original_tf_checkpoint_to_pytorch.py](https://github.com/huggingface/transformers/blob/master/transformers/convert_xlnet_original_tf_checkpoint_to_pytorch.py)?

> Could you then convert it to one a checkpoint readable by our models by using the script [convert_xlnet_original_tf_checkpoint_to_pytorch.py](https://github.com/huggingface/transformers/blob/master/transformers/convert_xlnet_original_tf_checkpoint_to_pytorch.py)?

I have tried with the script, but the problem is still exiting.
encoder_model = XLNetModel.from_pretrained("/data2/liping/xlnet/produce/")
    encoder_model.cuda()

-> 230                     param_applied = fn(param)
    231                 should_use_set_data = compute_should_use_set_data(param, param_applied)
    232                 if should_use_set_data:

~/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py in <lambda>(t)
    309             Module: self
    310         """
--> 311         return self._apply(lambda t: t.cuda(device))
    312 
    313     def cpu(self):

RuntimeError: CUDA error: device-side assert triggered

What happens once you have converted the original checkpoint to PyTorch? What is inside the folder "/data2/liping/xlnet/produce/" ?

> What happens once you have converted the original checkpoint to PyTorch? What is inside the folder "/data2/liping/xlnet/produce/" ?

Thank you for your help and i have convert the original checkpoint to PyTorch and load the xlnet pre-training model successful.

@lipingbj  Good to hear that you've fixed the problem. i just met the same problem when i use run_lm_finetuning.py and when i try convert with convert_bertabs_original_pytorch_checkpoint.py it just returned no module named'model_bertabs'. By the way i put the pytorch_model.bin that i have trained before in convert file ,is that right and how you fix the problem.Will be really appreciate for  your reply!