RuntimeError: CUDA error: device-side assert triggered when using Roberta
Author: Strideradu
Pull request: no
Labels: name: wontfix

## 🐛 Bug

<!-- Important information -->

Model I am using (Bert, XLNet....):
Roberta
Language I am using the model on (English, Chinese....):
English
The problem arise when using:
* [ ] the official example scripts: (give details)
* [x] my own modified scripts: (give details)

```
class QuestModel(nn.Module):
    def __init__(self, n_classes=30):
        super(QuestModel, self).__init__()
        self.model_name = 'QuestModel'
        self.bert_model = models[MODEL_NAME].from_pretrained(MODEL_NAME)
        self.fc = nn.Linear(LINEAR_LAYER[MODEL_NAME], n_classes)
        self.dropout = nn.Dropout(p=0.2)

    def forward(self, ids, seg_ids):
        attention_mask = (ids > 0).float()
        layers, pool_out = self.bert_model(input_ids=ids, token_type_ids=seg_ids, attention_mask=attention_mask)

        out = self.dropout(pool_out)
        logit = self.fc(out)
        return logit

......
for i, (ids, seg_ids, labels) in enumerate(data_loader):
            ids, seg_ids, labels = ids.cuda(), seg_ids.cuda(), labels.cuda()
            outputs = data_parallel(model, (ids, seg_ids))
            scores = torch.sigmoid(outputs)
            loss = loss_fn(outputs, labels)
            # loss = custom_loss(pred, y_batch.to(device))
            preds.append(outputs.cpu().numpy())
            original.append(labels.cpu().numpy())

            avg_loss += loss.item() / len(data_loader)
......
```

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details)
Google quest QA label


## To Reproduce

Steps to reproduce the behavior:

1. I have a script that worked fine when I use bert from transformers package
2. Then I change tokenizer and model to roberta
3. Always got the following errors

<!-- If you have a code sample, error messages, stack traces, please provide it here as well. -->

```
/opt/conda/conda-bld/pytorch_1565272279342/work/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = 2, SrcDim = 2, IdxDim = -2, IndexIsMajor = true]: block: [130,0,0], thread: [95,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
Traceback (most recent call last):
  File "/mnt/home/dunan/Learn/Kaggle/google_quest/kaggle_google_quest/train.py", line 267, in <module>
    train(args)
  File "/mnt/home/dunan/Learn/Kaggle/google_quest/kaggle_google_quest/train.py", line 138, in train
    score, val_loss = predict(model, val_loader)
  File "/mnt/home/dunan/Learn/Kaggle/google_quest/kaggle_google_quest/train.py", line 252, in predict
    preds.append(outputs.cpu().numpy())
RuntimeError: CUDA error: device-side assert triggered
```

<!-- A clear and concise description of what you expected to happen. -->

## Environment

* OS: CentOS
* Python version: 3.6
* PyTorch version: 1.2
* PyTorch Transformers version (or branch): 2.2.2
* Using GPU ? Yes
* Distributed of parallel setup ? All happened when I using 1, 2, 4 Gpu
* Any other relevant information:

## Additional context

<!-- Add any other context about the problem here. -->


Comments:
Bert, XLnet all work fine for me

Have you ever read in the Issues section, e.g. #1852, #1848, #1849 and #1805? They suggest different solutions for your problem, e.g. changing the input sequence limit to 128.

> ##  Bug
> Model I am using (Bert, XLNet....):
> Roberta
> Language I am using the model on (English, Chinese....):
> English
> The problem arise when using:
> 
> * [ ]  the official example scripts: (give details)
> * [x]  my own modified scripts: (give details)
> 
> ```
> class QuestModel(nn.Module):
>     def __init__(self, n_classes=30):
>         super(QuestModel, self).__init__()
>         self.model_name = 'QuestModel'
>         self.bert_model = models[MODEL_NAME].from_pretrained(MODEL_NAME)
>         self.fc = nn.Linear(LINEAR_LAYER[MODEL_NAME], n_classes)
>         self.dropout = nn.Dropout(p=0.2)
> 
>     def forward(self, ids, seg_ids):
>         attention_mask = (ids > 0).float()
>         layers, pool_out = self.bert_model(input_ids=ids, token_type_ids=seg_ids, attention_mask=attention_mask)
> 
>         out = self.dropout(pool_out)
>         logit = self.fc(out)
>         return logit
> 
> ......
> for i, (ids, seg_ids, labels) in enumerate(data_loader):
>             ids, seg_ids, labels = ids.cuda(), seg_ids.cuda(), labels.cuda()
>             outputs = data_parallel(model, (ids, seg_ids))
>             scores = torch.sigmoid(outputs)
>             loss = loss_fn(outputs, labels)
>             # loss = custom_loss(pred, y_batch.to(device))
>             preds.append(outputs.cpu().numpy())
>             original.append(labels.cpu().numpy())
> 
>             avg_loss += loss.item() / len(data_loader)
> ......
> ```
> 
> The tasks I am working on is:
> 
> * [ ]  an official GLUE/SQUaD task: (give the name)
> * [x]  my own task or dataset: (give details)
>   Google quest QA label
> 
> ## To Reproduce
> Steps to reproduce the behavior:
> 
> 1. I have a script that worked fine when I use bert from transformers package
> 2. Then I change tokenizer and model to roberta
> 3. Always got the following errors
> 
> ```
> /opt/conda/conda-bld/pytorch_1565272279342/work/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = 2, SrcDim = 2, IdxDim = -2, IndexIsMajor = true]: block: [130,0,0], thread: [95,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
> Traceback (most recent call last):
>   File "/mnt/home/dunan/Learn/Kaggle/google_quest/kaggle_google_quest/train.py", line 267, in <module>
>     train(args)
>   File "/mnt/home/dunan/Learn/Kaggle/google_quest/kaggle_google_quest/train.py", line 138, in train
>     score, val_loss = predict(model, val_loader)
>   File "/mnt/home/dunan/Learn/Kaggle/google_quest/kaggle_google_quest/train.py", line 252, in predict
>     preds.append(outputs.cpu().numpy())
> RuntimeError: CUDA error: device-side assert triggered
> ```
> 
> ## Environment
> * OS: CentOS
> * Python version: 3.6
> * PyTorch version: 1.2
> * PyTorch Transformers version (or branch): 2.2.2
> * Using GPU ? Yes
> * Distributed of parallel setup ? All happened when I using 1, 2, 4 Gpu
> * Any other relevant information:
> 
> ## Additional context

Currently, roberta model has a problem when handling multiple sentences.
Just removing token_type_ids from input or a bit modification will work.
(https://github.com/huggingface/transformers/issues/1538)

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.