Error when calculating loss and running backward
Author: zhongpeixiang
Pull request: no

I'm using the sentence classification example. I used my own dataset for emotionclassification (4 classes).

The hyper-parameters are as follows:
<pre>
args.max_seq_length = 100
args.do_train = True
args.do_eval = True
args.do_lower_case = True
args.train_batch_size = 32
args.eval_batch_size = 8
args.learning_rate = 2e-5
args.num_train_epochs = 3
args.warmup_proportion = 0.1
args.no_cuda = False
args.local_rank = -1
args.gpu_id = 1
args.seed = 412
args.gradient_accumulation_steps = 1
args.optimize_on_cpu = False
args.fp16 = False
args.loss_scale = 128
</pre>

I prepared my dataset accordingly and properly:
<pre>
12/04/2018 21:23:02 - INFO - __main__ -   *** Example ***
12/04/2018 21:23:02 - INFO - __main__ -   guid: train-1
12/04/2018 21:23:02 - INFO - __main__ -   tokens: [CLS] but i don ' t [ sep ] u just did [ sep ] i don ##t want to talk to u [SEP]
12/04/2018 21:23:02 - INFO - __main__ -   input_ids: 101 2021 1045 2123 1005 1056 1031 19802 1033 1057 2074 2106 1031 19802 1033 1045 2123 2102 2215 2000 2831 2000 1057 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12/04/2018 21:23:02 - INFO - __main__ -   input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12/04/2018 21:23:02 - INFO - __main__ -   segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12/04/2018 21:23:02 - INFO - __main__ -   label: angry (id = 3)
</pre>

When I run the following code, a runtime error occurred:

<pre>
for step, batch in enumerate(tqdm(train_dataloader, desc="Iteration")):
&nbsp &nbsp batch = tuple(t.to(device) for t in batch)
&nbsp &nbsp input_ids, input_mask, segment_ids, label_ids = batch
&nbsp &nbsp loss = model(input_ids, segment_ids, input_mask, label_ids)
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-20-1977b86302ed> in <module>()
     17             try:
---> 18                 loss.backward()
     19             except RuntimeError:

/raid5/peixiang/anaconda3/lib/python3.6/site-packages/torch/tensor.py in backward(self, gradient, retain_graph, create_graph)
     92         """
---> 93         torch.autograd.backward(self, gradient, retain_graph, create_graph)
     94 

/raid5/peixiang/anaconda3/lib/python3.6/site-packages/torch/autograd/__init__.py in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables)
     89         tensors, grad_tensors, retain_graph, create_graph,
---> 90         allow_unreachable=True)  # allow_unreachable flag
     91 

RuntimeError: cublas runtime error : the GPU program failed to execute at /opt/conda/conda-bld/pytorch_1532581333611/work/aten/src/THC/THCBlas.cu:411
</pre>

What might be the cause? The dataset? I run the MRPC example without any issue.

Comments:
I probably know the bug. The final output layer is for binary classification but I use it for 4-class classification. I thought BERT can automatically decide between sigmoid and soft max. I will replace it with my own classifier tomorrow and see how it goes.

The mismatched output size between BERT and our dataset is the bug. Also, remember to set the num_labels to your output size:

<pre>
output_size = 4
model.classifier = nn.Linear(768, output_size)
model.num_labels = output_size 
</pre>