Error while runing example
Author: chledowski
Pull request: no

Hi! 

I have a problem when running the example, could you please give me a hint on what may I be doing wrong?

I use:
`PYTHONPATH=. python examples/run_classifier.py --task_name MNLI --do_train --do_eval --do_lower_case --data_dir ../GLUE-baselines/glue_data/MNLI/ --bert_model bert-base-uncased --max_seq_len 40 --train_batch_size 10 --output_dir mnli/`

And obtain:
```
...
12/03/2018 21:11:10 - INFO - __main__ -   segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0
12/03/2018 21:11:10 - INFO - __main__ -   label: entailment (id = 1)
12/03/2018 21:11:10 - INFO - __main__ -   *** Example ***
12/03/2018 21:11:10 - INFO - __main__ -   guid: train-3
12/03/2018 21:11:10 - INFO - __main__ -   tokens: [CLS] how do you know ? all this is their information again . [SEP] this information belongs to them . [SEP]
12/03/2018 21:11:10 - INFO - __main__ -   input_ids: 101 2129 2079 2017 2113 1029 2035 2023 2003 2037 2592 2153 1012 102 2023 2592 7460 2000 2068 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12/03/2018 21:11:10 - INFO - __main__ -   input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12/03/2018 21:11:10 - INFO - __main__ -   segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12/03/2018 21:11:10 - INFO - __main__ -   label: entailment (id = 1)
12/03/2018 21:11:10 - INFO - __main__ -   *** Example ***
12/03/2018 21:11:10 - INFO - __main__ -   guid: train-4
12/03/2018 21:11:10 - INFO - __main__ -   tokens: [CLS] yeah i tell you what though if you go price some of those tennis shoes i can see why now you know they ' re getting up in [SEP] the tennis shoes have a range of prices . [SEP]
12/03/2018 21:11:10 - INFO - __main__ -   input_ids: 101 3398 1045 2425 2017 2054 2295 2065 2017 2175 3976 2070 1997 2216 5093 6007 1045 2064 2156 2339 2085 2017 2113 2027 1005 2128 2893 2039 1999 102 1996 5093 6007 2031 1037 2846 1997 7597 1012 102
12/03/2018 21:11:10 - INFO - __main__ -   input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
12/03/2018 21:11:10 - INFO - __main__ -   segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1
12/03/2018 21:11:10 - INFO - __main__ -   label: neutral (id = 2)
12/03/2018 21:14:39 - INFO - __main__ -   ***** Running training *****
12/03/2018 21:14:39 - INFO - __main__ -     Num examples = 392702
12/03/2018 21:14:39 - INFO - __main__ -     Batch size = 10
12/03/2018 21:14:39 - INFO - __main__ -     Num steps = 117810
Epoch:   0%|                                                                                                                                                                          | 0/3 [00:00<?, ?it/sTHCudaCheck FAIL file=/opt/conda/conda-bld/pytorch_1533672544752/work/aten/src/THC/generic/THCTensorMath.cu line=26 error=59 : device-side assert triggered                        | 0/39271 [00:00<?, ?it/s]
/opt/conda/conda-bld/pytorch_1533672544752/work/aten/src/THCUNN/ClassNLLCriterion.cu:105: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [1,0,0] Assertion `t >= 0 && t < n_classes` failed.
/opt/conda/conda-bld/pytorch_1533672544752/work/aten/src/THCUNN/ClassNLLCriterion.cu:105: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [3,0,0] Assertion `t >= 0 && t < n_classes` failed.
/opt/conda/conda-bld/pytorch_1533672544752/work/aten/src/THCUNN/ClassNLLCriterion.cu:105: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [6,0,0] Assertion `t >= 0 && t < n_classes` failed.

Traceback (most recent call last):
  File "examples/run_classifier.py", line 637, in <module>
    main()
  File "examples/run_classifier.py", line 558, in main
    loss.backward()
  File "/home/kchledowski/anaconda2/envs/glue/lib/python3.6/site-packages/torch/tensor.py", line 93, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/home/kchledowski/anaconda2/envs/glue/lib/python3.6/site-packages/torch/autograd/__init__.py", line 90, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: cuda runtime error (59) : device-side assert triggered at /opt/conda/conda-bld/pytorch_1533672544752/work/aten/src/THC/generic/THCTensorMath.cu:26
```

I would be very grateful for any suggestions where to look. Thanks!

Comments:
Hi!

In case you haven't already, modifying the source at https://github.com/huggingface/pytorch-pretrained-BERT/blob/e60e8a606837ff7f49e583de8492e55575155eb6/examples/run_classifier.py#L491 and turning it into

`cache_dir=PYTORCH_PRETRAINED_BERT_CACHE / 'distributed_{}'.format(args.local_rank), num_labels = 3)`

should get your finetuning started (you have three labels, `["contradiction", "entailment", "neutral"]`)

Thanks, it worked. I think it could be great if 3 classes was default when choosing MNLI :)