Keep getting the same `Target 1 is out of bounds` error with `LongformerForMultipleChoice`
Author: h56cho
Pull request: no

Hello,
I am a Transformer user who posted the similar question yesterday.
I am trying to use `LongformerForMultipleChoice` model, I've updated my code according to the answer that was provided. However, I am still getting the same error `Target out of bounds`. The correct answers are coded correctly, like my multiple-choice answers works very well with the `GPT2DoubleHeadsModel`. I am not sure why I am keep getting this error:
```python
# import the pre-trained HuggingFace Longformer tokenizer.
longformer_tokenizer = LongformerTokenizer.from_pretrained('allenai/longformer-base-4096')

# get the pre-trained HuggingFace Longformer
best_model_longformer = LongformerForMultipleChoice.from_pretrained('allenai/longformer-base-4096',
                                                         output_hidden_states = True)

# my multiple choice question has 4 options.
question_list = [main_question, main_question, main_question, main_question]
options_list = [option1, option2, option3, option4]
# unsqueeze the answer
mc_labels = torch.tensor(my_answer).unsqueeze(0)

encoded_dict = longformer_tokenizer(question_list, options_list,
                             return_tensors = 'pt',
                             add_prefix_space = True,
                             padding = True)
input_hidden_state = best_model_longformer(
                       **{k: v.unsqueeze(0) for k,v in encoded_dict.items()}, 
                        labels = mc_labels, return_dict=True)[2][0][:,:,:].detach()
```

and I am getting the error below:
```
Traceback (most recent call last):
  File "SEED_125_V20_15_LONGFORMER.py", line 427, in <module>
    main_function('/home/ec2-user/G1G2.txt','/home/ec2-user/G1G2_answer_num.txt', num_iter)
  File "SEED_125_V20_15_LONGFORMER.py", line 389, in main_function
    best_model_longformer)
  File "SEED_125_V20_15_LONGFORMER.py", line 198, in fill_MC_loss_accuracy_tensor
    input_hidden_state = best_model_longformer(**{k: v.unsqueeze(0) for k,v in encoded_dict.items()}, labels = mc_labels, return_dict = True)[2][0][:,:,:].detach()
  File "/home/ec2-user/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/ec2-user/anaconda3/lib/python3.7/site-packages/transformers/modeling_longformer.py", line 1808, in forward
    loss = loss_fct(reshaped_logits, labels)
  File "/home/ec2-user/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/ec2-user/anaconda3/lib/python3.7/site-packages/torch/nn/modules/loss.py", line 948, in forward
    ignore_index=self.ignore_index, reduction=self.reduction)
  File "/home/ec2-user/anaconda3/lib/python3.7/site-packages/torch/nn/functional.py", line 2422, in cross_entropy
    return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
  File "/home/ec2-user/anaconda3/lib/python3.7/site-packages/torch/nn/functional.py", line 2218, in nll_loss
    ret = torch._C._nn.nll_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
IndexError: Target 1 is out of bounds.
```
When I do instead `mc_labels = torch.tensor([my_answer]).unsqueeze(0)` (note the square brackets around `my_answer`), another error occurs, the error is something like `cannot process multiple answers`.

How can I solve this issue?

Thank you,

Comments:
Hey there. Please just reply on the previous issue you posted, rather than opening a new issue. 

I made a mistake in my previous answer, you shouldn't `unsqueeze` the answer, because it's just a tensor of shape (batch_size,).

I've created a notebook that illustrates how to use BertForMultipleChoice (LongformerForMultipleChoice would be the same): https://colab.research.google.com/drive/1mWx3R7-1lPldJqH26d3fnoyZX6Qa4IpV?usp=sharing