ValueError: too many values to unpack (expected 4)
Author: h56cho
Pull request: no

Hello, 
I am trying to feed in the hidden output of the embedding layer of the `LongformerForMultipleChoice` model directly into the m-th layer of the same model. Each of my multiple-choice question that has 4 options. I am trying to tweak the HuggingFace code for Longformer to carry out my task, but I am rather perplexed by the ValueError (see below) that I am getting. I know that this can be cumbersome to answer, but could you please help me?

When I do:
```Python
my_Longformer_multiple_choice_model.encoder.layer[layer_index].forward(hidden_output, 
                                                 attention_mask=my_attention_mask,output_attention=False)
```
, this ValueError is generated:
```Python

  File "<ipython-input-67-f93c8b17889e>", line 1, in <module>
    best_model_longformer.longformer.encoder.layer[0].forward(outputs,attention_mask=attention_mask,output_attentions=True)

  File "/Users/hyunjindominiquecho/opt/anaconda3/lib/python3.7/site-packages/transformers/modeling_longformer.py", line 718, in forward
    output_attentions=output_attentions,

  File "/Users/hyunjindominiquecho/opt/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)

  File "/Users/hyunjindominiquecho/opt/anaconda3/lib/python3.7/site-packages/transformers/modeling_longformer.py", line 693, in forward
    output_attentions,

  File "/Users/hyunjindominiquecho/opt/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)

  File "/Users/hyunjindominiquecho/opt/anaconda3/lib/python3.7/site-packages/transformers/modeling_longformer.py", line 192, in forward
    float_mask.new_ones(size=float_mask.size()), float_mask, self.one_sided_attn_window_size

File "/Users/hyunjindominiquecho/opt/anaconda3/lib/python3.7/site-packages/transformers/modeling_longformer.py", line 384, in _sliding_chunks_query_key_matmul
    batch_size, seq_len, num_heads, head_dim = query.size()

ValueError: too many values to unpack (expected 4)
```
In other words, the ValueError is generated here:
https://github.com/huggingface/transformers/blob/d99ed7ad618037ae878f0758157ed0764bd7f935/src/transformers/modeling_longformer.py#L279

In my code above, `my_attention_mask` is the same attention mask that I would specify under the regular `LongformerForMultipleChoice` command. `my_attention_mask` was generated by:
```Python
# I am using the LongformerForMultipleChoice model, where each multiple choice question has 4 options.
encoded_dict = longformer_tokenizer(question_list, option_list,
                             return_tensors = 'pt',
                             padding ='max_length')
my_attention_mask = {k: v.unsqueeze(0) for k,v in encoded_dict.items()}['attention_mask']
my_attention_mask 
>>> tensor([[[1, 1, 1,  ..., 0, 0, 0],
         [1, 1, 1,  ..., 0, 0, 0],
         [1, 1, 1,  ..., 0, 0, 0],
         [1, 1, 1,  ..., 0, 0, 0]]])

# I can use this my_attention_mask in the regular command without an error, as below:
longformer_output= my_Longformer_multiple_choice_model(input_ids=input_ids,....,attention_mask=my_attention_mask)
```
Also, the `hidden_output` in used in my code above was generated by the following:
```Python
encoded_dict = longformer_tokenizer(question_list, option_list,
                             return_tensors = 'pt',
                             padding ='max_length')
hidden_output = my_Longformer_multiple_choice_model(**{k: v.unsqueeze(0) for k,v in encoded_dict.items()}, labels = mc_labels)[2][0][:,:,:]
hidden_output.size()
>>> torch.Size([4, 4096, 768])
```

Since the layers of Transformer models in general are designed to take in the hidden output of the previous layer as their input, I am sure that the HuggingFace code for `LongformerForMultipleChoice` also somehow allows each layer to take the hidden vectors as their input. This is why I think what I am trying to do (feeding hidden output of embedding layer as an input to the m-th layer) is completely achievable....I do not get why I am getting the ValueError.

I am suspecting the form of `my_attention_mask` is causing the ValueError. Is there any way that I can avoid this ValueError without re-writing the whole function? What exactly is the `attention_mask` that is used as a parameter for the following function?:
https://github.com/huggingface/transformers/blob/d99ed7ad618037ae878f0758157ed0764bd7f935/src/transformers/modeling_longformer.py#L225

What should I pass for the `attention_mask` parameter in the command `my_Longformer_multiple_choice_model.encoder.layer[layer_index].forward(hidden_output, 
                                                 attention_mask,output_attention=False)`?

PS: The following seemed to suggest that I need to pass in the `extended_attention_mask` rather than the `attention_mask` itself:
        https://github.com/huggingface/transformers/blob/d99ed7ad618037ae878f0758157ed0764bd7f935/src/transformers/modeling_longformer.py#L1262

So I tried `my_Longformer_multiple_choice_model.encoder.layer[layer_index].forward(hidden_output, 
                                                 extended_attention_mask,output_attention=False)`, but I am still getting the same ValueError....

Please help.
Thank you,

Comments:
I solved the issue by myself

> I solved the issue by myself

How?

> I solved the issue by myself

@h56cho Would you mind sharing it? Thanks