ElectraForQuestionAnswering with SQuADHead
Author: bkiat1123
Pull request: no

# 🚀 Feature request

<!-- -->
Implement ElectraForQuestionAnswering as described in the paper. https://arxiv.org/abs/2003.10555

## Motivation
<!-- -->
In the original implementation, the authors use question answering module from XLNet rather than simple linear layer. There is a huge gap of performance between these 2 question answering module, especially on Squad2.0 like tasks. I suggest to follow the original implementation and rename the one with simple linear layer to ElectraForQuestionAnsweringSimple. 

## Your contribution

<!--  -->
My team and I have implemented it using SQuADHead from modeling utils. I can submit a PR and make other necessary changes. 

Code example:
```
from transformers import ElectraModel, ElectraPreTrainedModel
from transformers.modeling_utils import SQuADHead

class ElectraForQuestionAnswering(ElectraPreTrainedModel):
    def __init__(self, config):
        super().__init__(config)
        self.start_n_top = config.start_n_top
        self.end_n_top = config.end_n_top

        self.electra = ElectraModel(config)
        self.squad_head = SQuADHead(config)
        self.init_weights()

    def forward(
        self,
        input_ids=None,
        attention_mask=None,
        token_type_ids=None,
        head_mask=None,
        start_positions=None,
        end_positions=None,
        is_impossible=None,
        cls_index=None,
        p_mask=None,
        return_dict=None):

        return_dict = return_dict if return_dict is not None else self.config.use_return_dict

        transformer_outputs = self.electra(
            input_ids=input_ids,
            attention_mask=attention_mask,
            token_type_ids=token_type_ids,
            head_mask=head_mask,
            return_dict=return_dict,
        )
        hidden_states = transformer_outputs[0]

        return self.squad_head(hidden_states=hidden_states,
                                             start_positions=start_positions,
                                             end_positions=end_positions,
                                             cls_index=cls_index,
                                             is_impossible=is_impossible,
                                             p_mask=p_mask,
                                             return_dict=return_dict,
                                              )
```


Comments:
Hello! We would welcome a PR that offers this. Maybe instead of renaming the current QA model to `Simple` (which would break backwards-compatibility), we could add a new model called `ElectraForQuestionAnsweringBeamSearch`? What do you think?

Agreed. We should use a new name for the model for backward-compatibility. I will submit a PR soon.

This sounds great, thanks @bkiat1123!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.