Repository: huggingface/transformers

Issue:
Sharing Microsoft's DialogRPT (new dialog ranking model)
# 🌟 New model addition

## Model description

Thanks for the awesome work!

[DialogRPT](https://github.com/golsun/DialogRPT) (Dialog Ranking Pretrained Transformers) is a set of GPT-2 based dialogue ranking models recently released with an [EMNLP paper](https://arxiv.org/abs/2009.06978) by Microsoft Research. It's a follow-up work of [DialoGPT](https://huggingface.co/transformers/model_doc/dialogpt.html) (thanks for hosting it!)
The architecture is pretty simple: a `GPT2Model` followed by a `torch.nn.Linear(n_embd, 1, bias=False)`, and implemented based on a [previous HuggingFace commit](https://github.com/huggingface/transformers/commit/4d456542e9d381090f9a00b2bcc5a4cb07f6f3f7)
At first, I'm trying to create a model card for it, but then realized that it seems there's no existing model architecture in HuggingFace is compatible with DialogRPT. I noticed a lot of BERT-based sequence classification models, but ours is GPT-2 based. 

If there's a simple fix (or I missed something) please let me know!
If implementation in modeling_gpt2.py is necessary, I'm also glad to help!

## Open source status

* [x] the model implementation is available: (https://github.com/golsun/DialogRPT)
* [x] the model weights are available: (https://github.com/golsun/DialogRPT)
* [x] who are the authors: @golsun @dreasysnail



Discussion:
Hi @golsun! Thanks a lot for opening an issue and offering to contribute it!

Indeed, there is no `GPT2ForSequenceClassification` model in the library (yet!) I'm adding it right now with the goal of supporting DialogRPT. I'll get back to you in a bit.
Hi @golsun! `GPT2ForSequenceClassification` has been implemented on #7501 and I verified that I obtain the same results as you do on your README using your examples.

You should only need to upload your models on the model hub now! Some helpers regarding the configuration:

- You should upload a model configuration on the hub, for every model.
- You can simply copy-paste the `gpt2-medium` configuration that you can find [here](https://s3.amazonaws.com/models.huggingface.co/bert/gpt2-medium-config.json).
- You will need to add a `num_labels=1` field to these configurations.
- In the `architectures` field, you should put `GPT2ForSequenceClassification`
wow, super fast!!! 
thank you @LysandreJik , I'll update my repo to reflect this once the [pull](https://github.com/huggingface/transformers/pull/7501) is merged.


The pul request is now merged @golsun!
Thank you so much @LysandreJik !
I just tried `GPT2ForSequenceClassification` and it works! 👍 
Then I created this [model card](https://huggingface.co/microsoft/DialogRPT-updown), but `model = AutoModelForSequenceClassification.from_pretrained("microsoft/DialogRPT-updown")` gives me the following error, which can be reproduced with [this Notebook](https://colab.research.google.com/drive/1cAtfkbhqsRsT59y3imjR1APw3MHDMkuV?usp=sharing):
```
/content/transformers/src/transformers/modeling_auto.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
   1203                 config.__class__,
   1204                 cls.__name__,
-> 1205                 ", ".join(c.__name__ for c in MODEL_FOR_SEQUENCE_CLASSIFICATION_MAPPING.keys()),
   1206             )
   1207         )

ValueError: Unrecognized configuration class <class 'transformers.configuration_gpt2.GPT2Config'> for this kind of AutoModel: AutoModelForSequenceClassification.
Model type should be one of DistilBertConfig, AlbertConfig, CamembertConfig, XLMRobertaConfig, BartConfig, LongformerConfig, RobertaConfig, SqueezeBertConfig, BertConfig, XLNetConfig, MobileBertConfig, FlaubertConfig, XLMConfig, ElectraConfig, FunnelConfig, DebertaConfig.
```


Resolving patch:
diff --git a/src/transformers/modeling_auto.py b/src/transformers/modeling_auto.py
--- a/src/transformers/modeling_auto.py
+++ b/src/transformers/modeling_auto.py
@@ -129,7 +129,7 @@
     FunnelForTokenClassification,
     FunnelModel,
 )
-from .modeling_gpt2 import GPT2LMHeadModel, GPT2Model
+from .modeling_gpt2 import GPT2ForSequenceClassification, GPT2LMHeadModel, GPT2Model
 from .modeling_layoutlm import LayoutLMForMaskedLM, LayoutLMForTokenClassification, LayoutLMModel
 from .modeling_longformer import (
     LongformerForMaskedLM,
@@ -377,6 +377,7 @@
         (ElectraConfig, ElectraForSequenceClassification),
         (FunnelConfig, FunnelForSequenceClassification),
         (DebertaConfig, DebertaForSequenceClassification),
+        (GPT2Config, GPT2ForSequenceClassification),
     ]
 )
 


Test patch: