No model card for  roberta-large-finetuned-wsc
Author: ngoquanghuy99
Pull request: no

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

## Motivation

I can not finetune this model `roberta-large-finetuned-wsc`, it doesn't have model card.

## Your contribution
Please fix this!


Comments:
Are you talking about this model? https://huggingface.co/mrm8488/roberta-large-finetuned-wsc

> Are you talking about this model? https://huggingface.co/mrm8488/roberta-large-finetuned-wsc

Yes, this model i can not call by transformers. So how can i use it?

You can't call it from transformers? Do you get an error?
It seems I can load it:

```py
>>> from transformers import AutoModelForMaskedLM
>>> model = AutoModelForMaskedLM.from_pretrained("mrm8488/roberta-large-finetuned-wsc")
```

> You can't call it from transformers? Do you get an error?
> It seems I can load it:
> 
> ```python
> >>> from transformers import AutoModelForMaskedLM
> >>> model = AutoModelForMaskedLM.from_pretrained("mrm8488/roberta-large-finetuned-wsc")
> ```

I couldn't load it from "AutoModel". Thanks for your snippet! Anyways, should i finetune this for text classification task by removing Language Modeling head on top? Just for experiments!

You can load it in a text-classification auto model in order to fine-tune it to text-classification:

```py
>>> from transformers import AutoModelForSequenceClassification
... model = AutoModelForSequenceClassification.from_pretrained("mrm8488/roberta-large-finetuned-wsc")
```

It tells you that the LM head layers were discarded, and that it initialized randomly the layers for text-classification:

```
Some weights of the model checkpoint at mrm8488/roberta-large-finetuned-wsc were not used when initializing RobertaForSequenceClassification: ['lm_head.bias', 'lm_head.dense.weight', 'lm_head.dense.bias', 'lm_head.layer_norm.weight', 'lm_head.layer_norm.bias', 'lm_head.decoder.weight', 'lm_head.decoder.bias', 'roberta.pooler.dense.weight', 'roberta.pooler.dense.bias']
- This IS expected if you are initializing RobertaForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing RobertaForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of RobertaForSequenceClassification were not initialized from the model checkpoint at mrm8488/roberta-large-finetuned-wsc and are newly initialized: ['classifier.dense.weight', 'classifier.dense.bias', 'classifier.out_proj.weight', 'classifier.out_proj.bias']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
```

You should now fine-tune this on a text-classification dataset so that the randomly initialized layers may be trained!

Oh wow, i don't fine tune for classification by this way. I directly removed the LM head though. But still thank you Lysandre!

Happy to help!