problem with loading the pretrained model
I tried to run the script you provided and I get the error below:
RuntimeError: Error(s) in loading state_dict for RobertaClass:
Missing key(s) in state_dict: "roberta.pooler.dense.weight", "roberta.pooler.dense.bias".
Unexpected key(s) in state_dict: "classifier.out_proj.weight", "classifier.out_proj.bias".
size mismatch for classifier.dense.weight: copying a param with shape torch.Size([768, 768]) from checkpoint, the shape in current model is torch.Size([11, 768]).
size mismatch for classifier.dense.bias: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Size([11]).
Can you please help me with this?
Thanks for reporting this! I guess it's caused by some change of the libraries needed to run the script. I am currently in between holidays, but when I find the time I will retrain the model and publish it here. Then it will also be better integrated with some additional features that were introduced in the meantime on the model hub.
What you could do for now is downgrading
print(torch.version)
1.9.0+cu102
print(transformers.version)
4.15.0
should work.
Or you could train the model yourself, I added the data to the Hugging Face data set hub (https://huggingface.co/datasets/sem_eval_2018_task_1). This should make it possible to finetune a base model easily with 'Trainer' in transformers.
Hi! Please take a look at the new model that should be more powerful and easy to run: https://huggingface.co/maxpe/twitter-roberta-base-jun2022_sem_eval_2018_task_1.