Getting a warning and unable to run the code
Hi Team,
I am getting the following warning when trying to run the code shown in model card.
Some weights of the model checkpoint at Koodsml/KooBERT were not used when initializing BertModel: ['cls.predictions.bias', 'cls.predictions.transform.dense.bias', 'cls.predictions.decoder.weight', 'cls.predictions.decoder.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.LayerNorm.bias']
- This IS expected if you are initializing BertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing BertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Hey Rajath, these warnings are expected as this model was only pretrained on masked language modeling. If you have any errors running the code please post the error as well - alternatively you can try using any generic finetuning code that uses other HF transformers for tasks like classification.
Feel free to reopen incase there's still any persisting issue.