prajjwal1/bert-tiny · Some weights of BertForSequenceClassification were not initialized from the model

Feb 4

Do I load it correctly?

from transformers import AutoModelForSequenceClassification, AutoTokenizer
model_name = "prajjwal1/bert-tiny"
model = AutoModelForSequenceClassification.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

Error message:

Some weights of BertForSequenceClassification were not initialized from the model checkpoint at prajjwal1/bert-tiny and are newly initialized: ['classifier.bias', 'classifier.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

j35t3r

Feb 10

Well it should be loaded without "Auto":

from transformers import BertForSequenceClassification
model_name = "prajjwal1/bert-tiny"
model = BertForSequenceClassification.from_pretrained(model_name)

Otherwise, the classification layer is missing.

j35t3r changed discussion status to closed Feb 10