microsoft

/

deberta-v3-xsmall

like 40

Follow

Microsoft 5,151

I'm getting the following error when I'm loading the model

from transformers import AutoModel
model = AutoModel.from_pretrained("microsoft/deberta-v3-xsmall")

Some weights of the model checkpoint at microsoft/deberta-v3-xsmall were not used when initializing DebertaV2Model: ['deberta.embeddings.word_embeddings._weight', 'lm_predictions.lm_head.bias', 'mask_predictions.classifier.bias', 'mask_predictions.dense.weight', 'lm_predictions.lm_head.dense.weight', 'mask_predictions.LayerNorm.weight', 'lm_predictions.lm_head.LayerNorm.weight', 'lm_predictions.lm_head.dense.bias', 'lm_predictions.lm_head.LayerNorm.bias', 'mask_predictions.dense.bias', 'mask_predictions.LayerNorm.bias', 'mask_predictions.classifier.weight']

• This IS expected if you are initializing DebertaV2Model from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
• This IS NOT expected if you are initializing DebertaV2Model from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).