Some weights of {} were not initialized from the model checkpoint
Author: yeounyi
Pull request: no

I keep failing to load model checkpoint. 
I built a model inheriting PreTrainedModel and have roberta inside initialization. 
Training this model with trainer works fine, but when I try to load the checkpoint using ```from_pretrained```, it keeps failing to load the checkpoint. Can someone help me out? Thanks  

Structure of my model 
```
class MaskClassifier(PreTrainedModel):
    def __init__(self, config, path):
        super().__init__(config=config)
        self.roberta = RobertaModel.from_pretrained(path)
        self.max_mask = 10
        self.hidden_size = RobertaConfig().hidden_size
        self.linear1 = torch.nn.Linear(2 * self.hidden_size, self.hidden_size)
        self.linear2 = torch.nn.Linear(self.hidden_size, self.max_mask + 1)
        self.softmax = torch.nn.Softmax(dim=1)


    def forward(self, input_ids, attention_mask, token_type_ids, labels=None):
       ... 
        # Feed input to RoBERTa
```

Initialize before training
```
config = RobertaConfig()
config.max_position_embeddings = 514
config.type_vocab_size = 1
config.vocab_size = 50265

model = MaskClassifier(config=config, path='roberta-base')
```

Saving after training
```trainer.save_model('./slogan_pretrained')```

Loading the checkpoint
```    
config = RobertaConfig()
config.max_position_embeddings = 514
config.type_vocab_size = 1
config.vocab_size = 50265

 model = MaskClassifier.from_pretrained(path, config=config, path='roberta-base')
```


I found similair issue(https://github.com/huggingface/transformers/issues/2886), but I don't know exactly how I should override the function ```from_pretrained``` and even I tried overriding this functions, it still can't load the checkpoint. 


Error Message

> Some weights of MaskClassifier were not initialized from the model checkpoint at /home/yeoun/slogans/slogan_pretrained and are newly initialized: ['.roberta.embeddings.position_ids', '.roberta.embeddings.word_embeddings.weight', '.roberta.embeddings.position_embeddings.weight', '.roberta.embeddings.token_type_embeddings.weight', '.roberta.embeddings.LayerNorm.weight', '.roberta.embeddings.LayerNorm.bias', '.roberta.encoder.layer.0.attention.self.query.weight', '.roberta.encoder.layer.0.attention.self.query.bias', '.roberta.encoder.layer.0.attention.self.key.weight', '.roberta.encoder.layer.0.attention.self.key.bias', '.roberta.encoder.layer.0.attention.self.value.weight', ...


Comments:
Hi! Thanks for opening an issue. I see two issues with your setup here:
- Why are you using `from_pretrained` to load the `RobertaModel` inside your pre-trained model? You should just initialize a `RobertaModel` from the configuration imo.
- Instead of `PreTrainedModel`, I would instead use `RobertaPreTrainedModel`.

See the below script for an example of what I would recommend. I'm saving & reloading the model to make sure that all the weights get saved/loaded:

```py
from transformers import RobertaModel, RobertaConfig, logging
from transformers.models.roberta.modeling_roberta import RobertaPreTrainedModel
import torch

logging.set_verbosity_info()

class MaskClassifier(RobertaPreTrainedModel):
    def __init__(self, config):
        super().__init__(config=config)
        self.roberta = RobertaModel(config)
        self.max_mask = 10
        self.hidden_size = config.hidden_size
        self.linear1 = torch.nn.Linear(2 * self.hidden_size, self.hidden_size)
        self.linear2 = torch.nn.Linear(self.hidden_size, self.max_mask + 1)
        self.softmax = torch.nn.Softmax(dim=1)

        self.init_weights()

model = MaskClassifier.from_pretrained("roberta-base")
```

Let's see the logs now, for the first load using the `roberta-base` checkpoint:

```
Some weights of the model checkpoint at roberta-base were not used when initializing MaskClassifier: ['lm_head.bias', 'lm_head.dense.weight', 'lm_head.dense.bias', 'lm_head.layer_norm.weight', 'lm_head.layer_norm.bias', 'lm_head.decoder.weight']
- This IS expected if you are initializing MaskClassifier from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing MaskClassifier from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of MaskClassifier were not initialized from the model checkpoint at roberta-base and are newly initialized: ['roberta.embeddings.position_ids', 'linear1.weight', 'linear1.bias', 'linear2.weight', 'linear2.bias']
```

The warning tells you: you're not using the `lm_head` weights, and the following layers are initialized: `linear1` and `linear2`.
Since you're not using the LM head, and the two layers are the ones you just added, then there's nothing to worry about.

Let's try saving the model and reloading it again:

```py
model.save_pretrained("here")
MaskClassifier.from_pretrained("here")
```

The logs show:
```
All model checkpoint weights were used when initializing MaskClassifier.
All the weights of MaskClassifier were initialized from the model checkpoint at here.
```

Success :tada: 

Thanks a lot!!! It works 

@LysandreJik 

I really appreciate your help! You saved me from nightmares... 
Actually I have one more custom model, and I tried the same structure you showed me, but it fails to load the weights. The only difference is that I'm using RobertaForMaskedLM, not RobertaModel here. 

Model Structure
```
class MaskedLM(RobertaPreTrainedModel):
    def __init__(self, config):
        super().__init__(config=config)
        self.roberta = RobertaForMaskedLM(config)
        # self.tokenizer = RobertaTokenizer.from_pretrained('roberta-base')
        self.refinement_num = 3
        # self.mask_id = self.tokenizer.convert_tokens_to_ids([tokenizer.mask_token])[0] # 50264
        self.init_weights()
   def forward( ... )
```
Initialize Model
```
model = MaskedLM.from_pretrained('roberta-base')
```

Error Message 
```
Some weights of the model checkpoint at roberta-base were not used when initializing MaskedLM: ['lm_head.bias', 'lm_head.dense.weight', 'lm_head.dense.bias', 'lm_head.layer_norm.weight', 'lm_head.layer_norm.bias', 'lm_head.decoder.weight', 'roberta.embeddings.word_embeddings.weight', 'roberta.embeddings.position_embeddings.weight', 'roberta.embeddings.token_type_embeddings.weight', 'roberta.embeddings.LayerNorm.weight', 'roberta.embeddings.LayerNorm.bias', 'roberta.encoder.layer.0.attention.self.query.weight', ... ]
- This IS expected if you are initializing MaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing MaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of MaskedLM were not initialized from the model checkpoint at roberta-base and are newly initialized: ['roberta.roberta.embeddings.position_ids', 'roberta.roberta.embeddings.word_embeddings.weight', 'roberta.roberta.embeddings.position_embeddings.weight', 'roberta.roberta.embeddings.token_type_embeddings.weight', 'roberta.roberta.embeddings.LayerNorm.weight', 'roberta.roberta.embeddings.LayerNorm.bias', 'roberta.roberta.encoder.layer.0.attention.self.query.weight', ... ]
```

I don't know why this model has '**roberta.roberta**.embeddings.position_ids', not '**roberta**.embeddings.position_ids'

Hmmm, the issue here is that there is a difference between `RobertaModel`, which has the following weights:
```
embeddings.position_ids
embeddings.xxx
[...]
```
and `RobertaForMaskedLM`, which contains `RobertaModel` under the `roberta` prefix:
```
roberta.embeddings.position_ids
roberta.embeddings.xxx
[...]
lm_head.dense
lm_head.bias
[...]
```

I'm not entirely sure of what you're trying to achieve as I don't see your forward function, but I think you could prevent a lot of pain by redefining your model somewhat like `RobertaForMaskedLM` is setup:


```py
# Import the RobertaLMHead
from transformers.models.roberta.modeling_roberta import RobertaPreTrainedModel, RobertaLMHead


class MaskedLM(RobertaPreTrainedModel):
    def __init__(self, config):
        super().__init__(config=config)

        # Create the RoBERTa model and its head like in the MaskedLM layer
        self.roberta = RobertaModel(config)
        self.lm_head = RobertaLMHead(config)

        self.refinement_num = 3
        self.init_weights()

   def forward( ... )
        outputs = self.roberta(xxx)
        sequence_output = outputs[0]
        prediction_scores = self.lm_head(sequence_output)

        # Do your stuff!
```

This way you can load the checkpoint seamlessly in your model, as the naming with the prefixes will be correct.

Thanks!! I tried to build a MaskedLM with some refinements. After predicting multiple <mask> tokens, mask two random predicted tokens and predict them again. Anyway thanks a lot 🤗 🤗 🤗 

Glad I could help!

> (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).

Hi~ 
I just have a question about why ” (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).“  IS NOT expected? the knowledge in pretrained model is same as your task that you are going to finetune?
Thanks~
Best,
Pengbo