Unable to load weights .from_pretrained() for XLMRoberta Model
Author: akku1506
Pull request: no

###Defined a class that inherits from RobertaPretrainedModel

This class is defined to get from_pretrained()

    class XLMRobertaPreTrainedModel(RobertaPreTrainedModel):
        """
        An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained
        models.
        """
    
        config_class = XLMRobertaConfig
        base_model_prefix = "xlm-roberta-base"
    
This class is defined to concat image and text embedding

    class XLMRobertaWithImageConcatenationMultiOutputClassifier(XLMRobertaPreTrainedModel):
    
            def __init__(self, config_bert, num_labels=11, num_another_labels=4):
                print ('indise')
                super(XLMRobertaWithImageConcatenationMultiOutputClassifier, self).__init__(config_bert)
                configuration = XLMRobertaConfig(config_bert)
                self.num_labels = num_labels
                self.num_another_labels = num_another_labels
                self.bert = XLMRobertaModel(config_bert)
    
                self.backbone = models.resnet50(pretrained = True)
    
                num_features = self.backbone.fc.in_features
    
                self.img_hidden_size = 512
                self.backbone.fc = torch.nn.Linear(num_features, self.img_hidden_size)
    
                self.dropout = torch.nn.Dropout(configuration.hidden_dropout_prob)        
                self.pre_classify_hidden = 512
    
                self.pre_classify_fc = torch.nn.ReLU(torch.nn.Linear(configuration.hidden_size + self.img_hidden_size, self.pre_classify_hidden))
                self.classifier = torch.nn.Linear(self.pre_classify_hidden, num_labels)
                self.another_classifier = torch.nn.Linear(self.pre_classify_hidden, num_another_labels)
    
                self.apply(self._init_weights)

Trying to call XLMRobertaWithImageConcatenationMultiOutputClassifier from pretrained

                model = XLMRobertaWithImageConcatenationMultiOutputClassifier.from_pretrained('xlm-roberta-base')

The above code gives me error that some weights are not loaded. It lists all the weight layers. Not sure where I am wrong. Any help would be very helpful.

Comments:
Can you provide the error traceback!?

No error. But below thing clearly suggests that weights are not loaded

      Some weights of the model checkpoint at xlm-roberta-base were not used when initializing XLMRobertaWithImageConcatenationMultiOutputClassifier: ['roberta.encoder.layer.3.output.dense.bias', 'roberta.encoder.layer.10.attention.self.query.weight', 'roberta.embeddings.LayerNorm.bias', 'roberta.encoder.layer.11.intermediate.dense.bias', 'roberta.encoder.layer.7.attention.output.dense.weight', 'roberta.encoder.layer.6.attention.self.value.bias', 'roberta.encoder.layer.10.intermediate.dense.bias', 'roberta.encoder.layer.4.attention.self.value.bias', 'roberta.encoder.layer.9.attention.self.key.bias', 'roberta.encoder.layer.2.intermediate.dense.bias', 'roberta.encoder.layer.9.output.dense.bias', 'roberta.embeddings.LayerNorm.weight', 'roberta.encoder.layer.0.output.dense.weight', 'roberta.encoder.layer.4.attention.self.key.weight', 'roberta.encoder.layer.8.output.LayerNorm.bias', 'roberta.encoder.layer.10.output.dense.bias', 'roberta.encoder.layer.1.attention.self.key.weight', 'roberta.encoder.layer.4.attention.output.LayerNorm.weight', 'roberta.encoder.layer.10.output.dense.weight', 'roberta.encoder.layer.3.attention.self.value.weight', 'roberta.encoder.layer.3.attention.output.dense.weight', 'roberta.encoder.layer.4.attention.self.value.weight', 'roberta.encoder.layer.8.output.dense.bias', 'roberta.encoder.layer.3.intermediate.dense.weight', 'roberta.encoder.layer.2.attention.self.value.bias', 'roberta.encoder.layer.1.attention.self.query.weight', 'roberta.encoder.layer.2.attention.output.dense.bias', 'roberta.encoder.layer.5.intermediate.dense.bias', 'roberta.encoder.layer.4.attention.self.key.bias', 'roberta.encoder.layer.5.attention.self.value.bias', 'roberta.embeddings.token_type_embeddings.weight', 'roberta.encoder.layer.7.attention.self.query.weight', 'roberta.encoder.layer.2.attention.self.query.weight', 'roberta.encoder.layer.11.output.LayerNorm.bias', 'roberta.encoder.layer.11.attention.self.key.weight', 'roberta.encoder.layer.4.intermediate.dense.bias', 'roberta.encoder.layer.6.output.LayerNorm.bias', 'roberta.encoder.layer.7.attention.output.LayerNorm.weight', 'roberta.encoder.layer.6.attention.self.query.bias', 'roberta.encoder.layer.5.output.dense.bias', 'roberta.encoder.layer.9.attention.output.dense.weight', 'roberta.encoder.layer.8.intermediate.dense.weight', 'roberta.encoder.layer.1.output.dense.bias', 'roberta.encoder.layer.5.output.dense.weight', 'roberta.encoder.layer.8.attention.output.dense.weight', 'roberta.encoder.layer.10.output.LayerNorm.weight', 'roberta.encoder.layer.5.attention.output.dense.weight', 'roberta.encoder.layer.0.attention.self.query.weight', 'roberta.encoder.layer.4.output.LayerNorm.bias', 'roberta.encoder.layer.2.attention.output.LayerNorm.bias', 'roberta.encoder.layer.3.attention.output.LayerNorm.bias', 'roberta.encoder.layer.11.attention.output.dense.bias', 'roberta.encoder.layer.2.intermediate.dense.weight', 'roberta.encoder.layer.6.attention.output.LayerNorm.bias', 'roberta.encoder.layer.7.attention.self.query.bias', 'roberta.encoder.layer.7.intermediate.dense.weight', 'roberta.encoder.layer.4.attention.self.query.weight', 'roberta.encoder.layer.2.attention.output.dense.weight', 'roberta.encoder.layer.3.attention.output.dense.bias', 'roberta.encoder.layer.4.attention.self.query.bias', 'roberta.encoder.layer.3.attention.self.query.weight', 'roberta.encoder.layer.2.output.LayerNorm.bias', 'roberta.encoder.layer.9.attention.self.query.bias', 'roberta.encoder.layer.9.intermediate.dense.bias', 'roberta.encoder.layer.2.attention.output.LayerNorm.weight', 'roberta.encoder.layer.9.output.LayerNorm.weight', 'roberta.encoder.layer.0.attention.output.LayerNorm.weight', 'roberta.encoder.layer.11.attention.self.key.bias', 'roberta.encoder.layer.1.intermediate.dense.bias', 'roberta.encoder.layer.6.output.dense.bias', 'roberta.encoder.layer.4.output.dense.bias', 'roberta.pooler.dense.weight', 'roberta.encoder.layer.2.output.LayerNorm.weight', 'roberta.encoder.layer.5.intermediate.dense.weight', 'roberta.encoder.layer.7.output.dense.bias', 'roberta.encoder.layer.1.attention.output.dense.weight', 'roberta.encoder.layer.4.intermediate.dense.weight', 'roberta.encoder.layer.5.attention.self.query.bias', 'lm_head.dense.bias', 'roberta.encoder.layer.6.attention.self.key.weight', 'roberta.encoder.layer.10.attention.self.key.weight', 'roberta.encoder.layer.2.attention.self.query.bias', 'roberta.encoder.layer.11.attention.output.dense.weight', 'roberta.encoder.layer.1.intermediate.dense.weight', 'roberta.encoder.layer.7.attention.self.key.bias', 'roberta.encoder.layer.2.attention.self.key.bias', 'roberta.encoder.layer.4.attention.output.LayerNorm.bias', 'roberta.encoder.layer.9.attention.output.LayerNorm.weight', 'lm_head.decoder.weight', 'roberta.encoder.layer.7.output.dense.weight', 'roberta.encoder.layer.6.attention.output.dense.weight', 'roberta.encoder.layer.7.attention.output.dense.bias', 'roberta.encoder.layer.4.attention.output.dense.bias', 'roberta.encoder.layer.10.attention.output.LayerNorm.weight', 'roberta.encoder.layer.11.output.LayerNorm.weight', 'roberta.encoder.layer.1.output.LayerNorm.weight', 'roberta.encoder.layer.2.output.dense.weight', 'roberta.encoder.layer.6.attention.self.value.weight', 'roberta.encoder.layer.4.output.LayerNorm.weight', 'roberta.encoder.layer.8.attention.output.dense.bias', 'roberta.encoder.layer.11.attention.self.query.weight', 'roberta.encoder.layer.11.attention.self.query.bias', 'roberta.encoder.layer.0.attention.self.value.bias', 'roberta.encoder.layer.0.attention.self.query.bias', 'roberta.encoder.layer.9.attention.self.key.weight', 'roberta.encoder.layer.8.attention.self.value.weight', 'roberta.encoder.layer.1.attention.self.query.bias', 'roberta.encoder.layer.10.attention.output.dense.bias', 'roberta.encoder.layer.0.output.dense.bias', 'roberta.encoder.layer.3.attention.self.query.bias', 'roberta.encoder.layer.0.attention.self.value.weight', 'roberta.encoder.layer.8.attention.self.key.bias', 'roberta.encoder.layer.3.output.dense.weight', 'roberta.encoder.layer.11.intermediate.dense.weight', 'roberta.encoder.layer.5.output.LayerNorm.weight', 'roberta.encoder.layer.0.attention.self.key.bias', 'roberta.encoder.layer.3.attention.self.key.weight', 'roberta.encoder.layer.3.output.LayerNorm.bias', 'lm_head.layer_norm.bias', 'lm_head.layer_norm.weight', 'roberta.encoder.layer.6.intermediate.dense.bias', 'roberta.encoder.layer.9.output.LayerNorm.bias', 'roberta.encoder.layer.1.output.LayerNorm.bias', 'roberta.encoder.layer.7.output.LayerNorm.bias', 'roberta.encoder.layer.9.intermediate.dense.weight', 'roberta.encoder.layer.4.output.dense.weight', 'roberta.encoder.layer.8.output.dense.weight', 'roberta.encoder.layer.11.attention.output.LayerNorm.bias', 'roberta.embeddings.word_embeddings.weight', 'roberta.encoder.layer.0.intermediate.dense.weight', 'roberta.encoder.layer.5.attention.self.query.weight', 'roberta.encoder.layer.11.output.dense.bias', 'roberta.encoder.layer.10.attention.self.value.weight', 'roberta.encoder.layer.6.attention.self.query.weight', 'roberta.encoder.layer.8.intermediate.dense.bias', 'roberta.encoder.layer.11.output.dense.weight', 'roberta.encoder.layer.5.attention.output.LayerNorm.weight', 'roberta.encoder.layer.6.output.dense.weight', 'roberta.encoder.layer.2.attention.self.key.weight', 'roberta.encoder.layer.2.attention.self.value.weight', 'roberta.encoder.layer.3.attention.self.value.bias', 'roberta.encoder.layer.8.output.LayerNorm.weight', 'roberta.encoder.layer.1.attention.output.LayerNorm.weight', 'roberta.encoder.layer.7.attention.output.LayerNorm.bias', 'roberta.encoder.layer.6.intermediate.dense.weight', 'roberta.encoder.layer.9.output.dense.weight', 'roberta.encoder.layer.7.attention.self.value.bias', 'lm_head.bias', 'roberta.encoder.layer.3.intermediate.dense.bias', 'roberta.encoder.layer.5.output.LayerNorm.bias', 'roberta.encoder.layer.0.attention.output.dense.bias', 'roberta.encoder.layer.8.attention.output.LayerNorm.weight', 'roberta.encoder.layer.2.output.dense.bias', 'roberta.encoder.layer.6.attention.self.key.bias', 'roberta.encoder.layer.1.attention.self.key.bias', 'roberta.encoder.layer.0.output.LayerNorm.bias', 'roberta.encoder.layer.1.attention.self.value.weight', 'roberta.encoder.layer.0.attention.self.key.weight', 'roberta.encoder.layer.6.attention.output.dense.bias', 'roberta.pooler.dense.bias', 'roberta.encoder.layer.8.attention.self.key.weight', 'roberta.encoder.layer.5.attention.self.value.weight', 'roberta.encoder.layer.10.attention.self.key.bias', 'roberta.encoder.layer.10.attention.output.LayerNorm.bias', 'roberta.encoder.layer.0.output.LayerNorm.weight', 'roberta.encoder.layer.5.attention.output.LayerNorm.bias', 'roberta.encoder.layer.7.intermediate.dense.bias', 'roberta.encoder.layer.0.attention.output.LayerNorm.bias', 'roberta.encoder.layer.0.intermediate.dense.bias', 'roberta.encoder.layer.11.attention.self.value.bias', 'lm_head.dense.weight', 'roberta.encoder.layer.5.attention.self.key.bias', 'roberta.encoder.layer.5.attention.output.dense.bias', 'roberta.encoder.layer.6.attention.output.LayerNorm.weight', 'roberta.encoder.layer.10.attention.self.value.bias', 'roberta.encoder.layer.8.attention.self.value.bias', 'roberta.encoder.layer.8.attention.output.LayerNorm.bias', 'roberta.encoder.layer.9.attention.self.query.weight', 'roberta.encoder.layer.10.output.LayerNorm.bias', 'roberta.encoder.layer.9.attention.self.value.bias', 'roberta.encoder.layer.3.attention.self.key.bias', 'roberta.encoder.layer.7.output.LayerNorm.weight', 'roberta.encoder.layer.8.attention.self.query.weight', 'roberta.encoder.layer.8.attention.self.query.bias', 'roberta.encoder.layer.10.attention.self.query.bias', 'roberta.encoder.layer.5.attention.self.key.weight', 'roberta.encoder.layer.7.attention.self.key.weight', 'roberta.encoder.layer.10.intermediate.dense.weight', 'roberta.encoder.layer.9.attention.output.dense.bias', 'roberta.encoder.layer.11.attention.self.value.weight', 'roberta.encoder.layer.11.attention.output.LayerNorm.weight', 'roberta.encoder.layer.3.attention.output.LayerNorm.weight', 'roberta.encoder.layer.9.attention.self.value.weight', 'roberta.encoder.layer.1.attention.self.value.bias', 'roberta.encoder.layer.1.output.dense.weight', 'roberta.encoder.layer.7.attention.self.value.weight', 'roberta.encoder.layer.6.output.LayerNorm.weight', 'roberta.encoder.layer.3.output.LayerNorm.weight', 'roberta.encoder.layer.10.attention.output.dense.weight', 'roberta.encoder.layer.9.attention.output.LayerNorm.bias', 'roberta.encoder.layer.1.attention.output.dense.bias', 'roberta.encoder.layer.0.attention.output.dense.weight', 'roberta.encoder.layer.4.attention.output.dense.weight', 'roberta.encoder.layer.1.attention.output.LayerNorm.bias', 'roberta.embeddings.position_embeddings.weight']
      - This IS expected if you are initializing XLMRobertaWithImageConcatenationMultiOutputClassifier from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
      - This IS NOT expected if you are initializing XLMRobertaWithImageConcatenationMultiOutputClassifier from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
      Some weights of XLMRobertaWithImageConcatenationMultiOutputClassifier were not initialized from the model checkpoint at xlm-roberta-base and are newly initialized: ['backbone.layer2.0.downsample.1.weight', 'backbone.layer2.2.bn1.running_mean', 'backbone.layer4.1.bn3.num_batches_tracked', 'bert.encoder.layer.11.intermediate.dense.bias', 'backbone.layer2.3.bn3.weight', 'backbone.layer3.5.conv1.weight', 'bert.encoder.layer.10.output.LayerNorm.bias', 'bert.encoder.layer.0.output.LayerNorm.bias', 'bert.encoder.layer.10.intermediate.dense.weight', 'bert.encoder.layer.10.attention.self.key.bias', 'backbone.layer2.2.bn1.bias', 'backbone.layer2.1.conv2.weight', 'backbone.layer2.1.bn2.num_batches_tracked', 'bert.encoder.layer.2.attention.output.LayerNorm.weight', 'backbone.layer3.5.bn1.num_batches_tracked', 'backbone.layer2.2.bn2.num_batches_tracked', 'backbone.bn1.bias', 'bert.pooler.dense.bias', 'backbone.layer3.0.bn1.weight', 'another_classifier.bias', 'bert.encoder.layer.4.attention.self.key.bias', 'backbone.layer3.2.bn1.bias', 'bert.encoder.layer.7.output.LayerNorm.weight', 'backbone.layer4.0.bn3.num_batches_tracked', 'backbone.layer1.1.bn1.running_var', 'backbone.layer3.0.bn3.num_batches_tracked', 'backbone.layer4.0.conv1.weight', 'backbone.layer4.0.bn3.bias', 'bert.encoder.layer.9.attention.output.LayerNorm.bias', 'bert.encoder.layer.9.attention.self.value.weight', 'bert.encoder.layer.8.attention.output.LayerNorm.weight', 'bert.encoder.layer.0.intermediate.dense.weight', 'backbone.layer2.3.bn2.num_batches_tracked', 'bert.encoder.layer.8.attention.self.query.weight', 'backbone.layer1.1.bn3.running_mean', 'bert.encoder.layer.5.output.LayerNorm.weight', 'backbone.layer3.5.bn1.weight', 'backbone.layer2.1.conv1.weight', 'backbone.layer4.1.bn1.num_batches_tracked', 'bert.encoder.layer.7.attention.self.key.bias', 'backbone.layer3.3.bn1.bias', 'backbone.layer2.3.bn3.running_var', 'bert.encoder.layer.0.output.dense.weight', 'backbone.layer2.2.conv2.weight', 'backbone.layer3.4.bn3.running_mean', 'backbone.layer4.0.downsample.0.weight', 'bert.encoder.layer.9.output.LayerNorm.weight', 'backbone.layer3.0.bn1.bias', 'backbone.layer1.0.conv3.weight', 'backbone.layer1.1.bn2.bias', 'backbone.layer2.0.downsample.1.num_batches_tracked', 'backbone.layer2.2.bn3.num_batches_tracked', 'backbone.layer3.2.bn3.weight', 'backbone.layer1.0.bn2.running_mean', 'backbone.layer1.0.bn3.running_var', 'bert.encoder.layer.6.output.LayerNorm.bias', 'backbone.layer3.3.conv3.weight', 'backbone.layer4.1.bn2.weight', 'classifier.weight', 'classifier.bias', 'bert.encoder.layer.6.attention.self.key.weight', 'backbone.layer2.3.bn3.num_batches_tracked', 'bert.encoder.layer.4.attention.self.query.bias', 'bert.encoder.layer.11.output.dense.bias', 'backbone.layer1.2.bn1.running_var', 'backbone.layer2.3.bn1.running_var', 'bert.encoder.layer.3.attention.self.key.bias', 'bert.encoder.layer.3.attention.self.key.weight', 'backbone.layer3.0.conv3.weight', 'backbone.layer4.1.bn2.running_mean', 'backbone.layer3.1.conv2.weight', 'backbone.layer2.0.bn3.running_mean', 'backbone.layer3.0.downsample.1.running_var', 'bert.encoder.layer.0.attention.self.key.bias', 'bert.encoder.layer.1.attention.self.query.bias', 'backbone.layer4.1.bn2.running_var', 'backbone.layer4.2.bn1.running_var', 'bert.encoder.layer.10.output.dense.weight', 'bert.encoder.layer.3.output.dense.bias', 'bert.encoder.layer.0.attention.self.query.bias', 'backbone.layer4.2.bn3.bias', 'bert.encoder.layer.2.output.LayerNorm.weight', 'backbone.layer2.2.bn3.weight', 'backbone.layer2.3.bn3.bias', 'bert.encoder.layer.9.attention.output.dense.bias', 'backbone.layer3.0.bn3.running_mean', 'bert.encoder.layer.9.intermediate.dense.weight', 'backbone.layer1.0.bn1.num_batches_tracked', 'backbone.layer1.0.downsample.1.bias', 'backbone.layer1.1.bn2.running_var', 'backbone.layer3.5.bn3.num_batches_tracked', 'bert.encoder.layer.1.attention.output.dense.bias', 'backbone.layer3.0.bn1.num_batches_tracked', 'backbone.layer3.1.bn3.running_mean', 'backbone.layer3.4.bn1.weight', 'backbone.layer1.0.bn1.weight', 'backbone.bn1.running_var', 'backbone.layer4.1.bn1.weight', 'bert.embeddings.position_embeddings.weight', 'backbone.layer1.0.bn2.weight', 'backbone.layer3.1.bn1.num_batches_tracked', 'backbone.layer2.1.bn2.running_mean', 'backbone.bn1.num_batches_tracked', 'bert.encoder.layer.2.attention.self.query.bias', 'bert.encoder.layer.1.intermediate.dense.weight', 'backbone.layer3.2.conv1.weight', 'backbone.layer4.1.conv2.weight', 'bert.encoder.layer.7.attention.self.key.weight', 'bert.encoder.layer.1.attention.self.value.weight', 'bert.encoder.layer.8.attention.self.value.weight', 'backbone.layer3.0.conv1.weight', 'backbone.layer3.2.bn2.running_mean', 'bert.encoder.layer.1.output.LayerNorm.weight', 'bert.encoder.layer.4.attention.self.key.weight', 'bert.encoder.layer.2.attention.self.key.weight', 'backbone.layer1.2.bn2.running_mean', 'backbone.layer1.1.bn3.weight', 'backbone.layer1.1.bn2.weight', 'backbone.layer2.1.bn1.running_mean', 'bert.encoder.layer.4.intermediate.dense.bias', 'bert.encoder.layer.5.intermediate.dense.bias', 'backbone.layer3.3.conv2.weight', 'bert.encoder.layer.5.attention.output.dense.bias', 'backbone.layer2.3.bn1.bias', 'backbone.layer2.0.bn1.num_batches_tracked', 'bert.encoder.layer.11.attention.output.LayerNorm.bias', 'backbone.layer1.2.conv1.weight', 'bert.encoder.layer.2.attention.output.LayerNorm.bias', 'backbone.layer3.3.bn2.running_mean', 'bert.encoder.layer.3.attention.self.query.weight', 'backbone.layer3.2.bn2.bias', 'bert.encoder.layer.10.attention.self.value.weight', 'backbone.layer3.4.bn1.running_mean', 'bert.encoder.layer.10.attention.self.value.bias', 'backbone.layer3.5.bn2.running_var', 'bert.encoder.layer.5.attention.output.dense.weight', 'backbone.layer2.1.conv3.weight', 'bert.encoder.layer.11.attention.self.key.weight', 'backbone.layer1.0.bn2.bias', 'bert.encoder.layer.8.attention.output.dense.weight', 'backbone.layer2.0.bn2.running_mean', 'backbone.layer1.2.bn2.weight', 'bert.encoder.layer.1.attention.self.key.bias', 'backbone.layer2.0.downsample.1.bias', 'backbone.layer3.2.bn2.num_batches_tracked', 'backbone.layer3.5.bn3.bias', 'backbone.layer2.0.bn1.bias', 'backbone.layer1.2.bn2.bias', 'bert.encoder.layer.0.output.dense.bias', 'bert.encoder.layer.5.attention.output.LayerNorm.weight', 'backbone.layer3.0.bn2.running_var', 'backbone.layer1.0.bn3.running_mean', 'backbone.layer4.0.downsample.1.running_var', 'bert.encoder.layer.11.attention.self.value.bias', 'backbone.layer3.5.bn1.bias', 'bert.encoder.layer.2.output.dense.bias', 'backbone.layer4.0.downsample.1.weight', 'backbone.layer3.0.bn3.bias', 'bert.encoder.layer.7.attention.self.value.bias', 'backbone.layer4.0.bn1.running_mean', 'bert.encoder.layer.10.output.LayerNorm.weight', 'bert.encoder.layer.8.intermediate.dense.weight', 'backbone.layer2.0.bn3.bias', 'backbone.layer2.3.bn2.bias', 'bert.encoder.layer.9.attention.self.value.bias', 'backbone.layer4.2.bn2.num_batches_tracked', 'bert.encoder.layer.9.attention.self.key.weight', 'bert.embeddings.LayerNorm.bias', 'backbone.layer4.2.bn2.running_mean', 'bert.encoder.layer.2.output.dense.weight', 'backbone.layer1.0.downsample.1.running_var', 'backbone.layer1.0.downsample.1.num_batches_tracked', 'backbone.layer3.3.bn2.weight', 'bert.encoder.layer.1.attention.output.dense.weight', 'bert.encoder.layer.5.output.dense.bias', 'bert.encoder.layer.10.attention.output.dense.bias', 'backbone.layer3.3.bn1.running_mean', 'backbone.layer1.0.bn1.running_mean', 'backbone.layer3.1.bn1.bias', 'bert.encoder.layer.6.attention.output.dense.weight', 'bert.encoder.layer.5.intermediate.dense.weight', 'backbone.layer4.0.downsample.1.num_batches_tracked', 'bert.encoder.layer.0.attention.output.LayerNorm.bias', 'bert.encoder.layer.11.attention.self.key.bias', 'backbone.layer1.2.bn1.running_mean', 'backbone.layer3.5.bn2.bias', 'bert.encoder.layer.10.attention.self.query.weight', 'backbone.layer2.1.bn3.num_batches_tracked', 'bert.encoder.layer.5.attention.self.query.weight', 'bert.encoder.layer.0.attention.output.dense.weight', 'bert.encoder.layer.11.attention.output.LayerNorm.weight', 'backbone.layer2.0.bn2.bias', 'backbone.layer3.0.bn2.num_batches_tracked', 'backbone.layer1.0.downsample.1.weight', 'bert.encoder.layer.1.attention.self.key.weight', 'pre_classify_fc.inplace.weight', 'backbone.layer2.0.bn2.running_var', 'bert.encoder.layer.1.output.dense.bias', 'backbone.layer2.2.bn2.weight', 'bert.encoder.layer.3.attention.output.LayerNorm.bias', 'backbone.layer2.0.bn1.running_mean', 'backbone.layer1.0.bn2.num_batches_tracked', 'backbone.layer3.2.bn1.running_mean', 'backbone.layer2.1.bn3.bias', 'backbone.layer2.1.bn3.weight', 'backbone.layer3.3.conv1.weight', 'bert.embeddings.position_ids', 'bert.encoder.layer.3.output.dense.weight', 'bert.encoder.layer.6.output.dense.weight', 'backbone.layer3.5.bn1.running_mean', 'bert.encoder.layer.7.attention.output.LayerNorm.weight', 'backbone.layer3.3.bn2.num_batches_tracked', 'backbone.layer2.3.bn1.weight', 'backbone.layer4.2.bn1.num_batches_tracked', 'bert.encoder.layer.11.attention.self.query.weight', 'backbone.layer1.0.bn2.running_var', 'bert.encoder.layer.2.output.LayerNorm.bias', 'backbone.layer3.4.bn3.running_var', 'backbone.layer3.4.conv2.weight', 'bert.encoder.layer.4.attention.self.value.bias', 'pre_classify_fc.inplace.bias', 'bert.encoder.layer.9.output.dense.bias', 'bert.embeddings.word_embeddings.weight', 'backbone.layer3.1.bn2.weight', 'bert.encoder.layer.4.output.LayerNorm.weight', 'backbone.layer4.1.conv1.weight', 'backbone.layer4.0.bn2.num_batches_tracked', 'bert.encoder.layer.3.attention.self.value.bias', 'bert.encoder.layer.3.output.LayerNorm.weight', 'bert.encoder.layer.6.attention.self.query.weight', 'backbone.layer4.1.conv3.weight', 'backbone.layer2.2.bn1.running_var', 'backbone.layer3.1.conv1.weight', 'bert.encoder.layer.8.attention.self.query.bias', 'backbone.layer3.0.bn2.bias', 'bert.pooler.dense.weight', 'bert.encoder.layer.6.attention.self.value.bias', 'backbone.layer1.1.bn3.running_var', 'bert.encoder.layer.7.intermediate.dense.weight', 'bert.encoder.layer.0.attention.output.dense.bias', 'backbone.layer4.1.bn3.weight', 'bert.encoder.layer.3.attention.self.query.bias', 'backbone.layer3.3.bn1.num_batches_tracked', 'backbone.layer1.1.bn1.num_batches_tracked', 'backbone.layer3.1.bn1.weight', 'backbone.layer2.0.bn1.weight', 'bert.encoder.layer.9.intermediate.dense.bias', 'bert.encoder.layer.2.attention.self.query.weight', 'bert.encoder.layer.11.output.dense.weight', 'backbone.layer3.2.bn2.running_var', 'bert.encoder.layer.7.output.dense.weight', 'bert.encoder.layer.4.attention.output.LayerNorm.bias', 'bert.encoder.layer.3.attention.output.dense.bias', 'backbone.layer2.0.bn2.weight', 'backbone.layer3.2.bn3.num_batches_tracked', 'backbone.layer2.3.bn2.running_mean', 'backbone.layer1.0.downsample.0.weight', 'bert.encoder.layer.6.attention.output.LayerNorm.bias', 'backbone.layer3.3.bn3.running_mean', 'bert.encoder.layer.2.attention.self.value.weight', 'backbone.layer3.5.bn3.running_var', 'bert.encoder.layer.8.output.LayerNorm.weight', 'backbone.layer1.1.bn3.bias', 'backbone.layer1.2.bn3.weight', 'backbone.layer3.2.bn3.running_var', 'backbone.layer3.5.bn3.weight', 'backbone.layer3.0.bn1.running_mean', 'bert.embeddings.token_type_embeddings.weight', 'backbone.layer3.3.bn2.bias', 'bert.encoder.layer.3.intermediate.dense.weight', 'backbone.layer2.2.bn3.running_var', 'bert.encoder.layer.10.attention.output.LayerNorm.weight', 'backbone.layer3.1.bn3.bias', 'bert.encoder.layer.8.output.LayerNorm.bias', 'bert.encoder.layer.11.attention.output.dense.weight', 'bert.encoder.layer.11.attention.self.value.weight', 'bert.encoder.layer.2.attention.self.value.bias', 'bert.encoder.layer.6.attention.self.query.bias', 'backbone.layer1.1.conv2.weight', 'bert.encoder.layer.6.attention.output.LayerNorm.weight', 'backbone.layer1.1.bn2.num_batches_tracked', 'backbone.layer3.0.bn2.weight', 'backbone.layer3.5.bn3.running_mean', 'bert.encoder.layer.1.output.dense.weight', 'backbone.layer3.3.bn3.weight', 'backbone.layer3.1.bn2.running_var', 'bert.encoder.layer.10.intermediate.dense.bias', 'bert.encoder.layer.8.attention.self.value.bias', 'bert.encoder.layer.5.attention.self.value.bias', 'backbone.layer3.3.bn1.running_var', 'backbone.layer3.4.bn3.weight', 'backbone.layer1.2.bn3.running_var', 'bert.encoder.layer.9.attention.self.query.bias', 'backbone.layer1.1.bn1.weight', 'backbone.layer3.4.bn2.bias', 'backbone.layer2.0.bn1.running_var', 'backbone.layer1.1.bn1.bias', 'bert.encoder.layer.11.output.LayerNorm.bias', 'bert.embeddings.LayerNorm.weight', 'backbone.layer1.0.downsample.1.running_mean', 'bert.encoder.layer.7.attention.self.value.weight', 'bert.encoder.layer.10.attention.output.LayerNorm.bias', 'backbone.layer2.2.bn3.bias', 'backbone.layer4.2.bn1.weight', 'backbone.layer2.2.bn2.bias', 'backbone.layer2.2.bn1.num_batches_tracked', 'bert.encoder.layer.9.attention.self.query.weight', 'backbone.layer3.2.bn3.bias', 'bert.encoder.layer.7.attention.self.query.weight', 'backbone.layer3.1.bn1.running_var', 'backbone.layer1.1.bn3.num_batches_tracked', 'backbone.layer1.1.conv3.weight', 'bert.encoder.layer.8.attention.self.key.bias', 'bert.encoder.layer.6.intermediate.dense.weight', 'backbone.layer4.1.bn3.running_mean', 'backbone.layer2.2.bn1.weight', 'backbone.layer1.0.bn1.bias', 'backbone.layer3.0.downsample.1.num_batches_tracked', 'backbone.layer1.0.conv2.weight', 'backbone.layer4.2.bn2.running_var', 'backbone.layer4.2.bn3.num_batches_tracked', 'backbone.layer1.1.bn2.running_mean', 'bert.encoder.layer.6.output.dense.bias', 'backbone.layer2.0.bn3.running_var', 'backbone.layer3.1.bn1.running_mean', 'bert.encoder.layer.5.attention.self.value.weight', 'backbone.layer3.4.bn1.num_batches_tracked', 'backbone.layer4.2.bn1.bias', 'bert.encoder.layer.7.attention.self.query.bias', 'backbone.conv1.weight', 'backbone.fc.weight', 'backbone.layer4.1.bn1.bias', 'backbone.layer4.2.bn2.bias', 'backbone.layer4.0.bn1.weight', 'bert.encoder.layer.3.attention.self.value.weight', 'backbone.layer1.1.bn1.running_mean', 'bert.encoder.layer.6.output.LayerNorm.weight', 'bert.encoder.layer.0.output.LayerNorm.weight', 'bert.encoder.layer.11.attention.self.query.bias', 'backbone.layer4.0.bn2.running_var', 'backbone.layer3.4.conv3.weight', 'bert.encoder.layer.6.intermediate.dense.bias', 'bert.encoder.layer.4.attention.self.query.weight', 'backbone.layer1.2.bn2.num_batches_tracked', 'backbone.layer3.4.bn2.running_mean', 'backbone.bn1.weight', 'bert.encoder.layer.11.intermediate.dense.weight', 'backbone.layer2.0.conv1.weight', 'bert.encoder.layer.9.output.dense.weight', 'bert.encoder.layer.3.intermediate.dense.bias', 'bert.encoder.layer.9.attention.self.key.bias', 'backbone.layer2.3.bn1.num_batches_tracked', 'backbone.layer2.2.bn2.running_var', 'bert.encoder.layer.11.attention.output.dense.bias', 'backbone.layer3.5.bn2.weight', 'backbone.layer3.4.conv1.weight', 'bert.encoder.layer.7.output.LayerNorm.bias', 'backbone.fc.bias', 'backbone.layer3.4.bn3.num_batches_tracked', 'backbone.layer2.0.downsample.1.running_mean', 'bert.encoder.layer.6.attention.self.value.weight', 'backbone.layer3.0.downsample.1.bias', 'backbone.layer2.1.bn3.running_mean', 'bert.encoder.layer.1.attention.output.LayerNorm.bias', 'backbone.layer3.2.conv2.weight', 'backbone.layer4.2.conv3.weight', 'backbone.layer1.1.conv1.weight', 'bert.encoder.layer.2.attention.output.dense.bias', 'backbone.layer3.2.bn3.running_mean', 'backbone.layer3.0.downsample.0.weight', 'backbone.layer4.0.bn2.bias', 'backbone.layer4.1.bn3.bias', 'backbone.layer1.2.conv3.weight', 'bert.encoder.layer.9.output.LayerNorm.bias', 'backbone.layer3.5.bn1.running_var', 'bert.encoder.layer.7.attention.output.dense.bias', 'backbone.layer4.1.bn2.bias', 'bert.encoder.layer.10.attention.self.key.weight', 'backbone.layer1.2.bn1.weight', 'backbone.layer3.5.conv3.weight', 'backbone.layer4.1.bn2.num_batches_tracked', 'bert.encoder.layer.2.intermediate.dense.bias', 'backbone.layer2.0.conv3.weight', 'backbone.layer1.0.bn3.num_batches_tracked', 'backbone.layer2.2.conv1.weight', 'bert.encoder.layer.10.output.dense.bias', 'backbone.bn1.running_mean', 'backbone.layer2.2.bn2.running_mean', 'backbone.layer4.2.bn3.running_mean', 'bert.encoder.layer.5.attention.self.key.weight', 'backbone.layer4.0.bn1.running_var', 'bert.encoder.layer.5.output.dense.weight', 'backbone.layer4.0.bn3.running_var', 'backbone.layer4.2.bn3.running_var', 'backbone.layer3.4.bn2.running_var', 'backbone.layer2.1.bn2.running_var', 'backbone.layer2.2.conv3.weight', 'backbone.layer4.0.conv2.weight', 'backbone.layer4.2.bn1.running_mean', 'bert.encoder.layer.0.attention.self.query.weight', 'backbone.layer2.1.bn1.weight', 'backbone.layer2.0.downsample.1.running_var', 'backbone.layer1.0.bn1.running_var', 'backbone.layer4.2.conv1.weight', 'backbone.layer3.2.bn2.weight', 'backbone.layer3.3.bn3.bias', 'backbone.layer1.2.bn2.running_var', 'bert.encoder.layer.5.attention.self.query.bias', 'backbone.layer4.2.conv2.weight', 'backbone.layer3.1.conv3.weight', 'bert.encoder.layer.3.attention.output.dense.weight', 'bert.encoder.layer.8.attention.self.key.weight', 'backbone.layer3.2.conv3.weight', 'backbone.layer2.1.bn1.bias', 'bert.encoder.layer.4.output.LayerNorm.bias', 'backbone.layer3.2.bn1.running_var', 'bert.encoder.layer.4.output.dense.weight', 'backbone.layer2.1.bn2.weight', 'backbone.layer4.0.conv3.weight', 'backbone.layer1.2.bn3.bias', 'backbone.layer2.1.bn1.num_batches_tracked', 'backbone.layer3.2.bn1.weight', 'backbone.layer3.5.bn2.num_batches_tracked', 'backbone.layer4.0.downsample.1.running_mean', 'bert.encoder.layer.8.attention.output.dense.bias', 'backbone.layer3.0.bn2.running_mean', 'backbone.layer3.3.bn1.weight', 'bert.encoder.layer.0.attention.output.LayerNorm.weight', 'bert.encoder.layer.8.output.dense.weight', 'bert.encoder.layer.1.attention.self.value.bias', 'backbone.layer3.2.bn1.num_batches_tracked', 'backbone.layer2.3.conv2.weight', 'backbone.layer2.3.bn2.running_var', 'backbone.layer2.2.bn3.running_mean', 'backbone.layer1.2.bn1.bias', 'bert.encoder.layer.1.attention.self.query.weight', 'bert.encoder.layer.11.output.LayerNorm.weight', 'backbone.layer4.0.downsample.1.bias', 'bert.encoder.layer.10.attention.self.query.bias', 'backbone.layer3.1.bn2.bias', 'backbone.layer4.0.bn2.weight', 'backbone.layer2.3.bn1.running_mean', 'bert.encoder.layer.5.attention.self.key.bias', 'bert.encoder.layer.5.output.LayerNorm.bias', 'backbone.layer3.3.bn3.running_var', 'bert.encoder.layer.0.attention.self.key.weight', 'backbone.layer3.1.bn2.num_batches_tracked', 'bert.encoder.layer.6.attention.self.key.bias', 'backbone.layer2.0.bn2.num_batches_tracked', 'bert.encoder.layer.5.attention.output.LayerNorm.bias', 'bert.encoder.layer.0.attention.self.value.bias', 'backbone.layer2.3.conv1.weight', 'bert.encoder.layer.7.attention.output.dense.weight', 'bert.encoder.layer.2.attention.self.key.bias', 'backbone.layer1.2.bn3.num_batches_tracked', 'bert.encoder.layer.6.attention.output.dense.bias', 'bert.encoder.layer.7.output.dense.bias', 'backbone.layer3.0.downsample.1.weight', 'backbone.layer3.0.downsample.1.running_mean', 'backbone.layer3.4.bn1.bias', 'bert.encoder.layer.1.attention.output.LayerNorm.weight', 'backbone.layer1.0.bn3.bias', 'bert.encoder.layer.9.attention.output.LayerNorm.weight', 'bert.encoder.layer.3.attention.output.LayerNorm.weight', 'backbone.layer3.0.bn3.running_var', 'backbone.layer1.2.conv2.weight', 'backbone.layer3.1.bn3.weight', 'backbone.layer2.0.downsample.0.weight', 'backbone.layer3.1.bn3.num_batches_tracked', 'bert.encoder.layer.4.attention.output.LayerNorm.weight', 'backbone.layer3.4.bn3.bias', 'backbone.layer3.0.conv2.weight', 'backbone.layer1.2.bn1.num_batches_tracked', 'backbone.layer3.3.bn2.running_var', 'backbone.layer2.0.bn3.weight', 'bert.encoder.layer.1.output.LayerNorm.bias', 'bert.encoder.layer.9.attention.output.dense.weight', 'backbone.layer4.0.bn1.num_batches_tracked', 'bert.encoder.layer.8.intermediate.dense.bias', 'backbone.layer4.1.bn1.running_var', 'bert.encoder.layer.7.intermediate.dense.bias', 'bert.encoder.layer.0.attention.self.value.weight', 'backbone.layer4.1.bn3.running_var', 'backbone.layer3.4.bn2.weight', 'backbone.layer2.0.conv2.weight', 'bert.encoder.layer.2.attention.output.dense.weight', 'bert.encoder.layer.7.attention.output.LayerNorm.bias', 'backbone.layer4.1.bn1.running_mean', 'bert.encoder.layer.10.attention.output.dense.weight', 'bert.encoder.layer.4.attention.output.dense.bias', 'bert.encoder.layer.3.output.LayerNorm.bias', 'bert.encoder.layer.4.attention.self.value.weight', 'backbone.layer4.2.bn2.weight', 'backbone.layer4.0.bn2.running_mean', 'backbone.layer2.3.conv3.weight', 'bert.encoder.layer.0.intermediate.dense.bias', 'backbone.layer2.1.bn1.running_var', 'backbone.layer2.3.bn2.weight', 'backbone.layer3.5.conv2.weight', 'backbone.layer3.1.bn3.running_var', 'bert.encoder.layer.2.intermediate.dense.weight', 'backbone.layer1.0.bn3.weight', 'bert.encoder.layer.4.attention.output.dense.weight', 'backbone.layer3.4.bn2.num_batches_tracked', 'backbone.layer3.0.bn1.running_var', 'backbone.layer3.1.bn2.running_mean', 'bert.encoder.layer.1.intermediate.dense.bias', 'backbone.layer4.0.bn3.weight', 'another_classifier.weight', 'backbone.layer2.1.bn3.running_var', 'backbone.layer2.3.bn3.running_mean', 'backbone.layer4.2.bn3.weight', 'bert.encoder.layer.4.intermediate.dense.weight', 'bert.encoder.layer.4.output.dense.bias', 'backbone.layer2.1.bn2.bias', 'backbone.layer4.0.bn3.running_mean', 'backbone.layer1.2.bn3.running_mean', 'backbone.layer3.3.bn3.num_batches_tracked', 'backbone.layer2.0.bn3.num_batches_tracked', 'bert.encoder.layer.8.attention.output.LayerNorm.bias', 'backbone.layer1.0.conv1.weight', 'backbone.layer3.0.bn3.weight', 'backbone.layer3.4.bn1.running_var', 'bert.encoder.layer.8.output.dense.bias', 'backbone.layer4.0.bn1.bias', 'backbone.layer3.5.bn2.running_mean']
      You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

The name scopes you define in your model should be the same as the pretrained model you're trying to load, so that these weights can be correctly loaded into variables.

For example, according to the warning you pasted, `xlm-roberta-base` uses the name `roberta` as its base model, while in your model it is `bert`, so `from_pretrained` method cannot correspond the names between them.

@qqaatw Yes, you are right that bert encoder layers are being utilized in `xlm-roberta-base.` That's what i am unable to understand why it is initializing with bert base when `XLMRoberta` is base model in class `XLMRobertaWithImageConcatenationMultiOutputClassifier`

The model behind `xlm-roberta-base` checkpoint is `RobertaForMaskedLM`, which uses the name `roberta` as its base model, you can regard `roberta` as an alias of `bert` in Roberta-related models.

Here is it's constructor :
```
def __init__(self, config):
        super().__init__(config)

        if config.is_decoder:
            logger.warning(
                "If you want to use `RobertaForMaskedLM` make sure `config.is_decoder=False` for "
                "bi-directional self-attention."
            )

        self.roberta = RobertaModel(config, add_pooling_layer=False)
        self.lm_head = RobertaLMHead(config)

        # The LM head weights require special treatment only when they are tied with the word embeddings
        self.update_keys_to_ignore(config, ["lm_head.decoder.weight"])

        self.init_weights()
```

And therefore, to load the weights from `xlm-roberta-base`, the names of your custom model's variables should be exactly the same as that of Roberta, I.e. the name `self.bert` should be changed to `self.reberta`, and so do other variables.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.