How to combine two pretrained model in huggingface transformers？
Author: rangehow
Pull request: no

### Feature request

I want to combine two pretrained model(LLAMA and BERT) in a new  python class. More specific，The way I've tried is to define a new class c that inherits llama and load bert in c's \_\_init\_\_ function. 
![image](https://github.com/huggingface/transformers/assets/88258534/c5428b78-68ec-4cc2-8667-587b62853152)

So that I can use c.from_pretrained('llama_ckpt_dir') to load two model together.
`model=C.from_pretrained('llama_ckpt_dir',low_cpu_mem_usage=True)`

 After I use c.save_pretrained(),  even the checkpoint keeps total structure of llama and bert ,bert's params are all random initialize(weights Gaussian initialization bias all zero). （I checked this by torch.load the saved c checkpoint and print it out）

Sincerely requesting some help, what should be done?



### Motivation

Since trainer can be passed only one model at a time, so it seems a good feature that should be concerned for who wants to do things like train two model together?

But there is another difficulty that how to deal with two total diffrent tokenizer from bert and llama(even though this is not required for trainer(since tokenizer usually only used by data preprocess), but I hope I can fix this so that I can completely transform c into a total hf model)

### Your contribution

I'm not sure what I can help, but I can fully support anything that can contribute to this issue. 

Comments:
Hi @rangehow, thanks for raising this issue! 

Are you sure the weights save in `dir_to_bert_ckpt` aren't just gaussian initialized  with bias at 0 and that's what's being loaded in? If I make my own dummy model, following the pattern in the example, the weights for the `self.bert` layer are loaded in, as expected, from the checkpoint and the weights for `self.llama` and `self.lm_head` are randomly initialized, as expected.

Running: 

```py
from transformers import BertModel, LlamaModel, LlamaConfig, LlamaPreTrainedModel
from torch import nn

class Foo(LlamaPreTrainedModel):
    def __init_(self, config, *model_args, **model_kwargs):
        super().__init__(config)
        self.llama = LlamaModel(config)
        self.bert = BertModel.from_pretrained('bert-base-uncased')
        self.lm_head = nn.Linear(config.hidden_size, config.vocab_size)
        self.post_init()

config = LlamaConfig()
model_0 = Foo(config)
model_1 = Foo(config)

same_params = set()
different_params = set()
for ((name_0, param_0), (name_1, param_1)) in zip(model_0.named_parameters(), model_1.named_parameters()):
    assert name_0 == name_1
    if not (param_0 == param_1).all():
        different_params.add(name_0)
    else:
        same_params.add(name_0)
``` 

<details><summary>Same params</summary>{'bert.embeddings.LayerNorm.bias',
 'bert.embeddings.LayerNorm.weight',
 'bert.embeddings.position_embeddings.weight',
 'bert.embeddings.token_type_embeddings.weight',
 'bert.embeddings.word_embeddings.weight',
 'bert.encoder.layer.0.attention.output.LayerNorm.bias',
 'bert.encoder.layer.0.attention.output.LayerNorm.weight',
 'bert.encoder.layer.0.attention.output.dense.bias',
 'bert.encoder.layer.0.attention.output.dense.weight',
 'bert.encoder.layer.0.attention.self.key.bias',
 'bert.encoder.layer.0.attention.self.key.weight',
 'bert.encoder.layer.0.attention.self.query.bias',
 'bert.encoder.layer.0.attention.self.query.weight',
 'bert.encoder.layer.0.attention.self.value.bias',
 'bert.encoder.layer.0.attention.self.value.weight',
 'bert.encoder.layer.0.intermediate.dense.bias',
 'bert.encoder.layer.0.intermediate.dense.weight',
 'bert.encoder.layer.0.output.LayerNorm.bias',
 'bert.encoder.layer.0.output.LayerNorm.weight',
 'bert.encoder.layer.0.output.dense.bias',
 'bert.encoder.layer.0.output.dense.weight',
 'bert.encoder.layer.1.attention.output.LayerNorm.bias',
 'bert.encoder.layer.1.attention.output.LayerNorm.weight',
 'bert.encoder.layer.1.attention.output.dense.bias',
 'bert.encoder.layer.1.attention.output.dense.weight',
 'bert.encoder.layer.1.attention.self.key.bias',
 'bert.encoder.layer.1.attention.self.key.weight',
 'bert.encoder.layer.1.attention.self.query.bias',
 'bert.encoder.layer.1.attention.self.query.weight',
 'bert.encoder.layer.1.attention.self.value.bias',
 'bert.encoder.layer.1.attention.self.value.weight',
 'bert.encoder.layer.1.intermediate.dense.bias',
 'bert.encoder.layer.1.intermediate.dense.weight',
 'bert.encoder.layer.1.output.LayerNorm.bias',
 'bert.encoder.layer.1.output.LayerNorm.weight',
 'bert.encoder.layer.1.output.dense.bias',
 'bert.encoder.layer.1.output.dense.weight',
 'bert.encoder.layer.10.attention.output.LayerNorm.bias',
 'bert.encoder.layer.10.attention.output.LayerNorm.weight',
 'bert.encoder.layer.10.attention.output.dense.bias',
 'bert.encoder.layer.10.attention.output.dense.weight',
 'bert.encoder.layer.10.attention.self.key.bias',
 'bert.encoder.layer.10.attention.self.key.weight',
 'bert.encoder.layer.10.attention.self.query.bias',
 'bert.encoder.layer.10.attention.self.query.weight',
 'bert.encoder.layer.10.attention.self.value.bias',
 'bert.encoder.layer.10.attention.self.value.weight',
 'bert.encoder.layer.10.intermediate.dense.bias',
 'bert.encoder.layer.10.intermediate.dense.weight',
 'bert.encoder.layer.10.output.LayerNorm.bias',
 'bert.encoder.layer.10.output.LayerNorm.weight',
 'bert.encoder.layer.10.output.dense.bias',
 'bert.encoder.layer.10.output.dense.weight',
 'bert.encoder.layer.11.attention.output.LayerNorm.bias',
 'bert.encoder.layer.11.attention.output.LayerNorm.weight',
 'bert.encoder.layer.11.attention.output.dense.bias',
 'bert.encoder.layer.11.attention.output.dense.weight',
 'bert.encoder.layer.11.attention.self.key.bias',
 'bert.encoder.layer.11.attention.self.key.weight',
 'bert.encoder.layer.11.attention.self.query.bias',
 'bert.encoder.layer.11.attention.self.query.weight',
 'bert.encoder.layer.11.attention.self.value.bias',
 'bert.encoder.layer.11.attention.self.value.weight',
 'bert.encoder.layer.11.intermediate.dense.bias',
 'bert.encoder.layer.11.intermediate.dense.weight',
 'bert.encoder.layer.11.output.LayerNorm.bias',
 'bert.encoder.layer.11.output.LayerNorm.weight',
 'bert.encoder.layer.11.output.dense.bias',
 'bert.encoder.layer.11.output.dense.weight',
 'bert.encoder.layer.2.attention.output.LayerNorm.bias',
 'bert.encoder.layer.2.attention.output.LayerNorm.weight',
 'bert.encoder.layer.2.attention.output.dense.bias',
 'bert.encoder.layer.2.attention.output.dense.weight',
 'bert.encoder.layer.2.attention.self.key.bias',
 'bert.encoder.layer.2.attention.self.key.weight',
 'bert.encoder.layer.2.attention.self.query.bias',
 'bert.encoder.layer.2.attention.self.query.weight',
 'bert.encoder.layer.2.attention.self.value.bias',
 'bert.encoder.layer.2.attention.self.value.weight',
 'bert.encoder.layer.2.intermediate.dense.bias',
 'bert.encoder.layer.2.intermediate.dense.weight',
 'bert.encoder.layer.2.output.LayerNorm.bias',
 'bert.encoder.layer.2.output.LayerNorm.weight',
 'bert.encoder.layer.2.output.dense.bias',
 'bert.encoder.layer.2.output.dense.weight',
 'bert.encoder.layer.3.attention.output.LayerNorm.bias',
 'bert.encoder.layer.3.attention.output.LayerNorm.weight',
 'bert.encoder.layer.3.attention.output.dense.bias',
 'bert.encoder.layer.3.attention.output.dense.weight',
 'bert.encoder.layer.3.attention.self.key.bias',
 'bert.encoder.layer.3.attention.self.key.weight',
 'bert.encoder.layer.3.attention.self.query.bias',
 'bert.encoder.layer.3.attention.self.query.weight',
 'bert.encoder.layer.3.attention.self.value.bias',
 'bert.encoder.layer.3.attention.self.value.weight',
 'bert.encoder.layer.3.intermediate.dense.bias',
 'bert.encoder.layer.3.intermediate.dense.weight',
 'bert.encoder.layer.3.output.LayerNorm.bias',
 'bert.encoder.layer.3.output.LayerNorm.weight',
 'bert.encoder.layer.3.output.dense.bias',
 'bert.encoder.layer.3.output.dense.weight',
 'bert.encoder.layer.4.attention.output.LayerNorm.bias',
 'bert.encoder.layer.4.attention.output.LayerNorm.weight',
 'bert.encoder.layer.4.attention.output.dense.bias',
 'bert.encoder.layer.4.attention.output.dense.weight',
 'bert.encoder.layer.4.attention.self.key.bias',
 'bert.encoder.layer.4.attention.self.key.weight',
 'bert.encoder.layer.4.attention.self.query.bias',
 'bert.encoder.layer.4.attention.self.query.weight',
 'bert.encoder.layer.4.attention.self.value.bias',
 'bert.encoder.layer.4.attention.self.value.weight',
 'bert.encoder.layer.4.intermediate.dense.bias',
 'bert.encoder.layer.4.intermediate.dense.weight',
 'bert.encoder.layer.4.output.LayerNorm.bias',
 'bert.encoder.layer.4.output.LayerNorm.weight',
 'bert.encoder.layer.4.output.dense.bias',
 'bert.encoder.layer.4.output.dense.weight',
 'bert.encoder.layer.5.attention.output.LayerNorm.bias',
 'bert.encoder.layer.5.attention.output.LayerNorm.weight',
 'bert.encoder.layer.5.attention.output.dense.bias',
 'bert.encoder.layer.5.attention.output.dense.weight',
 'bert.encoder.layer.5.attention.self.key.bias',
 'bert.encoder.layer.5.attention.self.key.weight',
 'bert.encoder.layer.5.attention.self.query.bias',
 'bert.encoder.layer.5.attention.self.query.weight',
 'bert.encoder.layer.5.attention.self.value.bias',
 'bert.encoder.layer.5.attention.self.value.weight',
 'bert.encoder.layer.5.intermediate.dense.bias',
 'bert.encoder.layer.5.intermediate.dense.weight',
 'bert.encoder.layer.5.output.LayerNorm.bias',
 'bert.encoder.layer.5.output.LayerNorm.weight',
 'bert.encoder.layer.5.output.dense.bias',
 'bert.encoder.layer.5.output.dense.weight',
 'bert.encoder.layer.6.attention.output.LayerNorm.bias',
 'bert.encoder.layer.6.attention.output.LayerNorm.weight',
 'bert.encoder.layer.6.attention.output.dense.bias',
 'bert.encoder.layer.6.attention.output.dense.weight',
 'bert.encoder.layer.6.attention.self.key.bias',
 'bert.encoder.layer.6.attention.self.key.weight',
 'bert.encoder.layer.6.attention.self.query.bias',
 'bert.encoder.layer.6.attention.self.query.weight',
 'bert.encoder.layer.6.attention.self.value.bias',
 'bert.encoder.layer.6.attention.self.value.weight',
 'bert.encoder.layer.6.intermediate.dense.bias',
 'bert.encoder.layer.6.intermediate.dense.weight',
 'bert.encoder.layer.6.output.LayerNorm.bias',
 'bert.encoder.layer.6.output.LayerNorm.weight',
 'bert.encoder.layer.6.output.dense.bias',
 'bert.encoder.layer.6.output.dense.weight',
 'bert.encoder.layer.7.attention.output.LayerNorm.bias',
 'bert.encoder.layer.7.attention.output.LayerNorm.weight',
 'bert.encoder.layer.7.attention.output.dense.bias',
 'bert.encoder.layer.7.attention.output.dense.weight',
 'bert.encoder.layer.7.attention.self.key.bias',
 'bert.encoder.layer.7.attention.self.key.weight',
 'bert.encoder.layer.7.attention.self.query.bias',
 'bert.encoder.layer.7.attention.self.query.weight',
 'bert.encoder.layer.7.attention.self.value.bias',
 'bert.encoder.layer.7.attention.self.value.weight',
 'bert.encoder.layer.7.intermediate.dense.bias',
 'bert.encoder.layer.7.intermediate.dense.weight',
 'bert.encoder.layer.7.output.LayerNorm.bias',
 'bert.encoder.layer.7.output.LayerNorm.weight',
 'bert.encoder.layer.7.output.dense.bias',
 'bert.encoder.layer.7.output.dense.weight',
 'bert.encoder.layer.8.attention.output.LayerNorm.bias',
 'bert.encoder.layer.8.attention.output.LayerNorm.weight',
 'bert.encoder.layer.8.attention.output.dense.bias',
 'bert.encoder.layer.8.attention.output.dense.weight',
 'bert.encoder.layer.8.attention.self.key.bias',
 'bert.encoder.layer.8.attention.self.key.weight',
 'bert.encoder.layer.8.attention.self.query.bias',
 'bert.encoder.layer.8.attention.self.query.weight',
 'bert.encoder.layer.8.attention.self.value.bias',
 'bert.encoder.layer.8.attention.self.value.weight',
 'bert.encoder.layer.8.intermediate.dense.bias',
 'bert.encoder.layer.8.intermediate.dense.weight',
 'bert.encoder.layer.8.output.LayerNorm.bias',
 'bert.encoder.layer.8.output.LayerNorm.weight',
 'bert.encoder.layer.8.output.dense.bias',
 'bert.encoder.layer.8.output.dense.weight',
 'bert.encoder.layer.9.attention.output.LayerNorm.bias',
 'bert.encoder.layer.9.attention.output.LayerNorm.weight',
 'bert.encoder.layer.9.attention.output.dense.bias',
 'bert.encoder.layer.9.attention.output.dense.weight',
 'bert.encoder.layer.9.attention.self.key.bias',
 'bert.encoder.layer.9.attention.self.key.weight',
 'bert.encoder.layer.9.attention.self.query.bias',
 'bert.encoder.layer.9.attention.self.query.weight',
 'bert.encoder.layer.9.attention.self.value.bias',
 'bert.encoder.layer.9.attention.self.value.weight',
 'bert.encoder.layer.9.intermediate.dense.bias',
 'bert.encoder.layer.9.intermediate.dense.weight',
 'bert.encoder.layer.9.output.LayerNorm.bias',
 'bert.encoder.layer.9.output.LayerNorm.weight',
 'bert.encoder.layer.9.output.dense.bias',
 'bert.encoder.layer.9.output.dense.weight',
 'bert.pooler.dense.bias',
 'bert.pooler.dense.weight',
 'llama.layers.0.input_layernorm.weight',
 'llama.layers.0.post_attention_layernorm.weight',
 'llama.layers.1.input_layernorm.weight',
 'llama.layers.1.post_attention_layernorm.weight',
 'llama.layers.10.input_layernorm.weight',
 'llama.layers.10.post_attention_layernorm.weight',
 'llama.layers.11.input_layernorm.weight',
 'llama.layers.11.post_attention_layernorm.weight',
 'llama.layers.12.input_layernorm.weight',
 'llama.layers.12.post_attention_layernorm.weight',
 'llama.layers.13.input_layernorm.weight',
 'llama.layers.13.post_attention_layernorm.weight',
 'llama.layers.14.input_layernorm.weight',
 'llama.layers.14.post_attention_layernorm.weight',
 'llama.layers.15.input_layernorm.weight',
 'llama.layers.15.post_attention_layernorm.weight',
 'llama.layers.16.input_layernorm.weight',
 'llama.layers.16.post_attention_layernorm.weight',
 'llama.layers.17.input_layernorm.weight',
 'llama.layers.17.post_attention_layernorm.weight',
 'llama.layers.18.input_layernorm.weight',
 'llama.layers.18.post_attention_layernorm.weight',
 'llama.layers.19.input_layernorm.weight',
 'llama.layers.19.post_attention_layernorm.weight',
 'llama.layers.2.input_layernorm.weight',
 'llama.layers.2.post_attention_layernorm.weight',
 'llama.layers.20.input_layernorm.weight',
 'llama.layers.20.post_attention_layernorm.weight',
 'llama.layers.21.input_layernorm.weight',
 'llama.layers.21.post_attention_layernorm.weight',
 'llama.layers.22.input_layernorm.weight',
 'llama.layers.22.post_attention_layernorm.weight',
 'llama.layers.23.input_layernorm.weight',
 'llama.layers.23.post_attention_layernorm.weight',
 'llama.layers.24.input_layernorm.weight',
 'llama.layers.24.post_attention_layernorm.weight',
 'llama.layers.25.input_layernorm.weight',
 'llama.layers.25.post_attention_layernorm.weight',
 'llama.layers.26.input_layernorm.weight',
 'llama.layers.26.post_attention_layernorm.weight',
 'llama.layers.27.input_layernorm.weight',
 'llama.layers.27.post_attention_layernorm.weight',
 'llama.layers.28.input_layernorm.weight',
 'llama.layers.28.post_attention_layernorm.weight',
 'llama.layers.29.input_layernorm.weight',
 'llama.layers.29.post_attention_layernorm.weight',
 'llama.layers.3.input_layernorm.weight',
 'llama.layers.3.post_attention_layernorm.weight',
 'llama.layers.30.input_layernorm.weight',
 'llama.layers.30.post_attention_layernorm.weight',
 'llama.layers.31.input_layernorm.weight',
 'llama.layers.31.post_attention_layernorm.weight',
 'llama.layers.4.input_layernorm.weight',
 'llama.layers.4.post_attention_layernorm.weight',
 'llama.layers.5.input_layernorm.weight',
 'llama.layers.5.post_attention_layernorm.weight',
 'llama.layers.6.input_layernorm.weight',
 'llama.layers.6.post_attention_layernorm.weight',
 'llama.layers.7.input_layernorm.weight',
 'llama.layers.7.post_attention_layernorm.weight',
 'llama.layers.8.input_layernorm.weight',
 'llama.layers.8.post_attention_layernorm.weight',
 'llama.layers.9.input_layernorm.weight',
 'llama.layers.9.post_attention_layernorm.weight',
 'llama.norm.weight'}</details>


<details><summary>Different params</summary>{'llama.embed_tokens.weight',
 'llama.layers.0.mlp.down_proj.weight',
 'llama.layers.0.mlp.gate_proj.weight',
 'llama.layers.0.mlp.up_proj.weight',
 'llama.layers.0.self_attn.k_proj.weight',
 'llama.layers.0.self_attn.o_proj.weight',
 'llama.layers.0.self_attn.q_proj.weight',
 'llama.layers.0.self_attn.v_proj.weight',
 'llama.layers.1.mlp.down_proj.weight',
 'llama.layers.1.mlp.gate_proj.weight',
 'llama.layers.1.mlp.up_proj.weight',
 'llama.layers.1.self_attn.k_proj.weight',
 'llama.layers.1.self_attn.o_proj.weight',
 'llama.layers.1.self_attn.q_proj.weight',
 'llama.layers.1.self_attn.v_proj.weight',
 'llama.layers.10.mlp.down_proj.weight',
 'llama.layers.10.mlp.gate_proj.weight',
 'llama.layers.10.mlp.up_proj.weight',
 'llama.layers.10.self_attn.k_proj.weight',
 'llama.layers.10.self_attn.o_proj.weight',
 'llama.layers.10.self_attn.q_proj.weight',
 'llama.layers.10.self_attn.v_proj.weight',
 'llama.layers.11.mlp.down_proj.weight',
 'llama.layers.11.mlp.gate_proj.weight',
 'llama.layers.11.mlp.up_proj.weight',
 'llama.layers.11.self_attn.k_proj.weight',
 'llama.layers.11.self_attn.o_proj.weight',
 'llama.layers.11.self_attn.q_proj.weight',
 'llama.layers.11.self_attn.v_proj.weight',
 'llama.layers.12.mlp.down_proj.weight',
 'llama.layers.12.mlp.gate_proj.weight',
 'llama.layers.12.mlp.up_proj.weight',
 'llama.layers.12.self_attn.k_proj.weight',
 'llama.layers.12.self_attn.o_proj.weight',
 'llama.layers.12.self_attn.q_proj.weight',
 'llama.layers.12.self_attn.v_proj.weight',
 'llama.layers.13.mlp.down_proj.weight',
 'llama.layers.13.mlp.gate_proj.weight',
 'llama.layers.13.mlp.up_proj.weight',
 'llama.layers.13.self_attn.k_proj.weight',
 'llama.layers.13.self_attn.o_proj.weight',
 'llama.layers.13.self_attn.q_proj.weight',
 'llama.layers.13.self_attn.v_proj.weight',
 'llama.layers.14.mlp.down_proj.weight',
 'llama.layers.14.mlp.gate_proj.weight',
 'llama.layers.14.mlp.up_proj.weight',
 'llama.layers.14.self_attn.k_proj.weight',
 'llama.layers.14.self_attn.o_proj.weight',
 'llama.layers.14.self_attn.q_proj.weight',
 'llama.layers.14.self_attn.v_proj.weight',
 'llama.layers.15.mlp.down_proj.weight',
 'llama.layers.15.mlp.gate_proj.weight',
 'llama.layers.15.mlp.up_proj.weight',
 'llama.layers.15.self_attn.k_proj.weight',
 'llama.layers.15.self_attn.o_proj.weight',
 'llama.layers.15.self_attn.q_proj.weight',
 'llama.layers.15.self_attn.v_proj.weight',
 'llama.layers.16.mlp.down_proj.weight',
 'llama.layers.16.mlp.gate_proj.weight',
 'llama.layers.16.mlp.up_proj.weight',
 'llama.layers.16.self_attn.k_proj.weight',
 'llama.layers.16.self_attn.o_proj.weight',
 'llama.layers.16.self_attn.q_proj.weight',
 'llama.layers.16.self_attn.v_proj.weight',
 'llama.layers.17.mlp.down_proj.weight',
 'llama.layers.17.mlp.gate_proj.weight',
 'llama.layers.17.mlp.up_proj.weight',
 'llama.layers.17.self_attn.k_proj.weight',
 'llama.layers.17.self_attn.o_proj.weight',
 'llama.layers.17.self_attn.q_proj.weight',
 'llama.layers.17.self_attn.v_proj.weight',
 'llama.layers.18.mlp.down_proj.weight',
 'llama.layers.18.mlp.gate_proj.weight',
 'llama.layers.18.mlp.up_proj.weight',
 'llama.layers.18.self_attn.k_proj.weight',
 'llama.layers.18.self_attn.o_proj.weight',
 'llama.layers.18.self_attn.q_proj.weight',
 'llama.layers.18.self_attn.v_proj.weight',
 'llama.layers.19.mlp.down_proj.weight',
 'llama.layers.19.mlp.gate_proj.weight',
 'llama.layers.19.mlp.up_proj.weight',
 'llama.layers.19.self_attn.k_proj.weight',
 'llama.layers.19.self_attn.o_proj.weight',
 'llama.layers.19.self_attn.q_proj.weight',
 'llama.layers.19.self_attn.v_proj.weight',
 'llama.layers.2.mlp.down_proj.weight',
 'llama.layers.2.mlp.gate_proj.weight',
 'llama.layers.2.mlp.up_proj.weight',
 'llama.layers.2.self_attn.k_proj.weight',
 'llama.layers.2.self_attn.o_proj.weight',
 'llama.layers.2.self_attn.q_proj.weight',
 'llama.layers.2.self_attn.v_proj.weight',
 'llama.layers.20.mlp.down_proj.weight',
 'llama.layers.20.mlp.gate_proj.weight',
 'llama.layers.20.mlp.up_proj.weight',
 'llama.layers.20.self_attn.k_proj.weight',
 'llama.layers.20.self_attn.o_proj.weight',
 'llama.layers.20.self_attn.q_proj.weight',
 'llama.layers.20.self_attn.v_proj.weight',
 'llama.layers.21.mlp.down_proj.weight',
 'llama.layers.21.mlp.gate_proj.weight',
 'llama.layers.21.mlp.up_proj.weight',
 'llama.layers.21.self_attn.k_proj.weight',
 'llama.layers.21.self_attn.o_proj.weight',
 'llama.layers.21.self_attn.q_proj.weight',
 'llama.layers.21.self_attn.v_proj.weight',
 'llama.layers.22.mlp.down_proj.weight',
 'llama.layers.22.mlp.gate_proj.weight',
 'llama.layers.22.mlp.up_proj.weight',
 'llama.layers.22.self_attn.k_proj.weight',
 'llama.layers.22.self_attn.o_proj.weight',
 'llama.layers.22.self_attn.q_proj.weight',
 'llama.layers.22.self_attn.v_proj.weight',
 'llama.layers.23.mlp.down_proj.weight',
 'llama.layers.23.mlp.gate_proj.weight',
 'llama.layers.23.mlp.up_proj.weight',
 'llama.layers.23.self_attn.k_proj.weight',
 'llama.layers.23.self_attn.o_proj.weight',
 'llama.layers.23.self_attn.q_proj.weight',
 'llama.layers.23.self_attn.v_proj.weight',
 'llama.layers.24.mlp.down_proj.weight',
 'llama.layers.24.mlp.gate_proj.weight',
 'llama.layers.24.mlp.up_proj.weight',
 'llama.layers.24.self_attn.k_proj.weight',
 'llama.layers.24.self_attn.o_proj.weight',
 'llama.layers.24.self_attn.q_proj.weight',
 'llama.layers.24.self_attn.v_proj.weight',
 'llama.layers.25.mlp.down_proj.weight',
 'llama.layers.25.mlp.gate_proj.weight',
 'llama.layers.25.mlp.up_proj.weight',
 'llama.layers.25.self_attn.k_proj.weight',
 'llama.layers.25.self_attn.o_proj.weight',
 'llama.layers.25.self_attn.q_proj.weight',
 'llama.layers.25.self_attn.v_proj.weight',
 'llama.layers.26.mlp.down_proj.weight',
 'llama.layers.26.mlp.gate_proj.weight',
 'llama.layers.26.mlp.up_proj.weight',
 'llama.layers.26.self_attn.k_proj.weight',
 'llama.layers.26.self_attn.o_proj.weight',
 'llama.layers.26.self_attn.q_proj.weight',
 'llama.layers.26.self_attn.v_proj.weight',
 'llama.layers.27.mlp.down_proj.weight',
 'llama.layers.27.mlp.gate_proj.weight',
 'llama.layers.27.mlp.up_proj.weight',
 'llama.layers.27.self_attn.k_proj.weight',
 'llama.layers.27.self_attn.o_proj.weight',
 'llama.layers.27.self_attn.q_proj.weight',
 'llama.layers.27.self_attn.v_proj.weight',
 'llama.layers.28.mlp.down_proj.weight',
 'llama.layers.28.mlp.gate_proj.weight',
 'llama.layers.28.mlp.up_proj.weight',
 'llama.layers.28.self_attn.k_proj.weight',
 'llama.layers.28.self_attn.o_proj.weight',
 'llama.layers.28.self_attn.q_proj.weight',
 'llama.layers.28.self_attn.v_proj.weight',
 'llama.layers.29.mlp.down_proj.weight',
 'llama.layers.29.mlp.gate_proj.weight',
 'llama.layers.29.mlp.up_proj.weight',
 'llama.layers.29.self_attn.k_proj.weight',
 'llama.layers.29.self_attn.o_proj.weight',
 'llama.layers.29.self_attn.q_proj.weight',
 'llama.layers.29.self_attn.v_proj.weight',
 'llama.layers.3.mlp.down_proj.weight',
 'llama.layers.3.mlp.gate_proj.weight',
 'llama.layers.3.mlp.up_proj.weight',
 'llama.layers.3.self_attn.k_proj.weight',
 'llama.layers.3.self_attn.o_proj.weight',
 'llama.layers.3.self_attn.q_proj.weight',
 'llama.layers.3.self_attn.v_proj.weight',
 'llama.layers.30.mlp.down_proj.weight',
 'llama.layers.30.mlp.gate_proj.weight',
 'llama.layers.30.mlp.up_proj.weight',
 'llama.layers.30.self_attn.k_proj.weight',
 'llama.layers.30.self_attn.o_proj.weight',
 'llama.layers.30.self_attn.q_proj.weight',
 'llama.layers.30.self_attn.v_proj.weight',
 'llama.layers.31.mlp.down_proj.weight',
 'llama.layers.31.mlp.gate_proj.weight',
 'llama.layers.31.mlp.up_proj.weight',
 'llama.layers.31.self_attn.k_proj.weight',
 'llama.layers.31.self_attn.o_proj.weight',
 'llama.layers.31.self_attn.q_proj.weight',
 'llama.layers.31.self_attn.v_proj.weight',
 'llama.layers.4.mlp.down_proj.weight',
 'llama.layers.4.mlp.gate_proj.weight',
 'llama.layers.4.mlp.up_proj.weight',
 'llama.layers.4.self_attn.k_proj.weight',
 'llama.layers.4.self_attn.o_proj.weight',
 'llama.layers.4.self_attn.q_proj.weight',
 'llama.layers.4.self_attn.v_proj.weight',
 'llama.layers.5.mlp.down_proj.weight',
 'llama.layers.5.mlp.gate_proj.weight',
 'llama.layers.5.mlp.up_proj.weight',
 'llama.layers.5.self_attn.k_proj.weight',
 'llama.layers.5.self_attn.o_proj.weight',
 'llama.layers.5.self_attn.q_proj.weight',
 'llama.layers.5.self_attn.v_proj.weight',
 'llama.layers.6.mlp.down_proj.weight',
 'llama.layers.6.mlp.gate_proj.weight',
 'llama.layers.6.mlp.up_proj.weight',
 'llama.layers.6.self_attn.k_proj.weight',
 'llama.layers.6.self_attn.o_proj.weight',
 'llama.layers.6.self_attn.q_proj.weight',
 'llama.layers.6.self_attn.v_proj.weight',
 'llama.layers.7.mlp.down_proj.weight',
 'llama.layers.7.mlp.gate_proj.weight',
 'llama.layers.7.mlp.up_proj.weight',
 'llama.layers.7.self_attn.k_proj.weight',
 'llama.layers.7.self_attn.o_proj.weight',
 'llama.layers.7.self_attn.q_proj.weight',
 'llama.layers.7.self_attn.v_proj.weight',
 'llama.layers.8.mlp.down_proj.weight',
 'llama.layers.8.mlp.gate_proj.weight',
 'llama.layers.8.mlp.up_proj.weight',
 'llama.layers.8.self_attn.k_proj.weight',
 'llama.layers.8.self_attn.o_proj.weight',
 'llama.layers.8.self_attn.q_proj.weight',
 'llama.layers.8.self_attn.v_proj.weight',
 'llama.layers.9.mlp.down_proj.weight',
 'llama.layers.9.mlp.gate_proj.weight',
 'llama.layers.9.mlp.up_proj.weight',
 'llama.layers.9.self_attn.k_proj.weight',
 'llama.layers.9.self_attn.o_proj.weight',
 'llama.layers.9.self_attn.q_proj.weight',
 'llama.layers.9.self_attn.v_proj.weight',
 'lm_head.weight'}</details>




Note: 
* Your class should inherit from `LlamaPreTrainedModel`, however I tried with both `LlamaPreTrainedModel` and `LlamaForCausalLM` and it works
* Layernorm params are the same, even for randomly initialized layers as they're [initially set to 1 for the weight and 0 for the bias](https://pytorch.org/docs/stable/generated/torch.nn.LayerNorm.html)

> Hi @rangehow, thanks for raising this issue!
> 
> Are you sure the weights save in `dir_to_bert_ckpt` aren't just gaussian initialized with bias at 0 and that's what's being loaded in? If I make my own dummy model, following the pattern in the example, the weights for the `self.bert` layer are loaded in, as expected, from the checkpoint and the weights for `self.llama` and `self.lm_head` are randomly initialized, as expected.
> 
> Running:
> 
> ```python
> from transformers import BertModel, LlamaModel, LlamaConfig, LlamaPreTrainedModel
> from torch import nn
> 
> class Foo(LlamaPreTrainedModel):
>     def __init_(self, config, *model_args, **model_kwargs):
>         super().__init__(config)
>         self.llama = LlamaModel(config)
>         self.bert = BertModel.from_pretrained('bert-base-uncased')
>         self.lm_head = nn.Linear(config.hidden_size, config.vocab_size)
>         self.post_init()
> 
> config = LlamaConfig()
> model_0 = Foo(config)
> model_1 = Foo(config)
> 
> same_params = set()
> different_params = set()
> for ((name_0, param_0), (name_1, param_1)) in zip(model_0.named_parameters(), model_1.named_parameters()):
>     assert name_0 == name_1
>     if not (param_0 == param_1).all():
>         different_params.add(name_0)
>     else:
>         same_params.add(name_0)
> ```
> 
> Same params
> Different params
> Note:
> 
> * Your class should inherit from `LlamaPreTrainedModel`, however I tried with both `LlamaPreTrainedModel` and `LlamaForCausalLM` and it works
> * Layernorm params are the same, even for randomly initialized layers as they're [initially set to 1 for the weight and 0 for the bias](https://pytorch.org/docs/stable/generated/torch.nn.LayerNorm.html)

Sorry for the late reply and thank you for your patient help！ The first question that ckpt incorrect saved magically disappeared after I updated Transformers without editing any code (The difference I can observe is save_pretrained method default save safetensor instead pytorch file). 
I still have some doubts：
1. Is my approach an appropriate way to combine 2 pretrained model together so that I can use transformers trainer? 
2. **How can I save the strange combined model after training since they have two totally different tokenizer.** The solution I can think of now is after I initialize c class and save it ，I can delete the ckpt in bert folder and change the class c code from bert.from_pretrained(bert_folder) to load config and self.bert(BertConfig) .  Now I have to keep a c_folder (contains llama tokenizer, combined weights of bert and llama, llama config) and bert_folder( bert_tokenizer and bert_config). This doesn't seem like the normal form of a transformers model.

I am trying to jointly training retriever (a bert like model) and llm together using transformers's trainer .Not sure if other members of the community has tried this too, because RAG is really popular now  : )

@rangehow Thanks for updating about what made things work on your end. 

Regarding your other questions, these are best placed in our [forums](https://discuss.huggingface.co/). We try to reserve the github issues for feature requests and bug reports. You'll be able to have in-depth discussions and find out what other members of the community have tried there. 

In general:
1. It depends what you mean by "appropriate". You can certainly combine different modules into another `PretraindeModel` subclass. However, there's a few things to be aware of: 
* Using `from_pretrained` means the same checkpoint will be loaded everytime. So, if you train the model, these weight will update alongside the others in the model. When you reload, the weights in the `self.llama` model will expecting values from the fine-tuned bert model. Unless you freeze the weights in `self.bert` you'll likely encounter performance issues. 
* The `PretrainedModel` class you choose as the parent class controls how the model weights are initialized. If you use `from_pretrained` then the pretrained weights are loaded in, but if you use the config and `LlamaPreTrainedModel` then you will end up initializing bert's weights with Llama's logic.
2. If they have two different tokenizers, then why combine them into one model? If you need both tokenizers, you can look into combining them with a processor class