"Size mismatch"  error when trying to download pretrained ChatGPT-4 using transformers
Author: AnastassiyaP
Pull request: no
Labels: name: bug

### System Info

Hi! I have an error "size mismatch" when trying to download ChatGPT-4 using transformers

```
from transformers import AutoModelForCausalLM
model_p = AutoModelForCausalLM.from_pretrained("OpenAI-ChatGPT/ChatGPT-4" )
```

```
RuntimeError: Error(s) in loading state_dict for Qwen2ForCausalLM:
	size mismatch for model.embed_tokens.weight: copying a param with shape torch.Size([151936, 1024]) from checkpoint, the shape in current model is torch.Size([151936, 4096]).
	size mismatch for model.layers.0.self_attn.q_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.0.self_attn.q_proj.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.0.self_attn.k_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.0.self_attn.k_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.0.self_attn.v_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.0.self_attn.v_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.0.self_attn.o_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.0.mlp.gate_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.0.mlp.up_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.0.mlp.down_proj.weight: copying a param with shape torch.Size([1024, 4864]) from checkpoint, the shape in current model is torch.Size([4096, 16384]).
	size mismatch for model.layers.0.input_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.0.post_attention_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.1.self_attn.q_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.1.self_attn.q_proj.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.1.self_attn.k_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.1.self_attn.k_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.1.self_attn.v_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.1.self_attn.v_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.1.self_attn.o_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.1.mlp.gate_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.1.mlp.up_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.1.mlp.down_proj.weight: copying a param with shape torch.Size([1024, 4864]) from checkpoint, the shape in current model is torch.Size([4096, 16384]).
	size mismatch for model.layers.1.input_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.1.post_attention_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.2.self_attn.q_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.2.self_attn.q_proj.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.2.self_attn.k_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.2.self_attn.k_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.2.self_attn.v_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.2.self_attn.v_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.2.self_attn.o_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.2.mlp.gate_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.2.mlp.up_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.2.mlp.down_proj.weight: copying a param with shape torch.Size([1024, 4864]) from checkpoint, the shape in current model is torch.Size([4096, 16384]).
	size mismatch for model.layers.2.input_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.2.post_attention_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.3.self_attn.q_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.3.self_attn.q_proj.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.3.self_attn.k_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.3.self_attn.k_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.3.self_attn.v_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.3.self_attn.v_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.3.self_attn.o_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.3.mlp.gate_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.3.mlp.up_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.3.mlp.down_proj.weight: copying a param with shape torch.Size([1024, 4864]) from checkpoint, the shape in current model is torch.Size([4096, 16384]).
	size mismatch for model.layers.3.input_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.3.post_attention_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.4.self_attn.q_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.4.self_attn.q_proj.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.4.self_attn.k_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.4.self_attn.k_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.4.self_attn.v_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.4.self_attn.v_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.4.self_attn.o_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.4.mlp.gate_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.4.mlp.up_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.4.mlp.down_proj.weight: copying a param with shape torch.Size([1024, 4864]) from checkpoint, the shape in current model is torch.Size([4096, 16384]).
	size mismatch for model.layers.4.input_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.4.post_attention_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.5.self_attn.q_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.5.self_attn.q_proj.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.5.self_attn.k_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.5.self_attn.k_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.5.self_attn.v_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.5.self_attn.v_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.5.self_attn.o_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.5.mlp.gate_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.5.mlp.up_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.5.mlp.down_proj.weight: copying a param with shape torch.Size([1024, 4864]) from checkpoint, the shape in current model is torch.Size([4096, 16384]).
	size mismatch for model.layers.5.input_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.5.post_attention_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.6.self_attn.q_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.6.self_attn.q_proj.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.6.self_attn.k_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.6.self_attn.k_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.6.self_attn.v_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.6.self_attn.v_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.6.self_attn.o_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.6.mlp.gate_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.6.mlp.up_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.6.mlp.down_proj.weight: copying a param with shape torch.Size([1024, 4864]) from checkpoint, the shape in current model is torch.Size([4096, 16384]).
	size mismatch for model.layers.6.input_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.6.post_attention_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.7.self_attn.q_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.7.self_attn.q_proj.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.7.self_attn.k_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.7.self_attn.k_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.7.self_attn.v_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.7.self_attn.v_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.7.self_attn.o_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.7.mlp.gate_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.7.mlp.up_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.7.mlp.down_proj.weight: copying a param with shape torch.Size([1024, 4864]) from checkpoint, the shape in current model is torch.Size([4096, 16384]).
	size mismatch for model.layers.7.input_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.7.post_attention_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.8.self_attn.q_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.8.self_attn.q_proj.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.8.self_attn.k_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.8.self_attn.k_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.8.self_attn.v_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.8.self_attn.v_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.8.self_attn.o_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.8.mlp.gate_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.8.mlp.up_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.8.mlp.down_proj.weight: copying a param with shape torch.Size([1024, 4864]) from checkpoint, the shape in current model is torch.Size([4096, 16384]).
	size mismatch for model.layers.8.input_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.8.post_attention_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.9.self_attn.q_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.9.self_attn.q_proj.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.9.self_attn.k_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.9.self_attn.k_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.9.self_attn.v_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.9.self_attn.v_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.9.self_attn.o_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.9.mlp.gate_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.9.mlp.up_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.9.mlp.down_proj.weight: copying a param with shape torch.Size([1024, 4864]) from checkpoint, the shape in current model is torch.Size([4096, 16384]).
	size mismatch for model.layers.9.input_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.9.post_attention_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.10.self_attn.q_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.10.self_attn.q_proj.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.10.self_attn.k_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.10.self_attn.k_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.10.self_attn.v_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.10.self_attn.v_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.10.self_attn.o_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.10.mlp.gate_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.10.mlp.up_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.10.mlp.down_proj.weight: copying a param with shape torch.Size([1024, 4864]) from checkpoint, the shape in current model is torch.Size([4096, 16384]).
	size mismatch for model.layers.10.input_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.10.post_attention_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.11.self_attn.q_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.11.self_attn.q_proj.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.11.self_attn.k_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.11.self_attn.k_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.11.self_attn.v_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.11.self_attn.v_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.11.self_attn.o_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.11.mlp.gate_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.11.mlp.up_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.11.mlp.down_proj.weight: copying a param with shape torch.Size([1024, 4864]) from checkpoint, the shape in current model is torch.Size([4096, 16384]).
	size mismatch for model.layers.11.input_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.11.post_attention_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.12.self_attn.q_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.12.self_attn.q_proj.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.12.self_attn.k_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.12.self_attn.k_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.12.self_attn.v_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.12.self_attn.v_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.12.self_attn.o_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.12.mlp.gate_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.12.mlp.up_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.12.mlp.down_proj.weight: copying a param with shape torch.Size([1024, 4864]) from checkpoint, the shape in current model is torch.Size([4096, 16384]).
	size mismatch for model.layers.12.input_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.12.post_attention_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.13.self_attn.q_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.13.self_attn.q_proj.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.13.self_attn.k_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.13.self_attn.k_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.13.self_attn.v_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.13.self_attn.v_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.13.self_attn.o_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.13.mlp.gate_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.13.mlp.up_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.13.mlp.down_proj.weight: copying a param with shape torch.Size([1024, 4864]) from checkpoint, the shape in current model is torch.Size([4096, 16384]).
	size mismatch for model.layers.13.input_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.13.post_attention_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.14.self_attn.q_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.14.self_attn.q_proj.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.14.self_attn.k_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.14.self_attn.k_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.14.self_attn.v_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.14.self_attn.v_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.14.self_attn.o_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.14.mlp.gate_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.14.mlp.up_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.14.mlp.down_proj.weight: copying a param with shape torch.Size([1024, 4864]) from checkpoint, the shape in current model is torch.Size([4096, 16384]).
	size mismatch for model.layers.14.input_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.14.post_attention_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.15.self_attn.q_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.15.self_attn.q_proj.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.15.self_attn.k_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.15.self_attn.k_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.15.self_attn.v_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.15.self_attn.v_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.15.self_attn.o_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.15.mlp.gate_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.15.mlp.up_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.15.mlp.down_proj.weight: copying a param with shape torch.Size([1024, 4864]) from checkpoint, the shape in current model is torch.Size([4096, 16384]).
	size mismatch for model.layers.15.input_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.15.post_attention_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.16.self_attn.q_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.16.self_attn.q_proj.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.16.self_attn.k_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.16.self_attn.k_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.16.self_attn.v_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.16.self_attn.v_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.16.self_attn.o_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.16.mlp.gate_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.16.mlp.up_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.16.mlp.down_proj.weight: copying a param with shape torch.Size([1024, 4864]) from checkpoint, the shape in current model is torch.Size([4096, 16384]).
	size mismatch for model.layers.16.input_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.16.post_attention_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.17.self_attn.q_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.17.self_attn.q_proj.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.17.self_attn.k_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.17.self_attn.k_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.17.self_attn.v_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.17.self_attn.v_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.17.self_attn.o_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.17.mlp.gate_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.17.mlp.up_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.17.mlp.down_proj.weight: copying a param with shape torch.Size([1024, 4864]) from checkpoint, the shape in current model is torch.Size([4096, 16384]).
	size mismatch for model.layers.17.input_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.17.post_attention_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.18.self_attn.q_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.18.self_attn.q_proj.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.18.self_attn.k_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.18.self_attn.k_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.18.self_attn.v_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.18.self_attn.v_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.18.self_attn.o_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.18.mlp.gate_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.18.mlp.up_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.18.mlp.down_proj.weight: copying a param with shape torch.Size([1024, 4864]) from checkpoint, the shape in current model is torch.Size([4096, 16384]).
	size mismatch for model.layers.18.input_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.18.post_attention_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.19.self_attn.q_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.19.self_attn.q_proj.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.19.self_attn.k_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.19.self_attn.k_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.19.self_attn.v_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.19.self_attn.v_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.19.self_attn.o_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.19.mlp.gate_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.19.mlp.up_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.19.mlp.down_proj.weight: copying a param with shape torch.Size([1024, 4864]) from checkpoint, the shape in current model is torch.Size([4096, 16384]).
	size mismatch for model.layers.19.input_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.19.post_attention_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.20.self_attn.q_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.20.self_attn.q_proj.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.20.self_attn.k_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.20.self_attn.k_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.20.self_attn.v_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.20.self_attn.v_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.20.self_attn.o_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.20.mlp.gate_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.20.mlp.up_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.20.mlp.down_proj.weight: copying a param with shape torch.Size([1024, 4864]) from checkpoint, the shape in current model is torch.Size([4096, 16384]).
	size mismatch for model.layers.20.input_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.20.post_attention_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.21.self_attn.q_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.21.self_attn.q_proj.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.21.self_attn.k_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.21.self_attn.k_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.21.self_attn.v_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.21.self_attn.v_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.21.self_attn.o_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.21.mlp.gate_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.21.mlp.up_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.21.mlp.down_proj.weight: copying a param with shape torch.Size([1024, 4864]) from checkpoint, the shape in current model is torch.Size([4096, 16384]).
	size mismatch for model.layers.21.input_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.21.post_attention_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.22.self_attn.q_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.22.self_attn.q_proj.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.22.self_attn.k_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.22.self_attn.k_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.22.self_attn.v_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.22.self_attn.v_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.22.self_attn.o_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.22.mlp.gate_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.22.mlp.up_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.22.mlp.down_proj.weight: copying a param with shape torch.Size([1024, 4864]) from checkpoint, the shape in current model is torch.Size([4096, 16384]).
	size mismatch for model.layers.22.input_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.22.post_attention_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.23.self_attn.q_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.23.self_attn.q_proj.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.23.self_attn.k_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.23.self_attn.k_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.23.self_attn.v_proj.weight: copying a param with shape torch.Size([256, 1024]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
	size mismatch for model.layers.23.self_attn.v_proj.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for model.layers.23.self_attn.o_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for model.layers.23.mlp.gate_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.23.mlp.up_proj.weight: copying a param with shape torch.Size([4864, 1024]) from checkpoint, the shape in current model is torch.Size([16384, 4096]).
	size mismatch for model.layers.23.mlp.down_proj.weight: copying a param with shape torch.Size([1024, 4864]) from checkpoint, the shape in current model is torch.Size([4096, 16384]).
	size mismatch for model.layers.23.input_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.layers.23.post_attention_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for model.norm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for lm_head.weight: copying a param with shape torch.Size([151936, 1024]) from checkpoint, the shape in current model is torch.Size([151936, 4096]).
	You may consider adding `ignore_mismatched_sizes=True` in the model `from_pretrained` method.
```


If I use suggested parameter ignore_mismatched_sizes=True, I get another error:
``` 
 Some weights of the model checkpoint at OpenAI-ChatGPT[/ChatGPT-4](http://localhost:44537/ChatGPT-4) were not used when initializing Qwen2ForCausalLM:
```

Help is appreciated



### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("OpenAI-ChatGPT/ChatGPT-4" )

### Expected behavior

Model is downloaded without errors

Comments:
There is not a pretrained ChatGPT-4 available model from HF.  The code is looking at a available config and is mis matching the config weights with the model weights. I would need more information about your situation, but I hope this helps

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.