mlx-community/Mistral-22B-v0.1-4bit-MLX

l0d0v1c

Apr 12

ValueError: Expected shape (6144,) but received shape (4096,) for parameter model.norm.weight
mlx_lm 0.9.0

prince-canuma

MLX Community org Apr 12

Can you provide more details please.
Reproducible example and:

pip list

jpfoures

Apr 16

HI,

I face the same issue during load operation.

mlx 0.10.0
mlx-lm 0.9.0

here is a backtrace:

Fetching 11 files: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 11/11 [00:00<00:00, 56679.78it/s]
Traceback (most recent call last):
    model, tokenizer = load("mlx-community/Mistral-22B-v0.1-4bit-MLX")
  File "/opt/homebrew/lib/python3.10/site-packages/mlx_lm/utils.py", line 385, in load
    model = load_model(model_path, lazy)
  File "/opt/homebrew/lib/python3.10/site-packages/mlx_lm/utils.py", line 349, in load_model
    model.load_weights(list(weights.items()))
  File "/opt/homebrew/lib/python3.10/site-packages/mlx/nn/layers/base.py", line 223, in load_weights
    raise ValueError(
ValueError: Expected shape (6144, 2048) but received  shape (4096, 1792) for parameter model.layers.22.mlp.down_proj.weight

mlx-community
/

Mistral-22B-v0.1-4bit-MLX

error