Error when trying to load the model according to instructions
I used the command
from mlx_lm import load, generate
model, tokenizer = load("mlx-community/Meta-Llama-3.1-8B-4bit")
and got the error
ValueError Traceback (most recent call last)
in <cell line: 6>()
4 from mlx_lm import load, generate
5
----> 6 model, tokenizer = load("mlx-community/Meta-Llama-3.1-8B-4bit")
7 # response = generate(model, tokenizer, prompt="hello", verbose=True)
8
2 frames
/usr/local/lib/python3.10/dist-packages/mlx/nn/layers/base.py in load_weights(self, file_or_weights, strict)
162 if extras := (new_weights.keys() - curr_weights.keys()):
163 extras = " ".join(extras)
--> 164 raise ValueError(f"Received parameters not in model: {extras}.")
165 if missing := (curr_weights.keys() - new_weights.keys()):
166 missing = " ".join(missing)
ValueError: Received parameters not in model: model.embed_tokens.scales model.embed_tokens.biases.
Running in colab by the way with a T4. Do I have to be running on apple silicon for this model to work?
okay it's because the version installed of mlx was outdated. I didn't manage to fix it since pip couldn't get an up to date version of mlx. I guess the model is meant to run on apple silicone only