where is OLMo-7B-hf-4bit-mlx stored?
I tried the bare-bones example and from
model, tokenizer = load("mlx-community/OLMo-7B-hf-4bit-mlx")
I found a ton of files along the envs/mlx/lib/ path (suspiciously big with ~1GB), but no model.safetensors or OLMo-7B-hf-4bit-mlx.
On the second try (after a connection error) it seems there may be only a connection to the model on HF, but why then the big model.safetensors d/l firstplace?
Thx
G.
PS: I think
pip install ai2-olmo
is missing from the recipe.
I'm not sure what you mean. There are a lot of files in the envs/mlx/lib path that is the mlx lib path. The model should be downloaded under the Hugging Face cache, for example ~/.cache/huggingface/hub. And yes, if you're missing hf_olmo dependency, mlx-lm will throw an error to tell you to install hf_olmo. see details -> https://github.com/ml-explore/mlx-examples/blob/main/llms/mlx_lm/models/olmo.py#L11
~/.cache/huggingface/hub #/models--mlx-community--OLMo-7B-hf-4bit-mlx
Ah, thanks, Apple Spotlight shows much (and I was pleasantly surprised by the performance), but not what's in hidden folders. <sigh>
And re missing "pip install ai2-olmo" in the recipe I was referring to [Model card - Use with mlx] (https://huggingface.co/mlx-community/OLMo-7B-hf-4bit-mlx)
With the Python dependency jungle it could take you weeks until you are through amending until every single one has thrown an error.
Thanks
G.