Still showing as llama based when it should now be mistral according to card

#2
by spawn99 - opened

See screenshot
Screenshot 2023-11-24 at 12.28.05.png

NousResearch org
β€’
edited Nov 25, 2023

I wonder if this is on @TheBloke 's end rather than ours

I believe this is a known LM Studio bug. I make the GGUFs with the standard llama.cpp tool, and they write the same metadata for every GGUF of a given type. It does show Llama in the metadata of Mistral models, because it is llama architecture.

On HF I already copy all the metadata from the source repo to my repo, so on HF my model is tagged mistral like this source model is. So I believe LM Studio could differentiate Mistral based on that metadata.

It'd be good to hear from someone from LM Studio about this, as to whether anything has changed on their end, as to my mind this shouldn't be a new problem - I've not changed anything in my process, and I'm not aware that llama.cpp has changed the way mistral models are labelled - but I've only started hearing about it recently, always related to LM Studio.

@KerfuffleV2 am I right about all the llama.cpp aspects of that?

I guess now that we have lllama.cpp metadata modification scripts I could post-process the GGUF files to change some field to mistral if that would help out. But I'd need to confirm with LM Studio as to what they're reading when they put the 'llama' tag on a model.

I don't know what the HF handle of Yags (of LM Studio) is, but I have pinged him on Discord to bring his attention to this post.

I've heard back from Yags of LM Studio. This is definitely a bug in LM Studio, and he plans to address it in the future.

Sign up or log in to comment