Why is it showing in LM-Studio as 7B parameters?

#2
by yehiaserag - opened

Tested the Q8 version and it's working as expected except that it's showing with 7B instead of 8B

LM Studio Community org

I think that's just a glitch with the UI that should get cleaned up shortly :)

It is 7B according to the modelcard .gguf info
{
"name": "Meta-Llama-3-8B-Instruct",
"arch": "llama",
"quant": "Q8_0",
"context_length": 8192,
"embedding_length": 4096,
"num_layers": 32,
"rope": {
"freq_base": 500000,
"dimension_count": 128
},
"head_count": 32,
"head_count_kv": 8,
"parameters": "7B"
}

@MaxPlanck Yeah, just came to past this here

LM Studio Community org

Weird, can't imagine why GGUF is marking it as such, will see what the appropriate fix is

Sign up or log in to comment