Why is it showing in LM-Studio as 7B parameters?
Tested the Q8 version and it's working as expected except that it's showing with 7B instead of 8B
I think that's just a glitch with the UI that should get cleaned up shortly :)
It is 7B according to the modelcard .gguf info
{
"name": "Meta-Llama-3-8B-Instruct",
"arch": "llama",
"quant": "Q8_0",
"context_length": 8192,
"embedding_length": 4096,
"num_layers": 32,
"rope": {
"freq_base": 500000,
"dimension_count": 128
},
"head_count": 32,
"head_count_kv": 8,
"parameters": "7B"
}
@MaxPlanck Yeah, just came to past this here
Weird, can't imagine why GGUF is marking it as such, will see what the appropriate fix is
Got some answers here https://www.reddit.com/r/LocalLLaMA/comments/1cqjiuq/is_it_possible_to_edit_the_models_metadata_inside/
But didn't try it