config.json paramaters

#2
by Nexesenex - opened

On this model, it reads :

"max_position_embeddings": 2048,
"torch_dtype": "float16",

Shouldn't it be like on the original CodeLlama model and Airoboros c34b 2.1?

"max_position_embeddings": 16384,
"torch_dtype": "bfloat16",

Also, wouldn't "rope_theta": 1000000, be useful, as per the original CodeLlama model?

Thanks for publishing this, as well as for Samantha C34 1.11 (who is a real challenge for context obedient prompt and model schizophrenia between the Assistant and the Character Samantha), Eric!

Cognitive Computations org

I didn't set any of those values. I used CodeLlama as the base model

Cognitive Computations org
edited Sep 5, 2023

From where? Did you use a non-official source perhaps? The correct values are here: https://huggingface.co/codellama/CodeLlama-34b-hf/blob/main/config.json

It should have 16K and rope_theta 1M yeah. I'll change it for my quants and I put in a PR

Cognitive Computations org

Thank you Tom

Sign up or log in to comment