35B is not the right number of Active parameters
#9
by
kyo-takano
- opened
Although it seems like the number 35B was derived as round(140.6B * 2 / 8), the actual number of active parameters is approximately 39B. See this discussion for the math.
Although it would be too late to change the model name, the model card could be corrected accordingly.
Thanks for pointing this out! Fixed in https://huggingface.co/HuggingFaceH4/zephyr-orpo-141b-A35b-v0.1/discussions/13
lewtun
changed discussion status to
closed