fblgit

/

una-cybertron-7b-v2-bf16

like 116

32K i believe. That remains unaffected from zephyr generation, I followed their gym.

Hi, zephyr is 8k right?

i dont have enough GPU to test such thing.. do u ?
But give it a shot, the tok/mod states 32K

Nah, I don’t have a GPU, but thanks for the info!

Using Exllamav2_HF and alpha=1, the model breaks down above 8k tokens. When I set alpha=2.5, it worked with a 16k token limit.

I think it would be useful to mention in the model card that the base context is 8k.

hmmmmmm.. Interesting... can u share the full config ? I will update and cite you on the readme, thanks

No need to mention me. I used the exllamav2_HF loader, 8bpw-h8 exl2 quant, simple-1 preset.