Max tokens?
#2
by
kriss
- opened
Great model!!
Max tokens is 4K still? or could it accept longer context?
4k context as usual for llama 2, but with RoPE scaling you can go around 10k before it you feel any *noticable degradation. I usually keep at at 8k context with rope alpha 2.5-2.7