what is the context length of this model ?
#2
by
YairFr
- opened
is it 4K or 16K ?
The base CodeLlama was trained to 16K. It can theoretically be used at longer context lengths as well, and of course shorter if you want
WizardCoder specifically I think they fine tuned at 4K. But the model will still support 16K - though how well it will cope with the fine tuning at that length, I don't know
Is there anyway to get longer context with gguf files? I heard that llama cpp supports longer context but you need to do some configuration (which I do not know yet).