winglian

/

Llama-3-8b-64k-PoSE

like 70

Hi,

first thank you for your work!

I have a question regarding the model's context length. Could you clarify if the model supports a context length greater than 64k tokens out of the box?

Practically speaking, can I set the context size to 80k and more? I suppose that should be supported, because config says:

"max_position_embeddings": 131072

Just want to be sure.

Thanks!