Whats the maximum context length for this model?

#19
by Samvanity - opened

I want to use this with lmstudio and memgpt, and memgpt wants the context length set to the max:
https://memgpt.readthedocs.io/en/latest/lmstudio/#memgpt-lm-studio
it says: Make sure that "context length" is set (inside LM Studio's "Model Configuration" panel) to the max context length of the model you're using (e.g. 8000 for Mistral 7B variants).

Thanks!

@Samvanity > you ca check this for almost each model here on HF - go to the Files and versions tab up there > search for the config.json , click on it > look for the line "max_position_embeddings"

According to openrouter.ai rankings, the 8k context version of this model is exploding!

They refer to it as mythomax-l2-13b-8k, but I couldn't find it anywhere for download? Where is that?

@Gryphe ?

Owner

Heya! I believe someone created a 8k version but never published it, only making it available through a service of sorts.

Afraid that's about the extent of my knowledge at this point.

@Gryphe
Heya, thanks for this wonderful model.
So NTK RoPE scaling for this model is OK ? from 4k to 8k, thanks.

Owner

It should be perfectly fine, yeah! I got lotsa reports claiming that MythoMax should be very robust when it comes to RoPE scaling, even pushing it all the way to 16k.

wow, this is great, is there any relevant information to learn how to expand to 16k, thank you @Gryphe

Sign up or log in to comment