context length

#1
by mrfakename - opened

thanks for making this model! is the context length 32k?

From looking at the config.json it looks like there are references to "_name_or_path": "mistralai/Mistral-7B-v0.1" , so that's my best guess to the base model.

However, it also specifies a sliding window and 32k context. Confusing because Mistral v0.2 has 32k context and no sliding window, and v0.1 has 8k context and a sliding window.

I'd really like to know what's going on here also :-)

Mistral 0.2 has 32k context window, and no sliding window (vs 8k context in v0.1)

Sign up or log in to comment