context length
#1
by
mrfakename
- opened
thanks for making this model! is the context length 32k?
From looking at the config.json it looks like there are references to "_name_or_path": "mistralai/Mistral-7B-v0.1" , so that's my best guess to the base model.
However, it also specifies a sliding window and 32k context. Confusing because Mistral v0.2 has 32k context and no sliding window, and v0.1 has 8k context and a sliding window.
I'd really like to know what's going on here also :-)
Mistral 0.2 has 32k context window, and no sliding window (vs 8k context in v0.1)