Question about sliding window
#23
by
Guanzheng
- opened
According to the config.json, there is no sliding window in Mixtral? Does it mean Mixtral was trained on 32k full context length?
Hi
@Guanzheng
I think so yes! Perhaps
@TimeRobber
can confirm
Thank you!
Guanzheng
changed discussion status to
closed
There's no sliding window on Mixtral