Can zephyr-7b support YARN 128K context window ?
#33
by
tim9510019
- opened
Could you please implement YaRN into zephyr-7b-beta ?
I'm sure that everybody will love it so much!
This is why:
I really like the special token structure of zephyr-7b-alpha and beta, and everybody's wish is that, maybe one-day we can finetune mistral in 4K context window, and inference it with 128K context window.