Longer Context
#2
by
AKConsult
- opened
Any chance of extending the context window? Would love to have 16K or more for longer input.
I will look into this for v2. Now, I specifically used the 4k window, since the idea is to run it locally on a smartphone, and <4k just is faster probably to run locally.
I will look into this for v2. Now, I specifically used the 4k window, since the idea is to run it locally on a smartphone, and <4k just is faster probably to run locally.
that is not quite true. the only different between 4k and 128k is 128k are finetuned for longer context.