I am not sure why this model says it is 32K when it instantly breaks after 8k.
#1
by
SytanSD
- opened
I am not sure if 32k is referring to something else, but this model breaks instantly past 8k in oobabooga text gen web UI and silly tavern. I had some other friends test it and they found the same. We made sure to load above 8k context as well as matching the truncation to the 32k context, but it just vomits immediately after 8k like llama 3 base instruct.
An example at 9.1K: