Context expansion

#6
by Kotokin - opened

Hi, is it possible to use a model with a 16k context? Which alpha_value should I set? Or at least 12k

You can push this model to around 8K and still get okay results using an alpha_value of around 2.5. If you go much beyond that, the output suffers.

sophosympatheia changed discussion status to closed

Sign up or log in to comment