4096 to 8192
#1
by JNK333 - opened
this is a CoT model, and at lower quants other than Q8 it rambles alot more i dont think 4096 is the best, i hope a 8192 version can come out because 8192 gives the model enough space for very strong questions or multiple questions
it also doesnt work at all on my S24Ultra