Regression from 2.1 to 2.2.1
#3
by
antonisar
- opened
Using the same prompts, etc just changing the model it produces some incosistencies:
e.g.
Question1 what is the duration: (correct old model 2.1) "1 year" (new model 2.2.1" duration="march2012 to march2013" (directly copied frominput ext )
Question2 extract cs related tags from text: e.g. python, grafana (old model 2.1 all tags extracted new model misses a few)
Notes: Comments regarding the 4bit dolphin-2.2.1-mistral-7b.Q4_K_M.gguf and dolphin-2.1-mistral-7b.Q4_K_M.gguf version 4bit from TheBloke but they dont look to be due to quantization
I am also experiencing some troubles after the update. Inference is a lot slower and the model struggles to keep the required structure in the output