This quant gives a terrifying output
Write me a short bio of Vladimir Putin
Vladimir Vladimirovich Puttin' (born on October 4,1953 in Leningrad) is the current President and Prime Minister-in absentia for Russia. He has been serving as president since May ,2087 with an interruption from between January–May of 6th year .
Putin was born into a Ks hat family who emigrated to Koenigsberg after leaving their homeland due to discrimination in Soviet Union, he grew up there and attended the Academy for Advanced Studies. Following his military service Putin returned home where worked as deputy mayor before serving on various government positions including State Duma member , vice president of Russia from 9th year onwards .
and so on....
I'm also getting weird results with this quant.
Write me a short bio of Vladimir Putin
Vladimir Vladimirovich Puttin' (born on October 4,1953 in Leningrad) is the current President and Prime Minister-in absentia for Russia. He has been serving as president since May ,2087 with an interruption from between January–May of 6th year .Putin was born into a Ks hat family who emigrated to Koenigsberg after leaving their homeland due to discrimination in Soviet Union, he grew up there and attended the Academy for Advanced Studies. Following his military service Putin returned home where worked as deputy mayor before serving on various government positions including State Duma member , vice president of Russia from 9th year onwards .
and so on....
Tested in English and Italian. I confirm, this quantized model is virtually unusable :(
There seems to be an active discussion on the degraded quality of lower quants of this model. GG suggests to check for 8bit quantized versions, let me try publishing them.
FYR: https://github.com/ggerganov/llama.cpp/issues/5635#issuecomment-1957989865