cmp-nct commited on
Commit
7214001
1 Parent(s): c240ab5

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -0
README.md ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ This is a high quality quantization of Yi-VL-6B and of the visual transformer.
2
+ Q5_K is almost equal to fp16 in inference and Q6_K is basically the same (not as reliably tested for visual transformers but I assume it's equal to language models)
3
+ You currently need to apply this PR to make it work: https://github.com/ggerganov/llama.cpp/pull/5093 - this adds the additional normalization steps into the projection
4
+
5
+ I do not like this model, it's hallucinating more than anything else based on llava.