Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
This is a high quality quantization of Yi-VL-6B and of the visual transformer.
|
| 2 |
+
Q5_K is almost equal to fp16 in inference and Q6_K is basically the same (not as reliably tested for visual transformers but I assume it's equal to language models)
|
| 3 |
+
You currently need to apply this PR to make it work: https://github.com/ggerganov/llama.cpp/pull/5093 - this adds the additional normalization steps into the projection
|
| 4 |
+
|
| 5 |
+
I do not like this model, it's hallucinating more than anything else based on llava.
|