Results were generated using the CUDA backend and are sorted by Kullback-Leibler divergence relative to FP16. The "WT" importance matrices were created using varying numbers of Wikitext tokens and can be found here. Note: the FP16 logits used for the calculation of all metrics other than perplexity are stored in a binary file between runs. In order to save space this file does not contain the exact same FP32 logits but instead casts them to 16 bit unsigned integers (with some scaling). So the "f16" results are to be understood as the difference resulting only from this downcast.

Quantization	imatrix	Model size [GiB]	PPL	ΔPPL	KLD	Mean Δp	RMS Δp
f16	None	14.97	6.233160 ± 0.037828	0.001524 ± 0.000755	0.000551 ± 0.000002	0.001 ± 0.002 %	0.787 ± 0.004 %
q8_0	None	7.96	6.234284 ± 0.037878	0.002650 ± 0.001006	0.001355 ± 0.000006	-0.019 ± 0.003 %	1.198 ± 0.007 %
q6_K	None	6.14	6.253382 ± 0.038078	0.021748 ± 0.001852	0.005452 ± 0.000035	-0.007 ± 0.006 %	2.295 ± 0.019 %
q5_K_M	None	5.33	6.288607 ± 0.038338	0.056974 ± 0.002598	0.010762 ± 0.000079	-0.114 ± 0.008 %	3.160 ± 0.031 %
q5_K_S	None	5.21	6.336598 ± 0.038755	0.104964 ± 0.003331	0.016595 ± 0.000122	-0.223 ± 0.010 %	3.918 ± 0.036 %
q5_1	None	5.65	6.337857 ± 0.038677	0.106223 ± 0.003476	0.018045 ± 0.000139	-0.287 ± 0.011 %	4.123 ± 0.039 %
q5_0	None	5.21	6.363224 ± 0.038861	0.131591 ± 0.003894	0.022239 ± 0.000166	-0.416 ± 0.012 %	4.634 ± 0.043 %
q4_K_M	WT 10m	4.58	6.382937 ± 0.039055	0.151303 ± 0.004429	0.028152 ± 0.000240	-0.389 ± 0.014 %	5.251 ± 0.049 %
q4_K_M	None	4.58	6.407115 ± 0.039119	0.175482 ± 0.004620	0.031273 ± 0.000238	-0.596 ± 0.014 %	5.519 ± 0.050 %
q4_K_S	WT 10m	4.37	6.409697 ± 0.039189	0.178064 ± 0.004744	0.031951 ± 0.000259	-0.531 ± 0.015 %	5.645 ± 0.051 %
iq4_NL	WT 10m	4.35	6.455593 ± 0.039630	0.223959 ± 0.005201	0.035742 ± 0.000288	-0.590 ± 0.016 %	5.998 ± 0.054 %
iq4_XS	WT 10m	4.14	6.459705 ± 0.039595	0.228071 ± 0.005207	0.036334 ± 0.000284	-0.668 ± 0.016 %	6.044 ± 0.054 %
q4_K_S	None	4.37	6.500529 ± 0.039778	0.268895 ± 0.005638	0.043136 ± 0.000314	-0.927 ± 0.017 %	6.562 ± 0.055 %
q4_1	None	4.78	6.682737 ± 0.041285	0.451103 ± 0.008030	0.071683 ± 0.000505	-0.927 ± 0.017 %	8.512 ± 0.063 %
q4_0	None	4.34	6.700147 ± 0.041226	0.468514 ± 0.007951	0.071940 ± 0.000491	-1.588 ± 0.022 %	8.434 ± 0.061 %
q3_K_L	WT 10m	4.03	6.671223 ± 0.041427	0.439590 ± 0.008154	0.073077 ± 0.000529	-0.940 ± 0.023 %	8.662 ± 0.064 %
q3_K_M	WT 10m	3.74	6.734255 ± 0.041838	0.502622 ± 0.008901	0.084358 ± 0.000588	-1.198 ± 0.024 %	9.292 ± 0.065 %
q3_K_L	None	4.03	6.787876 ± 0.042104	0.556242 ± 0.009171	0.087176 ± 0.000614	-1.532 ± 0.025 %	9.432 ± 0.067 %
q3_K_M	None	3.74	6.888498 ± 0.042669	0.656864 ± 0.010071	0.101913 ± 0.000677	-1.990 ± 0.026 %	10.203 ± 0.068 %
iq3_M	WT 10m	3.53	6.898327 ± 0.041643	0.666694 ± 0.009449	0.102534 ± 0.000663	-3.178 ± 0.026 %	10.513 ± 0.066 %
iq3_S	WT 10m	3.42	6.965501 ± 0.042406	0.733867 ± 0.010245	0.111278 ± 0.000710	-3.066 ± 0.027 %	10.845 ± 0.068 %
iq3_XS	WT 10m	3.28	7.163043 ± 0.043772	0.931409 ± 0.012084	0.138693 ± 0.000857	-3.667 ± 0.031 %	12.148 ± 0.070 %
iq3_XXS	WT 10m	3.05	7.458436 ± 0.046404	1.226803 ± 0.015234	0.183625 ± 0.001042	-3.918 ± 0.035 %	13.836 ± 0.074 %
q3_K_S	WT 10m	3.41	7.602878 ± 0.046848	1.371244 ± 0.015688	0.199821 ± 0.001008	-5.046 ± 0.037 %	14.980 ± 0.070 %
q3_K_S	None	3.41	7.863786 ± 0.048885	1.632152 ± 0.017733	0.228217 ± 0.001079	-5.604 ± 0.038 %	15.541 ± 0.070 %
iq2_M	WT 10m	2.74	8.600799 ± 0.055124	2.369166 ± 0.025244	0.325989 ± 0.00160	-6.463 ± 0.046 %	18.519 ± 0.080 %
q2_K	WT 10k	2.96	8.652290 ± 0.055572	2.420657 ± 0.025587	0.331393 ± 0.001562	-6.606 ± 0.046 %	18.790 ± 0.078 %
q2_K	WT 100k	2.96	8.641993 ± 0.055406	2.410359 ± 0.025495	0.331672 ± 0.001569	-6.628 ± 0.047 %	18.856 ± 0.078 %
q2_K	WT 10m	2.96	8.647825 ± 0.055610	2.416191 ± 0.025683	0.332223 ± 0.001572	-6.500 ± 0.047 %	18.881 ± 0.078 %
q2_K	WT 1m	2.96	8.674365 ± 0.055743	2.442732 ± 0.025843	0.335308 ± 0.001576	-6.634 ± 0.047 %	19.009 ± 0.079 %
q2_K	WT 1k	2.96	8.682605 ± 0.055916	2.450972 ± 0.026069	0.337093 ± 0.001596	-6.596 ± 0.047 %	18.977 ± 0.079 %
q2_K_S	WT 10m	2.96	9.323778 ± 0.061551	3.092145 ± 0.031914	0.403360 ± 0.001787	-7.131 ± 0.049 %	20.050 ± 0.081 %
q2_K_S	WT 1m	2.96	9.329321 ± 0.061378	3.097688 ± 0.031816	0.403590 ± 0.001797	-7.289 ± 0.049 %	20.123 ± 0.081 %
q2_K_S	WT 100k	2.96	9.362973 ± 0.061740	3.131339 ± 0.032169	0.408367 ± 0.001802	-7.198 ± 0.050 %	20.132 ± 0.081 %
q2_K_S	WT 10k	2.96	9.376479 ± 0.062045	3.144846 ± 0.032464	0.408662 ± 0.001819	-7.141 ± 0.050 %	20.120 ± 0.081 %
q2_K_S	WT 1k	2.96	9.415200 ± 0.062475	3.183567 ± 0.032993	0.415865 ± 0.001846	-7.153 ± 0.050 %	20.311 ± 0.082 %
iq2_S	WT 10m	2.56	9.650781 ± 0.063209	3.419148 ± 0.034017	0.439197 ± 0.001976	-8.319 ± 0.052 %	21.491 ± 0.083 %
q2_K	None	2.96	9.751568 ± 0.063312	3.519934 ± 0.033863	0.445132 ± 0.001835	-9.123 ± 0.051 %	21.421 ± 0.079 %
iq2_XS	WT 10m	2.43	10.761424 ± 0.071056	4.529791 ± 0.042229	0.546290 ± 0.002133	-10.576 ± 0.056 %	23.872 ± 0.082 %
iq2_XXS	WT 10m	2.24	14.091782 ± 0.098396	7.860148 ± 0.070752	0.812022 ± 0.002741	-14.363 ± 0.065 %	28.576 ± 0.084 %
iq1_M	WT 10m	2.01	25.493722 ± 0.177903	19.262089 ± 0.152396	1.393084 ± 0.003529	-24.672 ± 0.077 %	38.287 ± 0.084 %
iq1_S	WT 1m	1.88	58.097760 ± 0.438604	51.866126 ± 0.416604	2.211278 ± 0.004688	-32.471 ± 0.087 %	46.418 ± 0.085 %
iq1_S	WT 1k	1.88	58.267851 ± 0.446208	52.036218 ± 0.424373	2.214858 ± 0.004778	-31.880 ± 0.089 %	46.330 ± 0.086 %
iq1_S	WT 100k	1.88	58.581498 ± 0.453145	52.349864 ± 0.431360	2.220834 ± 0.004818	-32.261 ± 0.089 %	46.002 ± 0.086 %
iq1_S	WT 10m	1.88	60.694593 ± 0.471290	54.462959 ± 0.449644	2.254554 ± 0.004868	-31.973 ± 0.088 %	46.271 ± 0.086 %
iq1_S	WT 10k	1.88	63.221324 ± 0.493077	56.989691 ± 0.471423	2.293527 ± 0.004885	-32.261 ± 0.089 %	46.562 ± 0.086 %

"in Q4 model which one better accurate render" i mean, it will follow almost 100% of my prompt say