perplexity.md

Quant	Size (MB)	PPL	Size (%)	Accuracy (%)	PPL error rate
IQ1_S	1820	26.3205	12.20	33.81	0.40
IQ1_M	1955	19.0360	13.10	46.75	0.28
IQ2_XXS	2182	13.3276	14.63	66.77	0.20
IQ2_XS	2379	11.7742	15.95	75.58	0.18
IQ2_S	2514	11.3084	16.85	78.69	0.17
IQ2_M	2695	10.3850	18.07	85.69	0.16
Q2_K_S	2730	11.2910	18.30	78.82	0.17
Q2_K	2912	11.1386	19.52	79.89	0.17
IQ3_XXS	3006	9.5453	20.15	93.23	0.14
IQ3_XS	3226	9.2103	21.63	96.62	0.14
Q3_K_S	3365	10.0571	22.56	88.49	0.16
IQ3_S	3382	9.2420	22.67	96.29	0.14
IQ3_M	3479	9.0709	23.32	98.11	0.13
Q3_K_M	3703	9.2078	24.82	96.65	0.14
Q3_K_L	3992	9.1908	26.76	96.83	0.14
IQ4_XS	4101	9.0166	27.49	98.70	0.14
Q4_0	4316	9.4186	28.93	94.49	0.14
IQ4_NL	4318	9.0297	28.95	98.55	0.14
Q4_K_S	4332	8.9634	29.04	99.28	0.13
Q4_K_M	4549	8.9107	30.50	99.87	0.13
Q4_1	4743	8.9614	31.80	99.31	0.13
Q5_K_S	5184	8.9042	34.75	99.94	0.13
Q5_0	5198	9.0533	34.85	98.30	0.14
Q5_K_M	5311	8.9100	35.60	99.88	0.13
Q5_1	5625	8.9230	37.71	99.73	0.13
Q6_K	6121	8.8800	41.03	100.22	0.13
Q8_0	7927	8.8534	53.14	100.52	0.13
F16	14917	8.8992	100	100	0.13