q3_k_m: Uses Q4_K for the attention.wv, attention.wo, and feed_forward.w2 tensors, else Q3_K 8336ecd verified datatab commited on Feb 29
q4_k_m: Recommended. Uses Q6_K for half of the attention.wv and feed_forward.w2 tensors, else Q4_K 1c9b3a7 verified datatab commited on Feb 29