--- license: llama3.1 language: - en base_model: - nvidia/OpenMath2-Llama3.1-8B pipeline_tag: text-generation tags: - math - nvidia - llama --- ## GGUF quantized version of OpenMath2-Llama3.1-8B project original [source](https://huggingface.co/nvidia/OpenMath2-Llama3.1-8B) (finetuned model) Q_2_K (not nice) Q_3_K_S (acceptable) Q_3_K_M is acceptable (good for running with CPU) Q_3_K_L (acceptable) Q_4_K_S (okay) Q_4_K_M is recommanded (balance) Q_5_K_S (good) Q_5_K_M (good in general) Q_6_K is good also; if you want a better result; take this one instead of Q_5_K_M Q_8_0 which is very good; need a reasonable size of RAM otherwise you might expect a long wait f16 is similar to the original hf model; opt this one or hf also fine; make sure you have a good machine *the latest update includes Q_4_0, Q_4_1 (belong to Q4 family) and Q_5_0, Q_5_1 (Q5 family) ### how to run it use any connector for interacting with gguf; i.e., [gguf-connector](https://pypi.org/project/gguf-connector/)