Update README.md

README.md

@@ -5,6 +5,13 @@ tags:
 - hpc
 - parallel
 - axonn
+datasets:
+- hpcgroup/hpc-instruct
+- ise-uiuc/Magicoder-OSS-Instruct-75K
+- nickrosh/Evol-Instruct-Code-80k-v1
+language:
+- en
+pipeline_tag: text-generation
 ---
 
 # HPC-Coder-v2
@@ -34,3 +41,9 @@ Below is an instruction that describes a task. Write a response that appropriate
 
 ```
 
+## Quantized Models
+
+4 and 8 bit quantized weights are available in the GGUF format for use with [llama.cpp](https://github.com/ggerganov/llama.cpp).
+The 4 bit model requires ~3.8 GB memory and can be found [here](https://huggingface.co/hpcgroup/hpc-coder-v2-6.7b-Q4_K_S-GGUF).
+The 8 bit model requires ~7.1 GB memory and can be found [here](https://huggingface.co/hpcgroup/hpc-coder-v2-6.7b-Q8_0-GGUF).
+Further information on how to use them with llama.cpp can be found in [its documentation](https://github.com/ggerganov/llama.cpp).