Update README.md

README.md

@@ -2,9 +2,39 @@
 license: cc-by-nc-4.0
 datasets:
 - victunes/nart-100k-synthetic-buddy-mixed-names
+base_model: victunes/TherapyBeagle-11B-v2
+inference: false
 ---
 **GGUF:** https://huggingface.co/victunes/TherapyBeagle-11B-v2-GGUF
 
+# TherapyBeagle-11B-v2-exl2
+Original model: [TherapyBeagle-11B-v2](https://huggingface.co/victunes/TherapyBeagle-11B-v2)  
+Model creator: [victunes](https://huggingface.co/victunes)
+
+## Quants
+[4bpw h6](https://huggingface.co/cgus/TherapyBeagle-11B-v2-exl2/tree/main)  
+[4.25bpw h6](https://huggingface.co/cgus/TherapyBeagle-11B-v2-exl2/tree/4.25bpw-h6)  
+[4.65bpw h6](https://huggingface.co/cgus/TherapyBeagle-11B-v2-exl2/tree/4.65bpw-h6)  
+[5bpw h6](https://huggingface.co/cgus/TherapyBeagle-11B-v2-exl2/tree/5bpw-h6)  
+[6bpw h6](https://huggingface.co/cgus/TherapyBeagle-11B-v2-exl2/tree/6bpw-h6)  
+[8bpw h8](https://huggingface.co/cgus/TherapyBeagle-11B-v2-exl2/tree/8bpw-h8)  
+
+## Quantization notes
+Made with exllamav2 0.0.18 with the default dataset.  
+Original BF16 .bin files were converted to FP16 safetensors.  
+When I compared 4bpw quants made from BF16 and FP16, there was about 0.1% quality loss for FP16.    
+I picked FP16 version because resulted files had fast loading times when version made from BF16 loaded about 100s slower.  
+Quantization metadata was removed from config.json to fix loading the model with some old Text-Generation-WebUI versions.  
+
+## How to run
+This quantization method uses GPU and requires Exllamav2 loader which can be found in following applications:
+
+[Text Generation Webui](https://github.com/oobabooga/text-generation-webui)  
+[KoboldAI](https://github.com/henk717/KoboldAI)  
+[ExUI](https://github.com/turboderp/exui)  
+[lollms-webui](https://github.com/ParisNeo/lollms-webui)
+
+# Original model card
 # TherapyBeagle 11B v2
 
 _Buddy is here for {{user}}._