mindrage
/

Manticore-13B-Chat-Pyg-Guanaco-GGML

Text Generation

Inference Endpoints

text-generation-inference

Model card Files Files and versions Community

mindrage commited on Jun 12, 2023

Commit

992458b

•

1 Parent(s): 9a903c8

Update README.md

Files changed (1) hide show

README.md +5 -1

README.md CHANGED Viewed

@@ -10,7 +10,11 @@ library_name: transformers
 # GGML of:
 Manticore-13b-Chat-Pyg by [openaccess-ai-collective](https://huggingface.co/openaccess-ai-collective/manticore-13b-chat-pyg) with the Guanaco 13b qLoRa by [TimDettmers](https://huggingface.co/timdettmers/guanaco-13b) applied through [Monero](https://huggingface.co/Monero/Manticore-13b-Chat-Pyg-Guanaco), quantized by [mindrage](https://huggingface.co/mindrage), uncensored
-(q4_0, q5_0 and q8_0 versions available)
 [link to GPTQ Version](https://huggingface.co/mindrage/Manticore-13B-Chat-Pyg-Guanaco-GPTQ-4bit-128g.no-act-order.safetensors)

 # GGML of:
 Manticore-13b-Chat-Pyg by [openaccess-ai-collective](https://huggingface.co/openaccess-ai-collective/manticore-13b-chat-pyg) with the Guanaco 13b qLoRa by [TimDettmers](https://huggingface.co/timdettmers/guanaco-13b) applied through [Monero](https://huggingface.co/Monero/Manticore-13b-Chat-Pyg-Guanaco), quantized by [mindrage](https://huggingface.co/mindrage), uncensored
+12.06.2023: Added versions quantized with the new method (less precision loss relative to compression ratio, but slower (for now)):
+q2_K, q3_KM, q4_KS, q4_KM, q5_KS
+Old Quant method:
+q4_0, q5_0 and q8_0 versions available
 [link to GPTQ Version](https://huggingface.co/mindrage/Manticore-13B-Chat-Pyg-Guanaco-GPTQ-4bit-128g.no-act-order.safetensors)