Main branch

README.md

@@ -0,0 +1,41 @@
+---
+datasets:
+- Norquinal/claude_multiround_chat_1k
+license: cc-by-nc-4.0
+---
+
+## Exllama v2 Quantizations of Mistral-7B-claude-chat
+
+Each branches contains an individual bits per weight.
+
+Conversion was done using wikitext.parquet as calibration dataset.
+
+Original model: https://huggingface.co/Norquinal/Mistral-7B-claude-chat
+
+# To download, you can use the following command:
+
+With git:
+
+```shell
+git clone --single-branch --branch 4.0 https://huggingface.co/bartowski/Mistral-7B-claude-chat-exl2
+```
+
+With huggingface hub:
+
+```shell
+pip3 install huggingface-hub
+```
+
+To download the `main` (only useful if you only care about measurement.json) branch to a folder called `Mistral-7B-claude-chat-exl2`:
+
+```shell
+mkdir Mistral-7B-claude-chat-exl2
+huggingface-cli download bartowski/Mistral-7B-claude-chat-exl2 --local-dir Mistral-7B-claude-chat-exl2 --local-dir-use-symlinks False
+```
+
+To download from a different branch, add the `--revision` parameter:
+
+```shell
+mkdir Mistral-7B-claude-chat-exl2
+huggingface-cli download bartowski/Mistral-7B-claude-chat-exl2 --revision 4.0 --local-dir Mistral-7B-claude-chat-exl2 --local-dir-use-symlinks False
+```