Update README.md
Browse filesCredit turboderp and the bloke
README.md
CHANGED
@@ -6,6 +6,8 @@ license: cc-by-nc-4.0
|
|
6 |
|
7 |
## Exllama v2 Quantizations of Mistral-7B-claude-chat
|
8 |
|
|
|
|
|
9 |
Each branches contains an individual bits per weight.
|
10 |
|
11 |
Conversion was done using wikitext.parquet as calibration dataset.
|
@@ -24,7 +26,7 @@ With git:
|
|
24 |
git clone --single-branch --branch 4.0 https://huggingface.co/bartowski/Mistral-7B-claude-chat-exl2
|
25 |
```
|
26 |
|
27 |
-
With huggingface hub:
|
28 |
|
29 |
```shell
|
30 |
pip3 install huggingface-hub
|
|
|
6 |
|
7 |
## Exllama v2 Quantizations of Mistral-7B-claude-chat
|
8 |
|
9 |
+
Using <a href="https://github.com/turboderp/exllamav2/releases/tag/v0.0.6">turboderp's ExLlamaV2 v0.0.6</a> for quantization.
|
10 |
+
|
11 |
Each branches contains an individual bits per weight.
|
12 |
|
13 |
Conversion was done using wikitext.parquet as calibration dataset.
|
|
|
26 |
git clone --single-branch --branch 4.0 https://huggingface.co/bartowski/Mistral-7B-claude-chat-exl2
|
27 |
```
|
28 |
|
29 |
+
With huggingface hub (credit to TheBloke for instructions):
|
30 |
|
31 |
```shell
|
32 |
pip3 install huggingface-hub
|