Update README.md
Browse files
README.md
CHANGED
|
@@ -10,6 +10,7 @@ tags:
|
|
| 10 |
- distillation
|
| 11 |
- dpo
|
| 12 |
- rlhf
|
|
|
|
| 13 |
license: apache-2.0
|
| 14 |
language:
|
| 15 |
- en
|
|
@@ -17,6 +18,20 @@ datasets:
|
|
| 17 |
- mlabonne/chatml_dpo_pairs
|
| 18 |
---
|
| 19 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 20 |
<center><img src="https://i.imgur.com/qIhaFNM.png"></center>
|
| 21 |
|
| 22 |
# NeuralHermes 2.5 - Mistral 7B
|
|
|
|
| 10 |
- distillation
|
| 11 |
- dpo
|
| 12 |
- rlhf
|
| 13 |
+
- exl2
|
| 14 |
license: apache-2.0
|
| 15 |
language:
|
| 16 |
- en
|
|
|
|
| 18 |
- mlabonne/chatml_dpo_pairs
|
| 19 |
---
|
| 20 |
|
| 21 |
+
EXL2 quantisation of NeuralHermes-2.5-Mistral-7B, for use with ExLLamaV2.
|
| 22 |
+
|
| 23 |
+
[Original model](https://huggingface.co/mlabonne/NeuralHermes-2.5-Mistral-7B) by @mlabonne.
|
| 24 |
+
|
| 25 |
+
**Model size:** 4.6GB (3x reduction), 5 bits-per-weight average, 6bpw on head.
|
| 26 |
+
|
| 27 |
+
**Calibration Data:** Wikitext [(parquet)](https://huggingface.co/datasets/wikitext/blob/refs%2Fconvert%2Fparquet/wikitext-2-v1/train/0000.parquet)
|
| 28 |
+
|
| 29 |
+
**Command:** `python convert.py -i convert/NeuralHermes-2.5-Mistral-7B -c convert/0000.parquet -o convert/temp2 -cf convert/nh-5bpw -b 5.0 -hb 6`
|
| 30 |
+
|
| 31 |
+
Layer measurements are provided in `measurement.json`` for further quantisation.
|
| 32 |
+
|
| 33 |
+
---
|
| 34 |
+
|
| 35 |
<center><img src="https://i.imgur.com/qIhaFNM.png"></center>
|
| 36 |
|
| 37 |
# NeuralHermes 2.5 - Mistral 7B
|