mo137 commited on
Commit
145d029
1 Parent(s): 0a44ff2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +24 -0
README.md CHANGED
@@ -1,3 +1,27 @@
1
  ---
2
  license: cc-by-nc-4.0
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: cc-by-nc-4.0
3
+ tags:
4
+ - exllamav2
5
+ - exl2
6
+ - Text Generation
7
+ - not-for-all-audiences
8
+ - nsfw
9
+ - Transformers
10
+ - llama
11
+ - text-generation-inference
12
  ---
13
+
14
+ # Amethyst 13B Mistral - EXL2 - 3 bpw
15
+ - Model creator: [Undi](https://huggingface.co/Undi95)
16
+ - Original model: [Amethyst 13B Mistral](https://huggingface.co/Undi95/Amethyst-13B-Mistral)
17
+
18
+ ## Description
19
+ - 3 bits per weight.
20
+
21
+ I converted the model using the convert.py script from the exllamav2 repo:
22
+ https://github.com/turboderp/exllamav2
23
+ Its documentation:
24
+ https://github.com/turboderp/exllamav2/blob/master/doc/convert.md
25
+
26
+ I used the WikiText-2-v1 dataset for calibration:
27
+ https://huggingface.co/datasets/wikitext/blob/refs%2Fconvert%2Fparquet/wikitext-2-v1/test/0000.parquet