Commit
•
4b20da9
1
Parent(s):
d904dec
Update README.md
Browse files
README.md
CHANGED
@@ -33,18 +33,16 @@ dtype: float16
|
|
33 |
```
|
34 |
|
35 |
First exllama quantization pass:
|
36 |
-
|
37 |
```
|
38 |
python convert.py --in_dir /home/alpha/FastModels/Capybara-Tess-Yi-34B-200K -o /home/alpha/FastModels/Capybara-Tess-Yi-34B-200K-exl2 -om /home/alpha/FastModels/capytessmes.json --cal_dataset /home/alpha/Documents/smol.parquet -l 2048 -r 80 -ml 2048 -mr 40 -gr 40 -ss 4096 -nr -b 3.5 -hb 6
|
39 |
```
|
40 |
|
41 |
Second exllama quantization pass:
|
42 |
-
|
43 |
```
|
44 |
python convert.py --in_dir /home/alpha/FastModels/Capybara-Tess-Yi-34B-200K -o /home/alpha/FastModels/Capybara-Tess-Yi-34B-200K-exl2 -m /home/alpha/FastModels/capytessmes.json --cal_dataset /home/alpha/Documents/medium.parquet -l 2048 -r 200 -ml 2048 -mr 40 -gr 200 -ss 4096 -b 3.1 -hb 6 -cf /home/alpha/FastModels/Capybara-Tess-Yi-34B-200K-exl2-31bpw -nr
|
45 |
```
|
46 |
|
47 |
-
Both
|
48 |
|
49 |
# Prompt Format:
|
50 |
|
|
|
33 |
```
|
34 |
|
35 |
First exllama quantization pass:
|
|
|
36 |
```
|
37 |
python convert.py --in_dir /home/alpha/FastModels/Capybara-Tess-Yi-34B-200K -o /home/alpha/FastModels/Capybara-Tess-Yi-34B-200K-exl2 -om /home/alpha/FastModels/capytessmes.json --cal_dataset /home/alpha/Documents/smol.parquet -l 2048 -r 80 -ml 2048 -mr 40 -gr 40 -ss 4096 -nr -b 3.5 -hb 6
|
38 |
```
|
39 |
|
40 |
Second exllama quantization pass:
|
|
|
41 |
```
|
42 |
python convert.py --in_dir /home/alpha/FastModels/Capybara-Tess-Yi-34B-200K -o /home/alpha/FastModels/Capybara-Tess-Yi-34B-200K-exl2 -m /home/alpha/FastModels/capytessmes.json --cal_dataset /home/alpha/Documents/medium.parquet -l 2048 -r 200 -ml 2048 -mr 40 -gr 200 -ss 4096 -b 3.1 -hb 6 -cf /home/alpha/FastModels/Capybara-Tess-Yi-34B-200K-exl2-31bpw -nr
|
43 |
```
|
44 |
|
45 |
+
Both models have Vicuna syntax, so:
|
46 |
|
47 |
# Prompt Format:
|
48 |
|