brucethemoose commited on
Commit
4b20da9
1 Parent(s): d904dec

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -3
README.md CHANGED
@@ -33,18 +33,16 @@ dtype: float16
33
  ```
34
 
35
  First exllama quantization pass:
36
-
37
  ```
38
  python convert.py --in_dir /home/alpha/FastModels/Capybara-Tess-Yi-34B-200K -o /home/alpha/FastModels/Capybara-Tess-Yi-34B-200K-exl2 -om /home/alpha/FastModels/capytessmes.json --cal_dataset /home/alpha/Documents/smol.parquet -l 2048 -r 80 -ml 2048 -mr 40 -gr 40 -ss 4096 -nr -b 3.5 -hb 6
39
  ```
40
 
41
  Second exllama quantization pass:
42
-
43
  ```
44
  python convert.py --in_dir /home/alpha/FastModels/Capybara-Tess-Yi-34B-200K -o /home/alpha/FastModels/Capybara-Tess-Yi-34B-200K-exl2 -m /home/alpha/FastModels/capytessmes.json --cal_dataset /home/alpha/Documents/medium.parquet -l 2048 -r 200 -ml 2048 -mr 40 -gr 200 -ss 4096 -b 3.1 -hb 6 -cf /home/alpha/FastModels/Capybara-Tess-Yi-34B-200K-exl2-31bpw -nr
45
  ```
46
 
47
- Both are 200K context models with Vicuna syntax, so:
48
 
49
  # Prompt Format:
50
 
 
33
  ```
34
 
35
  First exllama quantization pass:
 
36
  ```
37
  python convert.py --in_dir /home/alpha/FastModels/Capybara-Tess-Yi-34B-200K -o /home/alpha/FastModels/Capybara-Tess-Yi-34B-200K-exl2 -om /home/alpha/FastModels/capytessmes.json --cal_dataset /home/alpha/Documents/smol.parquet -l 2048 -r 80 -ml 2048 -mr 40 -gr 40 -ss 4096 -nr -b 3.5 -hb 6
38
  ```
39
 
40
  Second exllama quantization pass:
 
41
  ```
42
  python convert.py --in_dir /home/alpha/FastModels/Capybara-Tess-Yi-34B-200K -o /home/alpha/FastModels/Capybara-Tess-Yi-34B-200K-exl2 -m /home/alpha/FastModels/capytessmes.json --cal_dataset /home/alpha/Documents/medium.parquet -l 2048 -r 200 -ml 2048 -mr 40 -gr 200 -ss 4096 -b 3.1 -hb 6 -cf /home/alpha/FastModels/Capybara-Tess-Yi-34B-200K-exl2-31bpw -nr
43
  ```
44
 
45
+ Both models have Vicuna syntax, so:
46
 
47
  # Prompt Format:
48