Commit
•
f6d1b96
1
Parent(s):
1b192a7
Update README.md
Browse files
README.md
CHANGED
|
@@ -22,6 +22,8 @@ imatrix was run with default sampling settings besides the dataset (i think? i i
|
|
| 22 |
```
|
| 23 |
imatrix -m Kyllene-57B-v1.0.q6_K.gguf -f ~/exltabbytorcher220/exllamav2/conversion/standard_cal_data/techmulcodetiny.utf8 -o Kyllene-57B-v1.0.q6_K.gguf.imat --verbosity 1 -ngl 50 -cb -t 3 -b 256 --no_mmap
|
| 24 |
```
|
|
|
|
|
|
|
| 25 |
|
| 26 |
how-to because i'm grouchy but I did actually want people to have these. Remember to replace IQ2_M (appears only twice, near the end) with whatever you fancy. Q2_K might be more compatible.
|
| 27 |
```
|
|
|
|
| 22 |
```
|
| 23 |
imatrix -m Kyllene-57B-v1.0.q6_K.gguf -f ~/exltabbytorcher220/exllamav2/conversion/standard_cal_data/techmulcodetiny.utf8 -o Kyllene-57B-v1.0.q6_K.gguf.imat --verbosity 1 -ngl 50 -cb -t 3 -b 256 --no_mmap
|
| 24 |
```
|
| 25 |
+
51 layers was too many on a 3090 and I had to kill wayland (pro tip: tmux). needless to say you'll probably die if you tried something idiotic like using this on windows
|
| 26 |
+
--no_mmap was appropriate on my nigtmare vortex of 32GB DDR4, layered swap,tiny zrams and weird kernel parameters but maybe just omit it.
|
| 27 |
|
| 28 |
how-to because i'm grouchy but I did actually want people to have these. Remember to replace IQ2_M (appears only twice, near the end) with whatever you fancy. Q2_K might be more compatible.
|
| 29 |
```
|