mradermacher
commited on
Commit
•
ac67cbd
1
Parent(s):
adff976
auto-patch README.md
Browse files
README.md
CHANGED
@@ -33,10 +33,14 @@ more details, including on how to concatenate multi-part files.
|
|
33 |
|
34 |
| Link | Type | Size/GB | Notes |
|
35 |
|:-----|:-----|--------:|:------|
|
|
|
36 |
| [P1](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q2_K.gguf.part01of13) [P2](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q2_K.gguf.part02of13) [P3](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q2_K.gguf.part03of13) [P4](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q2_K.gguf.part04of13) [P5](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q2_K.gguf.part05of13) [P6](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q2_K.gguf.part06of13) [P7](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q2_K.gguf.part07of13) [P8](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q2_K.gguf.part08of13) [P9](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q2_K.gguf.part09of13) [P10](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q2_K.gguf.part10of13) [P11](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q2_K.gguf.part11of13) [P12](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q2_K.gguf.part12of13) [P13](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q2_K.gguf.part13of13) | i1-Q2_K | 622.5 | IQ3_XXS probably better |
|
37 |
| [P1](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_XXS.gguf.part01of16) [P2](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_XXS.gguf.part02of16) [P3](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_XXS.gguf.part03of16) [P4](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_XXS.gguf.part04of16) [P5](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_XXS.gguf.part05of16) [P6](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_XXS.gguf.part06of16) [P7](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_XXS.gguf.part07of16) [P8](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_XXS.gguf.part08of16) [P9](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_XXS.gguf.part09of16) [P10](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_XXS.gguf.part10of16) [P11](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_XXS.gguf.part11of16) [P12](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_XXS.gguf.part12of16) [P13](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_XXS.gguf.part13of16) [P14](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_XXS.gguf.part14of16) [P15](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_XXS.gguf.part15of16) [P16](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_XXS.gguf.part16of16) | i1-IQ3_XXS | 649.8 | lower quality |
|
38 |
| [P1](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part01of17) [P2](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part02of17) [P3](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part03of17) [P4](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part04of17) [P5](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part05of17) [P6](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part06of17) [P7](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part07of17) [P8](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part08of17) [P9](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part09of17) [P10](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part10of17) [P11](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part11of17) [P12](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part12of17) [P13](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part13of17) [P14](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part14of17) [P15](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part15of17) [P16](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part16of17) [P17](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part17of17) | i1-IQ3_M | 758.1 | |
|
|
|
39 |
| [P1](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part01of20) [P2](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part02of20) [P3](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part03of20) [P4](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part04of20) [P5](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part05of20) [P6](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part06of20) [P7](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part07of20) [P8](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part08of20) [P9](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part09of20) [P10](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part10of20) [P11](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part11of20) [P12](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part12of20) [P13](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part13of20) [P14](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part14of20) [P15](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part15of20) [P16](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part16of20) [P17](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part17of20) [P18](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part18of20) [P19](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part19of20) [P20](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part20of20) | i1-Q4_K_S | 961.5 | optimal size/speed/quality |
|
|
|
|
|
40 |
|
41 |
Here is a handy graph by ikawrakow comparing some lower-quality quant
|
42 |
types (lower is better):
|
|
|
33 |
|
34 |
| Link | Type | Size/GB | Notes |
|
35 |
|:-----|:-----|--------:|:------|
|
36 |
+
| [P1](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ2_M.gguf.part01of13) [P2](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ2_M.gguf.part02of13) [P3](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ2_M.gguf.part03of13) [P4](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ2_M.gguf.part04of13) [P5](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ2_M.gguf.part05of13) [P6](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ2_M.gguf.part06of13) [P7](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ2_M.gguf.part07of13) [P8](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ2_M.gguf.part08of13) [P9](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ2_M.gguf.part09of13) [P10](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ2_M.gguf.part10of13) [P11](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ2_M.gguf.part11of13) [P12](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ2_M.gguf.part12of13) [P13](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ2_M.gguf.part13of13) | i1-IQ2_M | 570.1 | |
|
37 |
| [P1](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q2_K.gguf.part01of13) [P2](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q2_K.gguf.part02of13) [P3](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q2_K.gguf.part03of13) [P4](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q2_K.gguf.part04of13) [P5](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q2_K.gguf.part05of13) [P6](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q2_K.gguf.part06of13) [P7](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q2_K.gguf.part07of13) [P8](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q2_K.gguf.part08of13) [P9](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q2_K.gguf.part09of13) [P10](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q2_K.gguf.part10of13) [P11](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q2_K.gguf.part11of13) [P12](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q2_K.gguf.part12of13) [P13](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q2_K.gguf.part13of13) | i1-Q2_K | 622.5 | IQ3_XXS probably better |
|
38 |
| [P1](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_XXS.gguf.part01of16) [P2](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_XXS.gguf.part02of16) [P3](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_XXS.gguf.part03of16) [P4](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_XXS.gguf.part04of16) [P5](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_XXS.gguf.part05of16) [P6](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_XXS.gguf.part06of16) [P7](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_XXS.gguf.part07of16) [P8](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_XXS.gguf.part08of16) [P9](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_XXS.gguf.part09of16) [P10](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_XXS.gguf.part10of16) [P11](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_XXS.gguf.part11of16) [P12](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_XXS.gguf.part12of16) [P13](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_XXS.gguf.part13of16) [P14](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_XXS.gguf.part14of16) [P15](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_XXS.gguf.part15of16) [P16](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_XXS.gguf.part16of16) | i1-IQ3_XXS | 649.8 | lower quality |
|
39 |
| [P1](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part01of17) [P2](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part02of17) [P3](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part03of17) [P4](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part04of17) [P5](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part05of17) [P6](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part06of17) [P7](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part07of17) [P8](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part08of17) [P9](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part09of17) [P10](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part10of17) [P11](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part11of17) [P12](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part12of17) [P13](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part13of17) [P14](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part14of17) [P15](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part15of17) [P16](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part16of17) [P17](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-IQ3_M.gguf.part17of17) | i1-IQ3_M | 758.1 | |
|
40 |
+
| [P1](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q3_K_M.gguf.part01of17) [P2](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q3_K_M.gguf.part02of17) [P3](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q3_K_M.gguf.part03of17) [P4](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q3_K_M.gguf.part04of17) [P5](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q3_K_M.gguf.part05of17) [P6](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q3_K_M.gguf.part06of17) [P7](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q3_K_M.gguf.part07of17) [P8](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q3_K_M.gguf.part08of17) [P9](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q3_K_M.gguf.part09of17) [P10](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q3_K_M.gguf.part10of17) [P11](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q3_K_M.gguf.part11of17) [P12](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q3_K_M.gguf.part12of17) [P13](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q3_K_M.gguf.part13of17) [P14](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q3_K_M.gguf.part14of17) [P15](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q3_K_M.gguf.part15of17) [P16](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q3_K_M.gguf.part16of17) [P17](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q3_K_M.gguf.part17of17) | i1-Q3_K_M | 814.9 | IQ3_S probably better |
|
41 |
| [P1](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part01of20) [P2](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part02of20) [P3](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part03of20) [P4](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part04of20) [P5](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part05of20) [P6](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part06of20) [P7](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part07of20) [P8](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part08of20) [P9](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part09of20) [P10](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part10of20) [P11](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part11of20) [P12](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part12of20) [P13](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part13of20) [P14](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part14of20) [P15](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part15of20) [P16](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part16of20) [P17](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part17of20) [P18](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part18of20) [P19](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part19of20) [P20](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_S.gguf.part20of20) | i1-Q4_K_S | 961.5 | optimal size/speed/quality |
|
42 |
+
| [P1](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_M.gguf.part01of22) [P2](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_M.gguf.part02of22) [P3](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_M.gguf.part03of22) [P4](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_M.gguf.part04of22) [P5](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_M.gguf.part05of22) [P6](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_M.gguf.part06of22) [P7](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_M.gguf.part07of22) [P8](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_M.gguf.part08of22) [P9](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_M.gguf.part09of22) [P10](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_M.gguf.part10of22) [P11](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_M.gguf.part11of22) [P12](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_M.gguf.part12of22) [P13](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_M.gguf.part13of22) [P14](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_M.gguf.part14of22) [P15](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_M.gguf.part15of22) [P16](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_M.gguf.part16of22) [P17](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_M.gguf.part17of22) [P18](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_M.gguf.part18of22) [P19](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_M.gguf.part19of22) [P20](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_M.gguf.part20of22) [P21](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_M.gguf.part21of22) [P22](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q4_K_M.gguf.part22of22) | i1-Q4_K_M | 1015.5 | fast, recommended |
|
43 |
+
| [P1](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q6_K.gguf.part01of29) [P2](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q6_K.gguf.part02of29) [P3](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q6_K.gguf.part03of29) [P4](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q6_K.gguf.part04of29) [P5](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q6_K.gguf.part05of29) [P6](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q6_K.gguf.part06of29) [P7](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q6_K.gguf.part07of29) [P8](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q6_K.gguf.part08of29) [P9](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q6_K.gguf.part09of29) [P10](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q6_K.gguf.part10of29) [P11](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q6_K.gguf.part11of29) [P12](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q6_K.gguf.part12of29) [P13](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q6_K.gguf.part13of29) [P14](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q6_K.gguf.part14of29) [P15](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q6_K.gguf.part15of29) [P16](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q6_K.gguf.part16of29) [P17](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q6_K.gguf.part17of29) [P18](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q6_K.gguf.part18of29) [P19](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q6_K.gguf.part19of29) [P20](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q6_K.gguf.part20of29) [P21](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q6_K.gguf.part21of29) [P22](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q6_K.gguf.part22of29) [P23](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q6_K.gguf.part23of29) [P24](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q6_K.gguf.part24of29) [P25](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q6_K.gguf.part25of29) [P26](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q6_K.gguf.part26of29) [P27](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q6_K.gguf.part27of29) [P28](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q6_K.gguf.part28of29) [P29](https://huggingface.co/mradermacher/FATLLAMA-1.7T-Instruct-i1-GGUF/resolve/main/FATLLAMA-1.7T-Instruct.i1-Q6_K.gguf.part29of29) | i1-Q6_K | 1390.9 | practically like static Q6_K |
|
44 |
|
45 |
Here is a handy graph by ikawrakow comparing some lower-quality quant
|
46 |
types (lower is better):
|