Hugging Face's logo

medmekk
/
Llama-3.2-1B-Instruct.GGUF

GGUF
Inference Endpoints
imatrix
conversational
Model card Files Community
medmekk HF staff commited on
Commit
52fc2aa
·
verified ·
1 Parent(s): 5f6cb1b

Upload quantized models

Browse files
Files changed (22) hide show
  1. .gitattributes +20 -0
  2. Llama-3.2-1B-Instruct-IQ3_M_imat.gguf +3 -0
  3. Llama-3.2-1B-Instruct-IQ3_XXS_imat.gguf +3 -0
  4. Llama-3.2-1B-Instruct-IQ4_NL_imat.gguf +3 -0
  5. Llama-3.2-1B-Instruct-IQ4_XS_imat.gguf +3 -0
  6. Llama-3.2-1B-Instruct-Q2_K.gguf +3 -0
  7. Llama-3.2-1B-Instruct-Q3_K_L.gguf +3 -0
  8. Llama-3.2-1B-Instruct-Q3_K_M.gguf +3 -0
  9. Llama-3.2-1B-Instruct-Q3_K_S.gguf +3 -0
  10. Llama-3.2-1B-Instruct-Q4_0.gguf +3 -0
  11. Llama-3.2-1B-Instruct-Q4_K_M.gguf +3 -0
  12. Llama-3.2-1B-Instruct-Q4_K_M_imat.gguf +3 -0
  13. Llama-3.2-1B-Instruct-Q4_K_S.gguf +3 -0
  14. Llama-3.2-1B-Instruct-Q4_K_S_imat.gguf +3 -0
  15. Llama-3.2-1B-Instruct-Q5_0.gguf +3 -0
  16. Llama-3.2-1B-Instruct-Q5_K_M.gguf +3 -0
  17. Llama-3.2-1B-Instruct-Q5_K_M_imat.gguf +3 -0
  18. Llama-3.2-1B-Instruct-Q5_K_S.gguf +3 -0
  19. Llama-3.2-1B-Instruct-Q5_K_S_imat.gguf +3 -0
  20. Llama-3.2-1B-Instruct-Q6_K.gguf +3 -0
  21. Llama-3.2-1B-Instruct-Q8_0.gguf +3 -0
  22. README.md +34 -0
.gitattributes CHANGED
@@ -33,3 +33,23 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ Llama-3.2-1B-Instruct-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
37
+ Llama-3.2-1B-Instruct-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
38
+ Llama-3.2-1B-Instruct-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
39
+ Llama-3.2-1B-Instruct-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
40
+ Llama-3.2-1B-Instruct-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
41
+ Llama-3.2-1B-Instruct-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
42
+ Llama-3.2-1B-Instruct-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
43
+ Llama-3.2-1B-Instruct-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
44
+ Llama-3.2-1B-Instruct-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
45
+ Llama-3.2-1B-Instruct-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
46
+ Llama-3.2-1B-Instruct-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
47
+ Llama-3.2-1B-Instruct-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
48
+ Llama-3.2-1B-Instruct-IQ3_M_imat.gguf filter=lfs diff=lfs merge=lfs -text
49
+ Llama-3.2-1B-Instruct-IQ3_XXS_imat.gguf filter=lfs diff=lfs merge=lfs -text
50
+ Llama-3.2-1B-Instruct-Q4_K_M_imat.gguf filter=lfs diff=lfs merge=lfs -text
51
+ Llama-3.2-1B-Instruct-Q4_K_S_imat.gguf filter=lfs diff=lfs merge=lfs -text
52
+ Llama-3.2-1B-Instruct-IQ4_NL_imat.gguf filter=lfs diff=lfs merge=lfs -text
53
+ Llama-3.2-1B-Instruct-IQ4_XS_imat.gguf filter=lfs diff=lfs merge=lfs -text
54
+ Llama-3.2-1B-Instruct-Q5_K_M_imat.gguf filter=lfs diff=lfs merge=lfs -text
55
+ Llama-3.2-1B-Instruct-Q5_K_S_imat.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.2-1B-Instruct-IQ3_M_imat.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:848b4fb777c026a2eae3e1033734d620b2772982ea9dc905ee8f8bcf594a407b
3
+ size 657289280
Llama-3.2-1B-Instruct-IQ3_XXS_imat.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7216551ff66ca41612412f62ab8db58abac017933ed9934d12345a3eeedc9950
3
+ size 562110528
Llama-3.2-1B-Instruct-IQ4_NL_imat.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2eda584fdcb4067620b9b11079a154f11cad9c734ff77011b945d1effa8c259c
3
+ size 773025856
Llama-3.2-1B-Instruct-IQ4_XS_imat.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:15f8c3ec48edf71a67b6096336fee7b77221421022aae1aa21b39616ea227531
3
+ size 743141440
Llama-3.2-1B-Instruct-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a870730e0fff8e6c4656e4f611faa16bde7b16f33c059292167a174bf15e5b2e
3
+ size 580874080
Llama-3.2-1B-Instruct-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ef44bca078435843c7788f9c03e7b495a099c131f845ed19dc0257e561e8caf3
3
+ size 732524384
Llama-3.2-1B-Instruct-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7b36f7efe910829de87ab1fcd9a55fdb0bf481d8e158345b13dc48486a324a79
3
+ size 690843488
Llama-3.2-1B-Instruct-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1444f1e3d528823058010ea5152b8f32a332f9bce37290d480c4d4e50b04eb41
3
+ size 641691488
Llama-3.2-1B-Instruct-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:eadfd8fd4e29d48e720eb87fc8242d3a8d4d2dacd52c722adc8e69e48c668efc
3
+ size 770928480
Llama-3.2-1B-Instruct-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:26bac8efd811cb41a80db4393dbe5c8360abd54b98954ec766aa4ba7dacc0bc5
3
+ size 807694176
Llama-3.2-1B-Instruct-Q4_K_M_imat.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a9a93ed1c1df05b83a365b2a1ca429657df063e642fde9027e54cbf07137d5e9
3
+ size 807694400
Llama-3.2-1B-Instruct-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5550376826ef08901a4145559647844d5e70a950d69145e83ae2d262ce5ce0e2
3
+ size 775647072
Llama-3.2-1B-Instruct-Q4_K_S_imat.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cdac3f90911b2c9e171af128787e7ce6a769fb2a64a90f5b2597db97ac1623b6
3
+ size 775647296
Llama-3.2-1B-Instruct-Q5_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f6aea9bca54d1b5033035771963e0bf24d307ff756ab476744a2b43ad2eeb68d
3
+ size 892563296
Llama-3.2-1B-Instruct-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7f5165ccccbd6953de2a35ca56300ebedfb70739a407dd096e3a5c658477aefa
3
+ size 911503200
Llama-3.2-1B-Instruct-Q5_K_M_imat.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ea756771308d485601da96d5ce6f62745888ea84fbc8d4b6bb0f3b214816a83b
3
+ size 911503424
Llama-3.2-1B-Instruct-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a14d69cc881f282405b8ba59ab6377a3eb7f2c3686077d1be796c87f6298c398
3
+ size 892563296
Llama-3.2-1B-Instruct-Q5_K_S_imat.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:852c37accb67fad10d2ae9ab8e36338b5365dbf43b039672534a4768f1c8905b
3
+ size 892563520
Llama-3.2-1B-Instruct-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4bf385159856b7c50a938b1228112318d9f99238a76880ea0f6381ab879982b3
3
+ size 1021800288
Llama-3.2-1B-Instruct-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:da49f51ced8c15546e7779beb677fb53eb5d0b3b38ac4607ac60d58d77074823
3
+ size 1321082720
README.md ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ # medmekk/Llama-3.2-1B-Instruct.GGUF
3
+ GGUF quantized versions of [meta-llama/Llama-3.2-1B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct)
4
+
5
+ ## Available Formats:
6
+ - `Q2_K`: Llama-3.2-1B-Instruct-Q2_K.gguf
7
+ - `Q3_K_S`: Llama-3.2-1B-Instruct-Q3_K_S.gguf
8
+ - `Q3_K_M`: Llama-3.2-1B-Instruct-Q3_K_M.gguf
9
+ - `Q3_K_L`: Llama-3.2-1B-Instruct-Q3_K_L.gguf
10
+ - `Q4_0`: Llama-3.2-1B-Instruct-Q4_0.gguf
11
+ - `Q4_K_S`: Llama-3.2-1B-Instruct-Q4_K_S.gguf
12
+ - `Q4_K_M`: Llama-3.2-1B-Instruct-Q4_K_M.gguf
13
+ - `Q5_0`: Llama-3.2-1B-Instruct-Q5_0.gguf
14
+ - `Q5_K_S`: Llama-3.2-1B-Instruct-Q5_K_S.gguf
15
+ - `Q5_K_M`: Llama-3.2-1B-Instruct-Q5_K_M.gguf
16
+ - `Q6_K`: Llama-3.2-1B-Instruct-Q6_K.gguf
17
+ - `Q8_0`: Llama-3.2-1B-Instruct-Q8_0.gguf
18
+ - `IQ3_M_IMAT`: Llama-3.2-1B-Instruct-IQ3_M_imat.gguf
19
+ - `IQ3_XXS_IMAT`: Llama-3.2-1B-Instruct-IQ3_XXS_imat.gguf
20
+ - `Q4_K_M_IMAT`: Llama-3.2-1B-Instruct-Q4_K_M_imat.gguf
21
+ - `Q4_K_S_IMAT`: Llama-3.2-1B-Instruct-Q4_K_S_imat.gguf
22
+ - `IQ4_NL_IMAT`: Llama-3.2-1B-Instruct-IQ4_NL_imat.gguf
23
+ - `IQ4_XS_IMAT`: Llama-3.2-1B-Instruct-IQ4_XS_imat.gguf
24
+ - `Q5_K_M_IMAT`: Llama-3.2-1B-Instruct-Q5_K_M_imat.gguf
25
+ - `Q5_K_S_IMAT`: Llama-3.2-1B-Instruct-Q5_K_S_imat.gguf
26
+
27
+ ## Usage with llama.cpp:
28
+ ```bash
29
+ # CLI:
30
+ llama-cli --hf-repo medmekk/Llama-3.2-1B-Instruct.GGUF --hf-file MODEL_FILE -p "Your prompt"
31
+
32
+ # Server:
33
+ llama-server --hf-repo medmekk/Llama-3.2-1B-Instruct.GGUF --hf-file MODEL_FILE -c 2048
34
+ ```