Hugging Face's logo

bartowski
/
Starling-LM-7B-beta-GGUF

Text Generation
Transformers
GGUF
English
reward model
RLHF
RLAIF
Inference Endpoints
Model card Files Community
bartowski commited on
Commit
d205a18
•
1 Parent(s): 8f55051

Llamacpp quants

Browse files
Files changed (18) hide show
  1. .gitattributes +16 -0
  2. README.md +43 -0
  3. Starling-LM-7B-beta-IQ3_M.gguf +3 -0
  4. Starling-LM-7B-beta-IQ3_S.gguf +3 -0
  5. Starling-LM-7B-beta-IQ4_NL.gguf +3 -0
  6. Starling-LM-7B-beta-IQ4_XS.gguf +3 -0
  7. Starling-LM-7B-beta-Q2_K.gguf +3 -0
  8. Starling-LM-7B-beta-Q3_K_L.gguf +3 -0
  9. Starling-LM-7B-beta-Q3_K_M.gguf +3 -0
  10. Starling-LM-7B-beta-Q3_K_S.gguf +3 -0
  11. Starling-LM-7B-beta-Q4_0.gguf +3 -0
  12. Starling-LM-7B-beta-Q4_K_M.gguf +3 -0
  13. Starling-LM-7B-beta-Q4_K_S.gguf +3 -0
  14. Starling-LM-7B-beta-Q5_0.gguf +3 -0
  15. Starling-LM-7B-beta-Q5_K_M.gguf +3 -0
  16. Starling-LM-7B-beta-Q5_K_S.gguf +3 -0
  17. Starling-LM-7B-beta-Q6_K.gguf +3 -0
  18. Starling-LM-7B-beta-Q8_0.gguf +3 -0
.gitattributes CHANGED
@@ -33,3 +33,19 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ Starling-LM-7B-beta-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
37
+ Starling-LM-7B-beta-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
38
+ Starling-LM-7B-beta-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
39
+ Starling-LM-7B-beta-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
40
+ Starling-LM-7B-beta-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
41
+ Starling-LM-7B-beta-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
42
+ Starling-LM-7B-beta-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
43
+ Starling-LM-7B-beta-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
44
+ Starling-LM-7B-beta-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
45
+ Starling-LM-7B-beta-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
46
+ Starling-LM-7B-beta-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
47
+ Starling-LM-7B-beta-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
48
+ Starling-LM-7B-beta-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
49
+ Starling-LM-7B-beta-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
50
+ Starling-LM-7B-beta-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
51
+ Starling-LM-7B-beta-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,43 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ datasets:
4
+ - berkeley-nest/Nectar
5
+ language:
6
+ - en
7
+ library_name: transformers
8
+ tags:
9
+ - reward model
10
+ - RLHF
11
+ - RLAIF
12
+ quantized_by: bartowski
13
+ pipeline_tag: text-generation
14
+ ---
15
+
16
+ ## Llamacpp Quantizations of Starling-LM-7B-beta
17
+
18
+ Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b2440">b2440</a> for quantization.
19
+
20
+ Original model: https://huggingface.co/Nexusflow/Starling-LM-7B-beta
21
+
22
+ Download a file (not the whole branch) from below:
23
+
24
+ | Filename | Quant type | File Size | Description |
25
+ | -------- | ---------- | --------- | ----------- |
26
+ | [Starling-LM-7B-beta-Q8_0.gguf](https://huggingface.co/bartowski/Starling-LM-7B-beta-GGUF/blob/main/Starling-LM-7B-beta-Q8_0.gguf) | Q8_0 | 7.69GB | Extremely high quality, generally unneeded but max available quant. |
27
+ | [Starling-LM-7B-beta-Q6_K.gguf](https://huggingface.co/bartowski/Starling-LM-7B-beta-GGUF/blob/main/Starling-LM-7B-beta-Q6_K.gguf) | Q6_K | 5.94GB | Very high quality, near perfect, *recommended*. |
28
+ | [Starling-LM-7B-beta-Q5_K_M.gguf](https://huggingface.co/bartowski/Starling-LM-7B-beta-GGUF/blob/main/Starling-LM-7B-beta-Q5_K_M.gguf) | Q5_K_M | 5.13GB | High quality, very usable. |
29
+ | [Starling-LM-7B-beta-Q5_K_S.gguf](https://huggingface.co/bartowski/Starling-LM-7B-beta-GGUF/blob/main/Starling-LM-7B-beta-Q5_K_S.gguf) | Q5_K_S | 4.99GB | High quality, very usable. |
30
+ | [Starling-LM-7B-beta-Q5_0.gguf](https://huggingface.co/bartowski/Starling-LM-7B-beta-GGUF/blob/main/Starling-LM-7B-beta-Q5_0.gguf) | Q5_0 | 4.99GB | High quality, older format, generally not recommended. |
31
+ | [Starling-LM-7B-beta-Q4_K_M.gguf](https://huggingface.co/bartowski/Starling-LM-7B-beta-GGUF/blob/main/Starling-LM-7B-beta-Q4_K_M.gguf) | Q4_K_M | 4.36GB | Good quality, similar to 4.25 bpw. |
32
+ | [Starling-LM-7B-beta-Q4_K_S.gguf](https://huggingface.co/bartowski/Starling-LM-7B-beta-GGUF/blob/main/Starling-LM-7B-beta-Q4_K_S.gguf) | Q4_K_S | 4.14GB | Slightly lower quality with small space savings. |
33
+ | [Starling-LM-7B-beta-IQ4_NL.gguf](https://huggingface.co/bartowski/Starling-LM-7B-beta-GGUF/blob/main/Starling-LM-7B-beta-IQ4_NL.gguf) | IQ4_NL | 4.15GB | Good quality, similar to Q4_K_S, new method of quanting, |
34
+ | [Starling-LM-7B-beta-IQ4_XS.gguf](https://huggingface.co/bartowski/Starling-LM-7B-beta-GGUF/blob/main/Starling-LM-7B-beta-IQ4_XS.gguf) | IQ4_XS | 3.94GB | Decent quality, new method with similar performance to Q4. |
35
+ | [Starling-LM-7B-beta-Q4_0.gguf](https://huggingface.co/bartowski/Starling-LM-7B-beta-GGUF/blob/main/Starling-LM-7B-beta-Q4_0.gguf) | Q4_0 | 4.10GB | Decent quality, older format, generally not recommended. |
36
+ | [Starling-LM-7B-beta-IQ3_M.gguf](https://huggingface.co/bartowski/Starling-LM-7B-beta-GGUF/blob/main/Starling-LM-7B-beta-IQ3_M.gguf) | IQ3_M | 3.28GB | Medium-low quality, new method with decent performance. |
37
+ | [Starling-LM-7B-beta-IQ3_S.gguf](https://huggingface.co/bartowski/Starling-LM-7B-beta-GGUF/blob/main/Starling-LM-7B-beta-IQ3_S.gguf) | IQ3_S | 3.18GB | Lower quality, new method with decent performance, recommended over Q3 quants. |
38
+ | [Starling-LM-7B-beta-Q3_K_L.gguf](https://huggingface.co/bartowski/Starling-LM-7B-beta-GGUF/blob/main/Starling-LM-7B-beta-Q3_K_L.gguf) | Q3_K_L | 3.82GB | Lower quality but usable, good for low RAM availability. |
39
+ | [Starling-LM-7B-beta-Q3_K_M.gguf](https://huggingface.co/bartowski/Starling-LM-7B-beta-GGUF/blob/main/Starling-LM-7B-beta-Q3_K_M.gguf) | Q3_K_M | 3.51GB | Even lower quality. |
40
+ | [Starling-LM-7B-beta-Q3_K_S.gguf](https://huggingface.co/bartowski/Starling-LM-7B-beta-GGUF/blob/main/Starling-LM-7B-beta-Q3_K_S.gguf) | Q3_K_S | 3.16GB | Low quality, not recommended. |
41
+ | [Starling-LM-7B-beta-Q2_K.gguf](https://huggingface.co/bartowski/Starling-LM-7B-beta-GGUF/blob/main/Starling-LM-7B-beta-Q2_K.gguf) | Q2_K | 2.71GB | Extremely low quality, *not* recommended.
42
+
43
+ Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski
Starling-LM-7B-beta-IQ3_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bcaed726d7a7244a2e4bffed21150a81aa07a994f827159dd274f01fb9af3b79
3
+ size 3284901984
Starling-LM-7B-beta-IQ3_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a03ab13862e47c497d0dfe9cd5189f6c5d07df8761b05afc07b33e0379ba64f2
3
+ size 3182403680
Starling-LM-7B-beta-IQ4_NL.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a3f52905a5dc0eeb8568c565cec639179b1d5846c81ffff76dc31a61ccf401df
3
+ size 4155065504
Starling-LM-7B-beta-IQ4_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8320f28768b95e42240c079a265550cb52975002a3cc48616d1eac1b25ecb666
3
+ size 3944399776
Starling-LM-7B-beta-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bac3febb5f292c3d2582455e9ee436eb47700c20768e663aa1e418fc44be96f4
3
+ size 2719251744
Starling-LM-7B-beta-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4b9b8805437f3c7812bc60d733dd4e63584beaa10568f66f11b3848b55116fba
3
+ size 3822035040
Starling-LM-7B-beta-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:28204ec7ee996e1cae4e82c9c3f2e74fc7d8a768871f360ad3adda70edf0fbce
3
+ size 3518996576
Starling-LM-7B-beta-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e36ec9a20e0cb13b66cb509b2081c5ef91f38caeed3f8c9487d06effb0e41aa5
3
+ size 3164577888
Starling-LM-7B-beta-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2940f65d15e516a26a2bfad581e1a44a944de6cd8fcd4698aab7a3bc39c30f09
3
+ size 4108928160
Starling-LM-7B-beta-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:85ad53b9cab2e73d929425f19db65f2da5817745026ed0fc696e5ce5f8ed0303
3
+ size 4368450720
Starling-LM-7B-beta-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c7a096d9bb3873062e7b888d65695f9ca04e3a2518d0c4862cdd4b439d2eea3e
3
+ size 4140385440
Starling-LM-7B-beta-Q5_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4ae58aaf6b269e7b8fc03aadb18974b5414e09ea77679195bb2a5dcec18d5a4e
3
+ size 4997728416
Starling-LM-7B-beta-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c67b033bff47e7b8574491c6c296c094e819488d146aca1c6326c10257450b99
3
+ size 5131421856
Starling-LM-7B-beta-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c9a9a71fcc9966160b5de05eda60d04419e3253660afcb81846ff980fb837770
3
+ size 4997728416
Starling-LM-7B-beta-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:23129c2e9590d6ca142faf361bd509abac1b5295b91c6b1c85e93f2d90ab163f
3
+ size 5942078688
Starling-LM-7B-beta-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:68919eea356c38c6c0e3ca8cddd6b5edd99aec186760c905440e486ec59e0e8a
3
+ size 7695875168