Xin Liu
commited on
Commit
•
2a2757f
1
Parent(s):
e657d51
Update models
Browse filesSigned-off-by: Xin Liu <sam@secondstate.io>
- README.md +8 -4
- gemma-1.1-7b-it-Q2_K.gguf +2 -2
- gemma-1.1-7b-it-Q3_K_L.gguf +2 -2
- gemma-1.1-7b-it-Q3_K_M.gguf +2 -2
- gemma-1.1-7b-it-Q3_K_S.gguf +2 -2
- gemma-1.1-7b-it-Q4_0.gguf +2 -2
- gemma-1.1-7b-it-Q4_K_M.gguf +2 -2
- gemma-1.1-7b-it-Q4_K_S.gguf +2 -2
- gemma-1.1-7b-it-Q5_0.gguf +2 -2
- gemma-1.1-7b-it-Q5_K_M.gguf +2 -2
- gemma-1.1-7b-it-Q5_K_S.gguf +2 -2
- gemma-1.1-7b-it-Q6_K.gguf +2 -2
- gemma-1.1-7b-it-Q8_0.gguf +2 -2
- gemma-1.1-7b-it-f16.gguf +2 -2
README.md
CHANGED
@@ -41,13 +41,17 @@ quantized_by: Second State Inc.
|
|
41 |
|
42 |
- Context size: `3072`
|
43 |
|
44 |
-
|
45 |
|
46 |
```bash
|
47 |
-
wasmedge --dir .:. --nn-preload default:GGML:AUTO:gemma-7b-it-Q5_K_M.gguf
|
|
|
|
|
|
|
|
|
48 |
```
|
49 |
|
50 |
-
- Run as LlamaEdge command app
|
51 |
|
52 |
```bash
|
53 |
wasmedge --dir .:. --nn-preload default:GGML:AUTO:gemma-7b-it-Q5_K_M.gguf llama-chat.wasm -p gemma-instruct -c 4096
|
@@ -71,4 +75,4 @@ quantized_by: Second State Inc.
|
|
71 |
| [gemma-1.1-7b-it-Q8_0.gguf](https://huggingface.co/second-state/gemma-1.1-7b-it-GGUF/blob/main/gemma-1.1-7b-it-Q8_0.gguf) | Q8_0 | 8 | 9.08 GB| very large, extremely low quality loss - not recommended |
|
72 |
| [gemma-1.1-7b-it-f16.gguf](https://huggingface.co/second-state/gemma-1.1-7b-it-GGUF/blob/main/gemma-1.1-7b-it-f16.gguf) | f16 | 16 | 34.2 GB| |
|
73 |
|
74 |
-
*Quantized with llama.cpp
|
|
|
41 |
|
42 |
- Context size: `3072`
|
43 |
|
44 |
+
- Run as LlamaEdge service
|
45 |
|
46 |
```bash
|
47 |
+
wasmedge --dir .:. --nn-preload default:GGML:AUTO:gemma-7b-it-Q5_K_M.gguf \
|
48 |
+
llama-api-server.wasm \
|
49 |
+
--prompt-template gemma-instruct \
|
50 |
+
--ctx-size 3072 \
|
51 |
+
--model-name gemma-1.1-7b
|
52 |
```
|
53 |
|
54 |
+
<!-- - Run as LlamaEdge command app
|
55 |
|
56 |
```bash
|
57 |
wasmedge --dir .:. --nn-preload default:GGML:AUTO:gemma-7b-it-Q5_K_M.gguf llama-chat.wasm -p gemma-instruct -c 4096
|
|
|
75 |
| [gemma-1.1-7b-it-Q8_0.gguf](https://huggingface.co/second-state/gemma-1.1-7b-it-GGUF/blob/main/gemma-1.1-7b-it-Q8_0.gguf) | Q8_0 | 8 | 9.08 GB| very large, extremely low quality loss - not recommended |
|
76 |
| [gemma-1.1-7b-it-f16.gguf](https://huggingface.co/second-state/gemma-1.1-7b-it-GGUF/blob/main/gemma-1.1-7b-it-f16.gguf) | f16 | 16 | 34.2 GB| |
|
77 |
|
78 |
+
*Quantized with llama.cpp b2589*
|
gemma-1.1-7b-it-Q2_K.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
-
size
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:b9cc84d76e1db3e9e054a7692f837f79177721a20d5dcdb1103d3e043d5a91d0
|
3 |
+
size 3481447392
|
gemma-1.1-7b-it-Q3_K_L.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
-
size
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:0140da983c11e342b46ccedec9b82220c2efe54fe5bda3d1c20bf445d677b1fa
|
3 |
+
size 4709067744
|
gemma-1.1-7b-it-Q3_K_M.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
-
size
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:8ca97bd41f823cd56954b23f1ebd5b5edb9145fc201f108c4486f2d98db2faac
|
3 |
+
size 4369329120
|
gemma-1.1-7b-it-Q3_K_S.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
-
size
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:509ea44fbc8ffe021e920d9b788cfab0d3f6c342f46f0043096269881a68cdbf
|
3 |
+
size 3982404576
|
gemma-1.1-7b-it-Q4_0.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
-
size
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:f7bea5456d3116ccb9c260f843de52980ab261e4638e827dcc50e0b5ccb80bad
|
3 |
+
size 5011844064
|
gemma-1.1-7b-it-Q4_K_M.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
-
size
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:87b5061a63b7229dab43454d3d8314fcd422e21a53664d3b47c68ffd5c92d2e7
|
3 |
+
size 5329759200
|
gemma-1.1-7b-it-Q4_K_S.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
-
size
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:669c00d0a65f83971986c3cc727cfeeb2be0011dc0fd455ca48bafdfacb9e7a1
|
3 |
+
size 5046447072
|
gemma-1.1-7b-it-Q5_0.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
-
size
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:cb314ce447ac0f60c18cd5d8c36e0ef9cfb47ab3b5cc23aa167c11b2a499cb90
|
3 |
+
size 5980728288
|
gemma-1.1-7b-it-Q5_K_M.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
-
size
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:aa05718f43a503beb5982cc772b18d6518a8947f8caf7dcc99aef5452b5a66c2
|
3 |
+
size 6144502752
|
gemma-1.1-7b-it-Q5_K_S.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
-
size
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:cb5354bdb0e50c2633640c5fa48d15960fd00108f60845ee8858ee549104aaf9
|
3 |
+
size 5980728288
|
gemma-1.1-7b-it-Q6_K.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
-
size
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:04fb3e9a78142df5f7352cf2cf5e6cc3a060937bdc131af55e266afed43ebf7f
|
3 |
+
size 7010167776
|
gemma-1.1-7b-it-Q8_0.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
-
size
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:70840b7af2b7191b0088c7c2139e9f829324c03ad808f52db5578b4d03e9272c
|
3 |
+
size 9077844960
|
gemma-1.1-7b-it-f16.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
-
size
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:8374dc9da250dffb1ef78505964e8c072fe6688882f93dd72cb870c8a6f0981b
|
3 |
+
size 17081756608
|