Xin Liu commited on
Commit
2a2757f
1 Parent(s): e657d51

Update models

Browse files

Signed-off-by: Xin Liu <sam@secondstate.io>

README.md CHANGED
@@ -41,13 +41,17 @@ quantized_by: Second State Inc.
41
 
42
  - Context size: `3072`
43
 
44
- <!-- - Run as LlamaEdge service
45
 
46
  ```bash
47
- wasmedge --dir .:. --nn-preload default:GGML:AUTO:gemma-7b-it-Q5_K_M.gguf llama-api-server.wasm -p gemma-instruct -c 4096
 
 
 
 
48
  ```
49
 
50
- - Run as LlamaEdge command app
51
 
52
  ```bash
53
  wasmedge --dir .:. --nn-preload default:GGML:AUTO:gemma-7b-it-Q5_K_M.gguf llama-chat.wasm -p gemma-instruct -c 4096
@@ -71,4 +75,4 @@ quantized_by: Second State Inc.
71
  | [gemma-1.1-7b-it-Q8_0.gguf](https://huggingface.co/second-state/gemma-1.1-7b-it-GGUF/blob/main/gemma-1.1-7b-it-Q8_0.gguf) | Q8_0 | 8 | 9.08 GB| very large, extremely low quality loss - not recommended |
72
  | [gemma-1.1-7b-it-f16.gguf](https://huggingface.co/second-state/gemma-1.1-7b-it-GGUF/blob/main/gemma-1.1-7b-it-f16.gguf) | f16 | 16 | 34.2 GB| |
73
 
74
- *Quantized with llama.cpp b2534*
 
41
 
42
  - Context size: `3072`
43
 
44
+ - Run as LlamaEdge service
45
 
46
  ```bash
47
+ wasmedge --dir .:. --nn-preload default:GGML:AUTO:gemma-7b-it-Q5_K_M.gguf \
48
+ llama-api-server.wasm \
49
+ --prompt-template gemma-instruct \
50
+ --ctx-size 3072 \
51
+ --model-name gemma-1.1-7b
52
  ```
53
 
54
+ <!-- - Run as LlamaEdge command app
55
 
56
  ```bash
57
  wasmedge --dir .:. --nn-preload default:GGML:AUTO:gemma-7b-it-Q5_K_M.gguf llama-chat.wasm -p gemma-instruct -c 4096
 
75
  | [gemma-1.1-7b-it-Q8_0.gguf](https://huggingface.co/second-state/gemma-1.1-7b-it-GGUF/blob/main/gemma-1.1-7b-it-Q8_0.gguf) | Q8_0 | 8 | 9.08 GB| very large, extremely low quality loss - not recommended |
76
  | [gemma-1.1-7b-it-f16.gguf](https://huggingface.co/second-state/gemma-1.1-7b-it-GGUF/blob/main/gemma-1.1-7b-it-f16.gguf) | f16 | 16 | 34.2 GB| |
77
 
78
+ *Quantized with llama.cpp b2589*
gemma-1.1-7b-it-Q2_K.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:9e4a940efdcc468736b1764bc90d16b23f88314b1ea95935ce839c736d59166d
3
- size 3481447424
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b9cc84d76e1db3e9e054a7692f837f79177721a20d5dcdb1103d3e043d5a91d0
3
+ size 3481447392
gemma-1.1-7b-it-Q3_K_L.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:79b85e8d1fbf55504367df3aa9706d82aed29fb23f4ef595775ae17cc53ff6c7
3
- size 4709067776
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0140da983c11e342b46ccedec9b82220c2efe54fe5bda3d1c20bf445d677b1fa
3
+ size 4709067744
gemma-1.1-7b-it-Q3_K_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:d140fb6359595da6fc8bc46d343b76800a4b1fbf7040a54bb013670f951e3a72
3
- size 4369329152
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8ca97bd41f823cd56954b23f1ebd5b5edb9145fc201f108c4486f2d98db2faac
3
+ size 4369329120
gemma-1.1-7b-it-Q3_K_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e698c5fe5c1ce84b3f3a4ba92d565c727eb3e32a4c3c2e07ea140859d3bdd9ca
3
- size 3982404608
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:509ea44fbc8ffe021e920d9b788cfab0d3f6c342f46f0043096269881a68cdbf
3
+ size 3982404576
gemma-1.1-7b-it-Q4_0.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:4613de65cc03ab74069d7362dbeee2c134093e05d0cd61a6c5a56076e446a7fb
3
- size 5011844096
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f7bea5456d3116ccb9c260f843de52980ab261e4638e827dcc50e0b5ccb80bad
3
+ size 5011844064
gemma-1.1-7b-it-Q4_K_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:680e96f3bc3400cd0b4c7cc823758afb98f5d44b06ffb5fe21ee37fd95c74aa6
3
- size 5329759232
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:87b5061a63b7229dab43454d3d8314fcd422e21a53664d3b47c68ffd5c92d2e7
3
+ size 5329759200
gemma-1.1-7b-it-Q4_K_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:013d926d89a09eb6b50ce3e925e009e4eaf05e3da4c300c8a1781cd514a9f866
3
- size 5046447104
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:669c00d0a65f83971986c3cc727cfeeb2be0011dc0fd455ca48bafdfacb9e7a1
3
+ size 5046447072
gemma-1.1-7b-it-Q5_0.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:8a1e9da4d1e2b5908da8532da1f78112a5879369d6a1a3bfc6ff796c7fa68e2d
3
- size 5980728320
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cb314ce447ac0f60c18cd5d8c36e0ef9cfb47ab3b5cc23aa167c11b2a499cb90
3
+ size 5980728288
gemma-1.1-7b-it-Q5_K_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:d62b6d5a0d00e76709217fcccc48c1680a74be2ec9fe2300b17e1c7f84669a8a
3
- size 6144502784
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:aa05718f43a503beb5982cc772b18d6518a8947f8caf7dcc99aef5452b5a66c2
3
+ size 6144502752
gemma-1.1-7b-it-Q5_K_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:dfe0027e846c63d67dfded677c49aa08f7aced3060d1a1fd7740197ddad92a19
3
- size 5980728320
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cb5354bdb0e50c2633640c5fa48d15960fd00108f60845ee8858ee549104aaf9
3
+ size 5980728288
gemma-1.1-7b-it-Q6_K.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:660b4c9bf162dd80091ca9840513d65caf3ee026fb2b634961f2d2b3e6d326b9
3
- size 7010167808
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:04fb3e9a78142df5f7352cf2cf5e6cc3a060937bdc131af55e266afed43ebf7f
3
+ size 7010167776
gemma-1.1-7b-it-Q8_0.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e5da21f32dfb0c7f7cae8e252ab6d84e1bbaa44924c7a1343761de1afdfabba3
3
- size 9077844992
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:70840b7af2b7191b0088c7c2139e9f829324c03ad808f52db5578b4d03e9272c
3
+ size 9077844960
gemma-1.1-7b-it-f16.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:312d10dbb653bd8d4ab3ec68f83efc61f947d268eba6baca97892d5c7834d910
3
- size 34156768192
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8374dc9da250dffb1ef78505964e8c072fe6688882f93dd72cb870c8a6f0981b
3
+ size 17081756608