update model files
Browse files
README.md
ADDED
@@ -0,0 +1,10 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# Speculative decoding with Qwen 32B + Qwen 1.5B
|
2 |
+
|
3 |
+
Example:
|
4 |
+
|
5 |
+
```sh
|
6 |
+
llama-server \
|
7 |
+
-m qwen2.5-coder-32b-instruct-q4_k_m.gguf \
|
8 |
+
-md qwen2.5-coder-1.5b-instruct-q4_k_m.gguf \
|
9 |
+
--ctx-size 65536
|
10 |
+
```
|
qwen2.5-coder-1.5b-instruct-q4_k_m.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:cc324af070c2ecbfd324a30884d2f951a7ff756aba85cb811a6ec436933bb046
|
3 |
+
size 1117320768
|
qwen2.5-coder-32b-instruct-q4_k_m.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:4d64b316b5e6319d9613e0d97935d9ebd631fc7e334da400d00085eca749d085
|
3 |
+
size 19851335872
|