Hugging Face's logo

c01zaut
/
Qwen2.5-3B-Instruct-RK3588-1.1.4

Text Generation
Transformers
Safetensors
English
qwen2
chat
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
c01zaut commited on
Commit
46936ed
·
verified ·
1 Parent(s): c81917e

Upload folder using huggingface_hub

Browse files
Files changed (20) hide show
  1. .gitattributes +12 -0
  2. Qwen2.5-3B-Instruct-rk3588-w8a8-opt-0-hybrid-ratio-0.0.rkllm +1 -1
  3. Qwen2.5-3B-Instruct-rk3588-w8a8-opt-0-hybrid-ratio-0.5.rkllm +1 -1
  4. Qwen2.5-3B-Instruct-rk3588-w8a8-opt-0-hybrid-ratio-1.0.rkllm +1 -1
  5. Qwen2.5-3B-Instruct-rk3588-w8a8-opt-1-hybrid-ratio-0.0.rkllm +1 -1
  6. Qwen2.5-3B-Instruct-rk3588-w8a8-opt-1-hybrid-ratio-0.5.rkllm +1 -1
  7. Qwen2.5-3B-Instruct-rk3588-w8a8-opt-1-hybrid-ratio-1.0.rkllm +1 -1
  8. Qwen2.5-3B-Instruct-rk3588-w8a8_g128-opt-0-hybrid-ratio-0.0.rkllm +3 -0
  9. Qwen2.5-3B-Instruct-rk3588-w8a8_g128-opt-0-hybrid-ratio-0.5.rkllm +3 -0
  10. Qwen2.5-3B-Instruct-rk3588-w8a8_g128-opt-0-hybrid-ratio-1.0.rkllm +3 -0
  11. Qwen2.5-3B-Instruct-rk3588-w8a8_g128-opt-1-hybrid-ratio-0.0.rkllm +3 -0
  12. Qwen2.5-3B-Instruct-rk3588-w8a8_g128-opt-1-hybrid-ratio-0.5.rkllm +3 -0
  13. Qwen2.5-3B-Instruct-rk3588-w8a8_g128-opt-1-hybrid-ratio-1.0.rkllm +3 -0
  14. Qwen2.5-3B-Instruct-rk3588-w8a8_g256-opt-0-hybrid-ratio-0.0.rkllm +3 -0
  15. Qwen2.5-3B-Instruct-rk3588-w8a8_g256-opt-0-hybrid-ratio-0.5.rkllm +3 -0
  16. Qwen2.5-3B-Instruct-rk3588-w8a8_g256-opt-0-hybrid-ratio-1.0.rkllm +3 -0
  17. Qwen2.5-3B-Instruct-rk3588-w8a8_g256-opt-1-hybrid-ratio-0.0.rkllm +3 -0
  18. Qwen2.5-3B-Instruct-rk3588-w8a8_g256-opt-1-hybrid-ratio-0.5.rkllm +3 -0
  19. Qwen2.5-3B-Instruct-rk3588-w8a8_g256-opt-1-hybrid-ratio-1.0.rkllm +3 -0
  20. README.md +1 -1
.gitattributes CHANGED
@@ -39,3 +39,15 @@ Qwen2.5-3B-Instruct-rk3588-w8a8-opt-0-hybrid-ratio-1.0.rkllm filter=lfs diff=lfs
39
  Qwen2.5-3B-Instruct-rk3588-w8a8-opt-1-hybrid-ratio-0.0.rkllm filter=lfs diff=lfs merge=lfs -text
40
  Qwen2.5-3B-Instruct-rk3588-w8a8-opt-1-hybrid-ratio-0.5.rkllm filter=lfs diff=lfs merge=lfs -text
41
  Qwen2.5-3B-Instruct-rk3588-w8a8-opt-1-hybrid-ratio-1.0.rkllm filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
39
  Qwen2.5-3B-Instruct-rk3588-w8a8-opt-1-hybrid-ratio-0.0.rkllm filter=lfs diff=lfs merge=lfs -text
40
  Qwen2.5-3B-Instruct-rk3588-w8a8-opt-1-hybrid-ratio-0.5.rkllm filter=lfs diff=lfs merge=lfs -text
41
  Qwen2.5-3B-Instruct-rk3588-w8a8-opt-1-hybrid-ratio-1.0.rkllm filter=lfs diff=lfs merge=lfs -text
42
+ Qwen2.5-3B-Instruct-rk3588-w8a8_g128-opt-0-hybrid-ratio-0.0.rkllm filter=lfs diff=lfs merge=lfs -text
43
+ Qwen2.5-3B-Instruct-rk3588-w8a8_g128-opt-0-hybrid-ratio-0.5.rkllm filter=lfs diff=lfs merge=lfs -text
44
+ Qwen2.5-3B-Instruct-rk3588-w8a8_g128-opt-0-hybrid-ratio-1.0.rkllm filter=lfs diff=lfs merge=lfs -text
45
+ Qwen2.5-3B-Instruct-rk3588-w8a8_g128-opt-1-hybrid-ratio-0.0.rkllm filter=lfs diff=lfs merge=lfs -text
46
+ Qwen2.5-3B-Instruct-rk3588-w8a8_g128-opt-1-hybrid-ratio-0.5.rkllm filter=lfs diff=lfs merge=lfs -text
47
+ Qwen2.5-3B-Instruct-rk3588-w8a8_g128-opt-1-hybrid-ratio-1.0.rkllm filter=lfs diff=lfs merge=lfs -text
48
+ Qwen2.5-3B-Instruct-rk3588-w8a8_g256-opt-0-hybrid-ratio-0.0.rkllm filter=lfs diff=lfs merge=lfs -text
49
+ Qwen2.5-3B-Instruct-rk3588-w8a8_g256-opt-0-hybrid-ratio-0.5.rkllm filter=lfs diff=lfs merge=lfs -text
50
+ Qwen2.5-3B-Instruct-rk3588-w8a8_g256-opt-0-hybrid-ratio-1.0.rkllm filter=lfs diff=lfs merge=lfs -text
51
+ Qwen2.5-3B-Instruct-rk3588-w8a8_g256-opt-1-hybrid-ratio-0.0.rkllm filter=lfs diff=lfs merge=lfs -text
52
+ Qwen2.5-3B-Instruct-rk3588-w8a8_g256-opt-1-hybrid-ratio-0.5.rkllm filter=lfs diff=lfs merge=lfs -text
53
+ Qwen2.5-3B-Instruct-rk3588-w8a8_g256-opt-1-hybrid-ratio-1.0.rkllm filter=lfs diff=lfs merge=lfs -text
Qwen2.5-3B-Instruct-rk3588-w8a8-opt-0-hybrid-ratio-0.0.rkllm CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a05bf822f5fdffae4845a903eceefa6cc9188a8eee96522cf512bebb5704714b
3
  size 3738346748
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ec95582efedb94b8ebf2a3ca2fa73bd49814cfb35b86c8bc94a3b423c6a73513
3
  size 3738346748
Qwen2.5-3B-Instruct-rk3588-w8a8-opt-0-hybrid-ratio-0.5.rkllm CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a05bf822f5fdffae4845a903eceefa6cc9188a8eee96522cf512bebb5704714b
3
  size 3738346748
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ec95582efedb94b8ebf2a3ca2fa73bd49814cfb35b86c8bc94a3b423c6a73513
3
  size 3738346748
Qwen2.5-3B-Instruct-rk3588-w8a8-opt-0-hybrid-ratio-1.0.rkllm CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a05bf822f5fdffae4845a903eceefa6cc9188a8eee96522cf512bebb5704714b
3
  size 3738346748
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ec95582efedb94b8ebf2a3ca2fa73bd49814cfb35b86c8bc94a3b423c6a73513
3
  size 3738346748
Qwen2.5-3B-Instruct-rk3588-w8a8-opt-1-hybrid-ratio-0.0.rkllm CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:67bd9b8abff97cbc60b9e8a32c86b1a63715968c3a76a8a01b5821fd0282b24e
3
  size 3738346748
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b44a9906d1566f6f32fd9c79f42ead0c1b1aeabe25bbe62904b94cdb4a6240db
3
  size 3738346748
Qwen2.5-3B-Instruct-rk3588-w8a8-opt-1-hybrid-ratio-0.5.rkllm CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:67bd9b8abff97cbc60b9e8a32c86b1a63715968c3a76a8a01b5821fd0282b24e
3
  size 3738346748
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b44a9906d1566f6f32fd9c79f42ead0c1b1aeabe25bbe62904b94cdb4a6240db
3
  size 3738346748
Qwen2.5-3B-Instruct-rk3588-w8a8-opt-1-hybrid-ratio-1.0.rkllm CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:67bd9b8abff97cbc60b9e8a32c86b1a63715968c3a76a8a01b5821fd0282b24e
3
  size 3738346748
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b44a9906d1566f6f32fd9c79f42ead0c1b1aeabe25bbe62904b94cdb4a6240db
3
  size 3738346748
Qwen2.5-3B-Instruct-rk3588-w8a8_g128-opt-0-hybrid-ratio-0.0.rkllm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5d380b0f352105a68fc4644ebe0d2d0655e0a7014ca2b816b38c93847fcf21e2
3
+ size 3969098164
Qwen2.5-3B-Instruct-rk3588-w8a8_g128-opt-0-hybrid-ratio-0.5.rkllm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5d380b0f352105a68fc4644ebe0d2d0655e0a7014ca2b816b38c93847fcf21e2
3
+ size 3969098164
Qwen2.5-3B-Instruct-rk3588-w8a8_g128-opt-0-hybrid-ratio-1.0.rkllm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5d380b0f352105a68fc4644ebe0d2d0655e0a7014ca2b816b38c93847fcf21e2
3
+ size 3969098164
Qwen2.5-3B-Instruct-rk3588-w8a8_g128-opt-1-hybrid-ratio-0.0.rkllm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c3f67bf999b75e8e9f82208644c3283a08dbfc216211e7075ea5a31157ac2301
3
+ size 3949984532
Qwen2.5-3B-Instruct-rk3588-w8a8_g128-opt-1-hybrid-ratio-0.5.rkllm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c3f67bf999b75e8e9f82208644c3283a08dbfc216211e7075ea5a31157ac2301
3
+ size 3949984532
Qwen2.5-3B-Instruct-rk3588-w8a8_g128-opt-1-hybrid-ratio-1.0.rkllm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c3f67bf999b75e8e9f82208644c3283a08dbfc216211e7075ea5a31157ac2301
3
+ size 3949984532
Qwen2.5-3B-Instruct-rk3588-w8a8_g256-opt-0-hybrid-ratio-0.0.rkllm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c217bae5969f51587c7aa537e24b4b72d88418bec1cfaf94fda8f6d56751f08c
3
+ size 3849410292
Qwen2.5-3B-Instruct-rk3588-w8a8_g256-opt-0-hybrid-ratio-0.5.rkllm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c217bae5969f51587c7aa537e24b4b72d88418bec1cfaf94fda8f6d56751f08c
3
+ size 3849410292
Qwen2.5-3B-Instruct-rk3588-w8a8_g256-opt-0-hybrid-ratio-1.0.rkllm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c217bae5969f51587c7aa537e24b4b72d88418bec1cfaf94fda8f6d56751f08c
3
+ size 3849410292
Qwen2.5-3B-Instruct-rk3588-w8a8_g256-opt-1-hybrid-ratio-0.0.rkllm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:413a02d4fccfba707afe778ed8b95d0e0d46a44fc09323d4e155c39f3c8c6e09
3
+ size 3840479444
Qwen2.5-3B-Instruct-rk3588-w8a8_g256-opt-1-hybrid-ratio-0.5.rkllm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:413a02d4fccfba707afe778ed8b95d0e0d46a44fc09323d4e155c39f3c8c6e09
3
+ size 3840479444
Qwen2.5-3B-Instruct-rk3588-w8a8_g256-opt-1-hybrid-ratio-1.0.rkllm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:413a02d4fccfba707afe778ed8b95d0e0d46a44fc09323d4e155c39f3c8c6e09
3
+ size 3840479444
README.md CHANGED
@@ -12,7 +12,7 @@ tags:
12
  ---
13
  # Qwen2.5-3B-Instruct-RK3588-1.1.2
14
 
15
- This version of Qwen2.5-3B-Instruct has been converted to run on the RK3588 NPU using ['w8a8'] quantization.
16
  This model has been optimized with the following LoRA:
17
 
18
  Compatible with RKLLM version: 1.1.2
 
12
  ---
13
  # Qwen2.5-3B-Instruct-RK3588-1.1.2
14
 
15
+ This version of Qwen2.5-3B-Instruct has been converted to run on the RK3588 NPU using ['w8a8', 'w8a8_g128', 'w8a8_g256', 'w8a8_g512'] quantization.
16
  This model has been optimized with the following LoRA:
17
 
18
  Compatible with RKLLM version: 1.1.2