Intern-Decision-4B โ€” Rockchip RKNN ้ƒจ็ฝฒๅŒ…

Intern-Decision-4B ็š„ RKNN ๅฏผๅ‡บ๏ผŒ ไพ› Rockchip ๅๅค„็†ๅ™จ๏ผˆRK1820 / RK3588๏ผ‰ไธŠ็š„ NPU ๆŽจ็†ไฝฟ็”จใ€‚

ไธŠๆธธๆ˜ฏๅŽŸๅง‹ๆƒ้‡๏ผ›ๆœฌไป“ๅบ“ๅชๅŒ…ๅซ RKNN ้‡ๅŒ–ๅฏผๅ‡บๅŽ็š„่ฟ่กŒๆœŸๆ–‡ไปถ๏ผŒไธ่ƒฝ็›ดๆŽฅ็”จ transformers ๅŠ ่ฝฝ๏ผŒ้œ€่ฆ้€š่ฟ‡ไธ‹้ข็š„ wdnai SDK ๅŠ ่ฝฝใ€‚


โš ๏ธ ่ฟ™ไธๆ˜ฏๅฏน่ฏๆจกๅž‹

Intern-Decision ไธ็”Ÿๆˆๆ–‡ๆœฌใ€‚ๅฎƒๅฏนไปปๅŠก้‡Œ็š„ๆฏไธชใ€Œๅ†ณ็ญ–ๅญ—ๆฎตใ€่พ“ๅ‡บ 62 ไธชๅ€™้€‰็ฌฆๅท ๏ผˆA-Z + a-z + 0-9๏ผ‰็š„ logits๏ผŒ็”ฑ CPU ไพงๆข็ฎ—ๆˆ้€‰้กนๆฆ‚็އ๏ผš

ๅญ—ๆฎต็ฑปๅž‹ ่ฏญไน‰ ่พ“ๅ‡บ
choice ้€‰ไธ€้กน ้€‰้กนๆฆ‚็އ๏ผˆๅฝ’ไธ€ๅŒ–๏ผŒๅˆ่ฎกไธบ 1๏ผ‰
score ๆ‰“ๅˆ† ๆฆ‚็އๅŠ ๆƒ่ฏ„ๅˆ†
noul ๆ˜ฏๅฆ ่‚ฏๅฎšๆฆ‚็އ๏ผˆไธๅฝ’ไธ€ๅŒ–๏ผŒๅ„้กน็‹ฌ็ซ‹๏ผ‰

่ฐƒ็”จๆ–นๅฟ…้กปไผ ไปปๅŠก้…็ฝฎ๏ผˆtask=๏ผ‰ใ€‚็ญ”ๆกˆๅฝขๅฆ‚ {"category": "account", "urgency": 0.7676}๏ผŒ ๅฎŒๆ•ดๆฆ‚็އ / logits ้šๆ–‡ๆœฌ้™„ไปถไธ€่ตทๅ›žไผ ใ€‚


ๆ–‡ไปถ่ฏดๆ˜Ž

ๆ–‡ไปถ ๅคงๅฐ ่ฏดๆ˜Ž
Intern-Decision-4B-llm.rknn 25 MB LLM ่ฎก็ฎ—ๅ›พ
Intern-Decision-4B-llm.weight 2.0 GB LLM ๆƒ้‡
Intern-Decision-4B-llm.embed.bin 1.2 GB LLM embedding ่กจ๏ผˆ่ฏ่กจ 248320๏ผ‰
Intern-Decision-4B-llm.tokenizer.gguf 11 MB GGUF ้ข„ๅˆ†่ฏๅ™จ
Intern-Decision-4B-vision.rknn 4.6 MB ่ง†่ง‰็ผ–็ ๅ™จ่ฎก็ฎ—ๅ›พ๏ผˆ384ร—384๏ผ‰
Intern-Decision-4B-vision.weight 651 MB ่ง†่ง‰็ผ–็ ๅ™จๆƒ้‡
intern-decision-4b.json 4 KB ๆจกๅž‹ๆธ…ๅ•๏ผˆwdnai SDK ็”จ๏ผ‰
Intern-Decision-4B-llm_report/model_report.html 220 KB LLM ่ฝฌๆขๆŠฅๅ‘Š๏ผˆRKNX ๅฏ่ง†ๅŒ–๏ผ‰
Intern-Decision-4B-vision_report/model_report.html 84 KB ่ง†่ง‰่ฝฌๆขๆŠฅๅ‘Š

ไฝฟ็”จ

1. ๆ”พ็ฝฎ็›ฎๅฝ•

้ป˜่ฎค็›ฎๅฝ• /userdata/models/Intern_Decision-4B๏ผŒไนŸๅฏไปฅ็”จ็Žฏๅขƒๅ˜้‡ๆŒ‡ๅฎš๏ผš

export WDNAI_INTERN_DECISION_4B_DIR=/path/to/Intern_Decision-4B

2. ๅŠ ่ฝฝ

from wdnai.model import SystemLanguageModel

spec = SystemLanguageModel.intern_decision_4b()
session = spec.session()          # LLM + ่ง†่ง‰็ผ–็ ๅ™จๅธธ้ฉป๏ผŒ็บฆ 3100 MB ๆ˜พๅญ˜

3. ไธ‹ๅ†ณ็ญ–ไปปๅŠก

from wdnai.decision import parse_decision_task

task = parse_decision_task({
    "questions": {
        "ๅนฟๅ‘Š": {"type": "noul", "criteria": {"no": "ๆฒกๆœ‰ๅนฟๅ‘Š", "yes": "ๅซๅนฟๅ‘Š"}},
        "่ฏˆ้ช—": {"type": "noul", "criteria": {"no": "ๆฒกๆœ‰่ฏˆ้ช—", "yes": "ๅซ่ฏˆ้ช—"}},
    }
})
text, attachments = session.run_prompt(
    prompt="้™ๆ—ถไผ˜ๆƒ ๏ผŒๅŠ ๅพฎไฟก้ข†็บขๅŒ…",
    attachment=[image],           # ๅฏ้€‰
    task=task,
)

ๅฎžๆต‹ๆ€ง่ƒฝ๏ผˆRK3588 + 1 ๅผ  1828 ๅๅก๏ผ‰

prefill-only ๆจกๅž‹๏ผšๆฏไธชๅญ—ๆฎต่ท‘ไธ€ๆฌก max_new_tokens=1 ็š„ๆŽจ็†๏ผŒ่งฃ็  token ๆ’ไธบ 0 โ‡’ ๅžๅๆŒ‡ๆ ‡ๆ˜ฏ้ข„ๅกซๅ……้€Ÿ็އ๏ผˆๆ็คบ่ฏๅค„็†้€Ÿๅบฆ๏ผ‰๏ผŒไธๆ˜ฏ็”Ÿๆˆ้€Ÿๅบฆใ€‚

็ปดๅบฆๆ•ฐ ็บฏๆ–‡ๆœฌ ๅ›พๆ–‡
1 0.55 s 0.94 s
4 2.68 s 3.70 s
7 6.30 s 8.04 s
16 23.30 s 26.55 s

โš ๏ธ ่€—ๆ—ถ้š็ปดๅบฆๆ•ฐๅ‘ˆๅนณๆ–นๅขž้•ฟ๏ผšๅญ—ๆฎต i ็š„ prompt ๅ‰็ผ€ๅŒ…ๅซๅญ—ๆฎต 0..iโˆ’1 ็š„ๅฎšไน‰ โ‡’ ๆ€ป prefill ้‡ โˆ Nยฒใ€‚ๅ›พ็‰‡็š„ 151 ไธช่ง†่ง‰ token ไนŸไผš่ขซๆฏไธชๅญ—ๆฎต้‡ๆ–ฐ prefill ไธ€้ใ€‚


้ƒจ็ฝฒๆณจๆ„ไบ‹้กน

  • โš ๏ธ ๆฏไธชๅญ—ๆฎตๅฟ…้กปๆขไธ€ๆฌก LLM session๏ผšๆจกๅž‹ๆœ‰ linear-attention ๅฑ‚๏ผŒ้€’ๅฝ’็Šถๆ€่ทจ่ฐƒ็”จ ๆฎ‹็•™๏ผŒkeep_history=0 ๅชๆธ… KV cache ไธๆธ…้€’ๅฝ’็Šถๆ€ โ‡’ ๆฏไธชๅญ—ๆฎต้ƒฝ่ฆ recreate_llm_session()ใ€‚ๅฎ˜ๆ–น C++ ๅ‚่€ƒๅฎž็ŽฐๅŒ็†๏ผˆๆฏๅญ—ๆฎต rknn3_session_init/destroy๏ผ‰ใ€‚
  • โš ๏ธ prompt ้‡Œ็š„ ChatML ๆ ‡่ฎฐๆ˜ฏใ€Œๅ†™่ฟ›ๆ–‡ๆœฌใ€็š„๏ผŒไธๆ˜ฏ chat template๏ผšๅฏผๅ‡บๆจกๅž‹็š„ chat_template ๆ˜ฏ็ฉบ็š„๏ผŒC ไพงๅฟ…้กปไฟๆŒ set_chat_template("", "", "")ใ€‚ๅฎŒๆ•ดๅฝขๆ€ๆ˜ฏ <|im_start|>system\nโ€ฆ<|im_end|>\n<|im_start|>user\nโ€ฆ<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n{้ชจๆžถ}<|im_end|>\n โ€”โ€” ๅฐ‘ๅ†™่ฟ™ 6 ไธชๆ ‡่ฎฐๅฐฑๅฐ‘ 66 ๅญ—่Š‚ / 6 ไธช token๏ผŒtoken ไฝ็ฝฎๆ•ดไฝ“้”™ไฝใ€ๆฆ‚็އๆ˜Žๆ˜พๅ็งปใ€‚
  • โš ๏ธ ๆŽงๅˆถ token ๅฟ…้กปๆ˜ฏๅ• token๏ผš<decision>=248077ใ€<think>=248068ใ€</think>=248069ใ€‚
  • โš ๏ธ ้ข„ๅˆ†่ฏๆญฃๅˆ™ๆŒ‰ GGUF ็š„ tokenizer.ggml.pre ้€‰๏ผŒไธ่ฆ็”จ ByteLevel ่‡ชๅธฆ็š„ GPT-2 ๆญฃๅˆ™๏ผŒๅฆๅˆ™ token id ๆ•ดไฝ“้”™ไฝใ€‚
  • โš ๏ธ ๆฆ‚็އๆ กๅ‡†ๆธฉๅบฆๆŒ‰ embedding ๅฎฝๅบฆๆŸฅ่กจ๏ผˆ4B = 1.9924182891845703๏ผ‰๏ผŒ ่ฟ่กŒๆ—ถไปŽ RKNN ๅ†…ๅตŒ็š„ LLM_CONFIG ่‡ชๅŠจ่ฏปๅ–ใ€‚
  • โœ… ๅ•ๅกๅฏๅŒ้ฉป็•™ LLM + vision๏ผšๅฎžๆต‹ไธ€ๅผ  1828 ๅๅกไธŠ LLM๏ผˆweight 2013MB + internal 44MB + kvcache 297MB๏ผ‰ไธŽ vision๏ผˆweight 651MB + internal 65MB๏ผ‰ๅŒๆ—ถ้ฉป็•™ใ€‚
  • โœ… ไธŽๅฎ˜ๆ–น C++ demo ้€ไฝไธ€่‡ด๏ผšprompt ๅญ—่Š‚ใ€token id ๅบๅˆ—ใ€ๅŽŸๅง‹ logitsใ€ ๆ กๅ‡†ๆฆ‚็އใ€่ฏ„ๅˆ†ใ€ๆธฉๅบฆๅ…จ้ƒจๅฏนๆ‹้€š่ฟ‡ใ€‚

License

Apache-2.0 โ€”โ€” ไธŽไธŠๆธธ internlm/Intern-Decision-4B ไธ€่‡ดใ€‚

ๆœฌไป“ๅบ“็š„ RKNN ๅฏผๅ‡บ็”ฑ wdnai / wardner ๅทฅๅ…ท้“พ็”Ÿๆˆใ€‚

Downloads last month
215
GGUF
Model size
0 params
Architecture
qwen35
Hardware compatibility
Log In to add your hardware

We're not able to determine the quantization variants.

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for wardner/Intern_Decision-4B

Finetuned
Qwen/Qwen3.5-4B
Quantized
(5)
this model