Instructions to use wardner/Intern_Decision-4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use wardner/Intern_Decision-4B with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf wardner/Intern_Decision-4B # Run inference directly in the terminal: llama cli -hf wardner/Intern_Decision-4B
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf wardner/Intern_Decision-4B # Run inference directly in the terminal: llama cli -hf wardner/Intern_Decision-4B
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf wardner/Intern_Decision-4B # Run inference directly in the terminal: ./build/bin/llama cli -hf wardner/Intern_Decision-4B
Use Docker
docker model run hf.co/wardner/Intern_Decision-4B
- Llama new
- LM Studio
- Jan
- vLLM
How to use wardner/Intern_Decision-4B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "wardner/Intern_Decision-4B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "wardner/Intern_Decision-4B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/wardner/Intern_Decision-4B
- Ollama
How to use wardner/Intern_Decision-4B with Ollama:
ollama run hf.co/wardner/Intern_Decision-4B
- Unsloth Desktop
- Pi
How to use wardner/Intern_Decision-4B with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf wardner/Intern_Decision-4B
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "wardner/Intern_Decision-4B" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use wardner/Intern_Decision-4B with Docker Model Runner:
docker model run hf.co/wardner/Intern_Decision-4B
- Lemonade
How to use wardner/Intern_Decision-4B with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull wardner/Intern_Decision-4B
Run and chat with the model
lemonade run user.Intern_Decision-4B-{{QUANT_TAG}}List all available models
lemonade list
- Hermes Agent
How to use wardner/Intern_Decision-4B with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf wardner/Intern_Decision-4B
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default wardner/Intern_Decision-4B
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use wardner/Intern_Decision-4B with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf wardner/Intern_Decision-4B
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "wardner/Intern_Decision-4B" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Intern-Decision-4B โ Rockchip RKNN ้จ็ฝฒๅ
Intern-Decision-4B ็ RKNN ๅฏผๅบ๏ผ ไพ Rockchip ๅๅค็ๅจ๏ผRK1820 / RK3588๏ผไธ็ NPU ๆจ็ไฝฟ็จใ
ไธๆธธๆฏๅๅงๆ้๏ผๆฌไปๅบๅชๅ
ๅซ RKNN ้ๅๅฏผๅบๅ็่ฟ่กๆๆไปถ๏ผไธ่ฝ็ดๆฅ็จ
transformers ๅ ่ฝฝ๏ผ้่ฆ้่ฟไธ้ข็ wdnai SDK ๅ ่ฝฝใ
โ ๏ธ ่ฟไธๆฏๅฏน่ฏๆจกๅ
Intern-Decision ไธ็ๆๆๆฌใๅฎๅฏนไปปๅก้็ๆฏไธชใๅณ็ญๅญๆฎตใ่พๅบ 62 ไธชๅ้็ฌฆๅท
๏ผA-Z + a-z + 0-9๏ผ็ logits๏ผ็ฑ CPU ไพงๆข็ฎๆ้้กนๆฆ็๏ผ
| ๅญๆฎต็ฑปๅ | ่ฏญไน | ่พๅบ |
|---|---|---|
choice |
้ไธ้กน | ้้กนๆฆ็๏ผๅฝไธๅ๏ผๅ่ฎกไธบ 1๏ผ |
score |
ๆๅ | ๆฆ็ๅ ๆ่ฏๅ |
noul |
ๆฏๅฆ | ่ฏๅฎๆฆ็๏ผไธๅฝไธๅ๏ผๅ้กน็ฌ็ซ๏ผ |
่ฐ็จๆนๅฟ
้กปไผ ไปปๅก้
็ฝฎ๏ผtask=๏ผใ็ญๆกๅฝขๅฆ {"category": "account", "urgency": 0.7676}๏ผ
ๅฎๆดๆฆ็ / logits ้ๆๆฌ้ไปถไธ่ตทๅไผ ใ
ๆไปถ่ฏดๆ
| ๆไปถ | ๅคงๅฐ | ่ฏดๆ |
|---|---|---|
Intern-Decision-4B-llm.rknn |
25 MB | LLM ่ฎก็ฎๅพ |
Intern-Decision-4B-llm.weight |
2.0 GB | LLM ๆ้ |
Intern-Decision-4B-llm.embed.bin |
1.2 GB | LLM embedding ่กจ๏ผ่ฏ่กจ 248320๏ผ |
Intern-Decision-4B-llm.tokenizer.gguf |
11 MB | GGUF ้ขๅ่ฏๅจ |
Intern-Decision-4B-vision.rknn |
4.6 MB | ่ง่ง็ผ็ ๅจ่ฎก็ฎๅพ๏ผ384ร384๏ผ |
Intern-Decision-4B-vision.weight |
651 MB | ่ง่ง็ผ็ ๅจๆ้ |
intern-decision-4b.json |
4 KB | ๆจกๅๆธ
ๅ๏ผwdnai SDK ็จ๏ผ |
Intern-Decision-4B-llm_report/model_report.html |
220 KB | LLM ่ฝฌๆขๆฅๅ๏ผRKNX ๅฏ่งๅ๏ผ |
Intern-Decision-4B-vision_report/model_report.html |
84 KB | ่ง่ง่ฝฌๆขๆฅๅ |
ไฝฟ็จ
1. ๆพ็ฝฎ็ฎๅฝ
้ป่ฎค็ฎๅฝ /userdata/models/Intern_Decision-4B๏ผไนๅฏไปฅ็จ็ฏๅขๅ้ๆๅฎ๏ผ
export WDNAI_INTERN_DECISION_4B_DIR=/path/to/Intern_Decision-4B
2. ๅ ่ฝฝ
from wdnai.model import SystemLanguageModel
spec = SystemLanguageModel.intern_decision_4b()
session = spec.session() # LLM + ่ง่ง็ผ็ ๅจๅธธ้ฉป๏ผ็บฆ 3100 MB ๆพๅญ
3. ไธๅณ็ญไปปๅก
from wdnai.decision import parse_decision_task
task = parse_decision_task({
"questions": {
"ๅนฟๅ": {"type": "noul", "criteria": {"no": "ๆฒกๆๅนฟๅ", "yes": "ๅซๅนฟๅ"}},
"่ฏ้ช": {"type": "noul", "criteria": {"no": "ๆฒกๆ่ฏ้ช", "yes": "ๅซ่ฏ้ช"}},
}
})
text, attachments = session.run_prompt(
prompt="้ๆถไผๆ ๏ผๅ ๅพฎไฟก้ข็บขๅ
",
attachment=[image], # ๅฏ้
task=task,
)
ๅฎๆตๆง่ฝ๏ผRK3588 + 1 ๅผ 1828 ๅๅก๏ผ
prefill-only ๆจกๅ๏ผๆฏไธชๅญๆฎต่ทไธๆฌก max_new_tokens=1 ็ๆจ็๏ผ่งฃ็ token ๆไธบ 0
โ ๅๅๆๆ ๆฏ้ขๅกซๅ
้็๏ผๆ็คบ่ฏๅค็้ๅบฆ๏ผ๏ผไธๆฏ็ๆ้ๅบฆใ
| ็ปดๅบฆๆฐ | ็บฏๆๆฌ | ๅพๆ |
|---|---|---|
| 1 | 0.55 s | 0.94 s |
| 4 | 2.68 s | 3.70 s |
| 7 | 6.30 s | 8.04 s |
| 16 | 23.30 s | 26.55 s |
โ ๏ธ ่ๆถ้็ปดๅบฆๆฐๅๅนณๆนๅข้ฟ๏ผๅญๆฎต i ็ prompt ๅ็ผๅ ๅซๅญๆฎต 0..iโ1 ็ๅฎไน โ ๆป prefill ้ โ Nยฒใๅพ็็ 151 ไธช่ง่ง token ไนไผ่ขซๆฏไธชๅญๆฎต้ๆฐ prefill ไธ้ใ
้จ็ฝฒๆณจๆไบ้กน
- โ ๏ธ ๆฏไธชๅญๆฎตๅฟ
้กปๆขไธๆฌก LLM session๏ผๆจกๅๆ linear-attention ๅฑ๏ผ้ๅฝ็ถๆ่ทจ่ฐ็จ
ๆฎ็๏ผ
keep_history=0ๅชๆธ KV cache ไธๆธ ้ๅฝ็ถๆ โ ๆฏไธชๅญๆฎต้ฝ่ฆrecreate_llm_session()ใๅฎๆน C++ ๅ่ๅฎ็ฐๅ็๏ผๆฏๅญๆฎตrknn3_session_init/destroy๏ผใ - โ ๏ธ prompt ้็ ChatML ๆ ่ฎฐๆฏใๅ่ฟๆๆฌใ็๏ผไธๆฏ chat template๏ผๅฏผๅบๆจกๅ็
chat_templateๆฏ็ฉบ็๏ผC ไพงๅฟ ้กปไฟๆset_chat_template("", "", "")ใๅฎๆดๅฝขๆๆฏ<|im_start|>system\nโฆ<|im_end|>\n<|im_start|>user\nโฆ<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n{้ชจๆถ}<|im_end|>\nโโ ๅฐๅ่ฟ 6 ไธชๆ ่ฎฐๅฐฑๅฐ 66 ๅญ่ / 6 ไธช token๏ผtoken ไฝ็ฝฎๆดไฝ้ไฝใๆฆ็ๆๆพๅ็งปใ - โ ๏ธ ๆงๅถ token ๅฟ
้กปๆฏๅ token๏ผ
<decision>=248077ใ<think>=248068ใ</think>=248069ใ - โ ๏ธ ้ขๅ่ฏๆญฃๅๆ GGUF ็
tokenizer.ggml.pre้๏ผไธ่ฆ็จ ByteLevel ่ชๅธฆ็ GPT-2 ๆญฃๅ๏ผๅฆๅ token id ๆดไฝ้ไฝใ - โ ๏ธ ๆฆ็ๆ กๅๆธฉๅบฆๆ embedding ๅฎฝๅบฆๆฅ่กจ๏ผ4B =
1.9924182891845703๏ผ๏ผ ่ฟ่กๆถไป RKNN ๅ ๅต็LLM_CONFIG่ชๅจ่ฏปๅใ - โ ๅๅกๅฏๅ้ฉป็ LLM + vision๏ผๅฎๆตไธๅผ 1828 ๅๅกไธ LLM๏ผweight 2013MB + internal 44MB + kvcache 297MB๏ผไธ vision๏ผweight 651MB + internal 65MB๏ผๅๆถ้ฉป็ใ
- โ ไธๅฎๆน C++ demo ้ไฝไธ่ด๏ผprompt ๅญ่ใtoken id ๅบๅใๅๅง logitsใ ๆ กๅๆฆ็ใ่ฏๅใๆธฉๅบฆๅ จ้จๅฏนๆ้่ฟใ
License
Apache-2.0 โโ ไธไธๆธธ internlm/Intern-Decision-4B ไธ่ดใ
ๆฌไปๅบ็ RKNN ๅฏผๅบ็ฑ wdnai / wardner ๅทฅๅ
ท้พ็ๆใ
- Downloads last month
- 215
We're not able to determine the quantization variants.