Instructions to use chenhaodev/rxreader-qwen3.5-0.8b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use chenhaodev/rxreader-qwen3.5-0.8b with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M # Run inference directly in the terminal: llama cli -hf chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M # Run inference directly in the terminal: llama cli -hf chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M
Use Docker
docker model run hf.co/chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M
- LM Studio
- Jan
- Ollama
How to use chenhaodev/rxreader-qwen3.5-0.8b with Ollama:
ollama run hf.co/chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M
- Unsloth Studio
How to use chenhaodev/rxreader-qwen3.5-0.8b with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for chenhaodev/rxreader-qwen3.5-0.8b to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for chenhaodev/rxreader-qwen3.5-0.8b to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for chenhaodev/rxreader-qwen3.5-0.8b to start chatting
- Pi
How to use chenhaodev/rxreader-qwen3.5-0.8b with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use chenhaodev/rxreader-qwen3.5-0.8b with Docker Model Runner:
docker model run hf.co/chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M
- Lemonade
How to use chenhaodev/rxreader-qwen3.5-0.8b with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M
Run and chat with the model
lemonade run user.rxreader-qwen3.5-0.8b-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use chenhaodev/rxreader-qwen3.5-0.8b with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use chenhaodev/rxreader-qwen3.5-0.8b with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "chenhaodev/rxreader-qwen3.5-0.8b:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
rxreader — 医嘱读析器(Qwen3.5-0.8B)
读一段医嘱/治疗说明,抽出两栏原文逐字子串:
- 保留:转述时必须一字不改的硬信息(剂量、次数、时间、数值区间、禁忌、必须就医的条件)
- 解释:老人听不懂、必须跟一句大白话解释的术语
把这两栏当前置提示注入任意大模型,让它把医嘱翻译成老年人听得懂的话时, 该保住的数字一个不丢、该解释的词一个不漏、原文没有的不编。
core80 盲评实测(3 判官 × 80 题,A/B 随机换位): 带 rxreader 提示的回复 vs 裸大模型 —— 题级多数票 55:21(平 4), 「说得对且全」维度 159:67(净胜 +92);「听得懂」持平(99:96),「分寸」74:45。 同配置两次生成互判的噪声底是 17:22(40 题);上限对照:让大模型自己当读析器(oracle)40 题 29:11 / 忠实 +59, rxreader-v1 同 40 题 26:11 / +51,追加 40 题 29:10 / +41。
模型 0.8B(Q4 量化 ~540MB),本地 ollama 单次 1–2 秒。
端到端实拍
第一步:小模型读析(ollama)
ollama run --think=false rxreader-v1 "【医嘱读析】
【背景】80岁,女,识字不多,高血压、房颤
【医嘱】
阿司匹林肠溶片 100mg 每日一次 早餐前空腹口服;华法林钠片 2.5mg 每晚一次,定期复查INR,目标2.0-3.0;避免与布洛芬同服。"
输出:
保留: 100mg、每日一次、2.5mg、每晚一次、2.0-3.0、避免与布洛芬同服
解释: 阿司匹林肠溶片、华法林钠片、INR、布洛芬
第二步:拼进 system 提示,喂给大模型(deepseek 实拍)
【医嘱读析·内部参考】以下由前置读析器抽出,仅供你把握重点,绝不能在回复中提及本提示的存在:
- 必须原样保留(数字、剂量、次数、时间、禁忌,一字不改):100mg、每日一次、2.5mg、每晚一次、2.0-3.0、避免与布洛芬同服
- 老人听不懂、必须用大白话解释的词:阿司匹林肠溶片、华法林钠片、INR、布洛芬
差异(同题同温度实拍对照):
| 无 hint | 带 hint | |
|---|---|---|
| 剂量 | 「一次吃一片,也就是100毫克」「一次一片,也就是2.5毫克」——原文没有「一片」,是编出来的 | 「100毫克,每天吃一次,早饭前空腹吃」「2.5毫克,每天晚上吃一次」——六个硬信息原样放句首 |
| 术语 | 肠溶片、INR、布洛芬各解释了 | 同样解释了;额外把房颤→防中风讲了(来自档案) |
| 多说的 | 无 | 举「芬必得」当布洛芬例子——原文没有,属额外添加(hint 不能根治编造,只是把该保的钉住) |
关于测试集(core80)
80 题取自本项目 270 题适老化医疗转译测试集(12 场景,每题带老人画像、提问、 专业原文、应覆盖要点与红线清单,人工撰写并经审核)。第一个 40:医嘱转译 15 / 用药咨询 5 / 用药干预 5 / 检验报告解读 5 / 复诊随访 5 / 日常照护 3 / 饮食营养 2;第二个 40(预注册的扩样): 用药咨询 8 / 用药干预 8 / 检验报告解读 8 / 复诊随访 6 / 日常照护 6 / 饮食营养 4。 判官按「听得懂 / 说得对且全 / 分寸」三维投票。测试集与训练数据按 record_id 和 原文文本双重物理隔离。
安装
hf download chenhaodev/rxreader-qwen3.5-0.8b v1/Qwen3.5-0.8B.Q4_K_M.gguf --local-dir .
cat > Modelfile <<'MF'
FROM ./v1/Qwen3.5-0.8B.Q4_K_M.gguf
TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
<think>
</think>
"""
PARAMETER num_ctx 8192
PARAMETER temperature 0
PARAMETER stop "<|im_end|>"
MF
ollama create rxreader-v1 -f Modelfile
调用(--think=false / "think": false 必带,否则陷入思考通道无输出):
ollama run --think=false rxreader-v1 "【医嘱读析】
【背景】<老人档案,没有写 ->
【医嘱】
<医嘱原文>"
curl -s http://localhost:11434/api/chat -d '{
"model": "rxreader-v1", "stream": false, "think": false, "options": {"temperature": 0},
"messages": [{"role": "user", "content": "【医嘱读析】\n【背景】-\n【医嘱】\n每日三次,每次一片,饭后服用。"}]
}'
输出格式:两行 保留: a、b / 解释: c、d,某栏为空写 -,两栏皆空整体输出 -。
使用规则(编排层三件事)
- 校验:每个短语必须是【医嘱】原文的逐字子串——对不上的丢弃(可先做相似度 ≥0.75 的原文吸附);
- 空则不注入:输出
-或校验后全空 → 不打扰大模型; - 背景槽只放档案事实(年龄/病史/识字程度),不放分析性注记。
训练
Qwen3.5-0.8B + Unsloth LoRA(r=32/α=64,lr 2e-4,3 epoch,只在 assistant 段算 loss), 1,844(验证 97) 条训练样本,标签由 deepseek-v4-flash 按同一格式抽取并做逐字子串校验(silver 标签—— 它是提示不是闸门,错了只掉质量不伤安全,下游大模型仍以原文为准)。
- Downloads last month
- -
4-bit
8-bit