Instructions to use visteon/Sima-Qwen3.5-Dflash-Custom-4k-Modalix-Deploy with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use visteon/Sima-Qwen3.5-Dflash-Custom-4k-Modalix-Deploy with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="visteon/Sima-Qwen3.5-Dflash-Custom-4k-Modalix-Deploy")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("visteon/Sima-Qwen3.5-Dflash-Custom-4k-Modalix-Deploy", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use visteon/Sima-Qwen3.5-Dflash-Custom-4k-Modalix-Deploy with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "visteon/Sima-Qwen3.5-Dflash-Custom-4k-Modalix-Deploy"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "visteon/Sima-Qwen3.5-Dflash-Custom-4k-Modalix-Deploy",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/visteon/Sima-Qwen3.5-Dflash-Custom-4k-Modalix-Deploy

SGLang

How to use visteon/Sima-Qwen3.5-Dflash-Custom-4k-Modalix-Deploy with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "visteon/Sima-Qwen3.5-Dflash-Custom-4k-Modalix-Deploy" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "visteon/Sima-Qwen3.5-Dflash-Custom-4k-Modalix-Deploy",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "visteon/Sima-Qwen3.5-Dflash-Custom-4k-Modalix-Deploy" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "visteon/Sima-Qwen3.5-Dflash-Custom-4k-Modalix-Deploy",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use visteon/Sima-Qwen3.5-Dflash-Custom-4k-Modalix-Deploy with Docker Model Runner:
```
docker model run hf.co/visteon/Sima-Qwen3.5-Dflash-Custom-4k-Modalix-Deploy
```

Sima-Qwen3.5-Dflash-Custom-4k — Modalix Device Deployable

Compiled NPU deployable for Qwen3.5-4B target + finetuned DFlash draft on SiMa.ai Modalix, 4k context. Contains the MLA ELF binaries and devkit assets for both models, ready to drop onto a device and serve with llima.

Companion weight-only repo (safetensors + quantize recipe): visteon/Sima-Qwen3.5-Dflash-Custom-4k

.
├── Qwen3.5-4B-Autoround-Safetensors/          # TARGET model (VLM, Autoround W4A16)
│   ├── devkit/                                 # config, tokenizer, chat template, embeddings
│   └── elf_files/                              # 140× MLA stage-1 ELF binaries
└── Qwen3.5-4B-DFlash-FT-RTN-INT8-Safetensors/  # DRAFT model (finetuned DFlash, RTN-INT8)
    ├── devkit/                                 # config, precision, vlm_config
    └── elf_files/                              # 44× MLA stage-1 ELF binaries (draft + dflash context)

Component	Quantization	ELF count	Role
`Qwen3.5-4B-Autoround-Safetensors`	Autoround W4A16 (compressed-tensors)	140	Target / verifier model
`Qwen3.5-4B-DFlash-FT-RTN-INT8-Safetensors`	RTN symmetric INT8 per-channel	44	Speculative draft (DFlash)

Context: --max_num_tokens 4096 (4k)
Language group size: 128 (target decode ELFs use n128 cache-token variants)
Speculative block size: 16
Target layer ids for draft: [1, 5, 9, 13, 17, 21, 25, 29] (every 4th layer)

Runtime prerequisites

Validated on device runtime:

sima-lmm  0.4.0+add-dflash.928f45c41480

Requires the add_dflash branch of sima_lmm (DFlash speculative decoding support). Older runtimes without speculative_method dflash will not load the draft.

Compile-time toolchain versions (for reproducibility, not needed on device):

# target
Python 3.13.2, torch 2.11.0+cu128, transformers 5.10.1,
llmcompressor 0.12.0, auto-round 0.13.0, compressed-tensors 0.17.1

# draft
torch 2.11.0+cu128, llmcompressor 0.14.0, compressed-tensors 0.19.0

Deploy to device

Copy both model directories to the device (e.g. under /media/nvme/):

scp -r Qwen3.5-4B-Autoround-Safetensors \
      Qwen3.5-4B-DFlash-FT-RTN-INT8-Safetensors \
      sima@<device>:/media/nvme/

ssh sima@<device>
mkdir -p /media/nvme/llima/models
ln -s /media/nvme/Qwen3.5-4B-DFlash-FT-RTN-INT8-Safetensors \
      /media/nvme/llima/models/Qwen3.5-4B-DFlash-FT-RTN-INT8
# optional: also register the target alone
ln -s /media/nvme/Qwen3.5-4B-Autoround-Safetensors \
      /media/nvme/llima/models/Qwen3.5-4B-Autoround

Serve:

llima run Qwen3.5-4B-DFlash-FT-RTN-INT8 --mode web --log_level INFO
# HTTP server on :9998, OpenAI-compatible /v1/chat/completions

Add --log_level DEBUG to get per-request server-side stats in the run log:

Number of generated tokens: ...
Number of tokens contributed by draft model: ... (%)
TTFT: ...
TPS: max=..., avg=..., mid=..., min=...

Example request

curl -s -X POST http://<device>:9998/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen3.5-4B-DFlash-FT-RTN-INT8",
    "messages": [{"role":"user","content":"Write a 200-word essay about electric vehicles."}],
    "max_tokens": 300,
    "stream": false
  }'

Streaming (SSE) is supported with "stream": true; chunks include a from_draft flag on each content delta.

On-device benchmark (Modalix, DEBUG log, per-request)

Workload	Tokens	Draft accept %	TTFT	avg TPS	median TPS
Function-calling (short JSON)	76	86.8%	0.27s	68.6	102.1
Function-calling (multi-call JSON)	144	84.0%	0.23s	58.3	104.0
Translation	22	63.6%	0.23s	25.8	29.6
List generation	244	57.4%	0.23s	23.9	29.4
Small conversation	90	53.3%	0.23s	21.7	29.1
Essay (long decode)	358	54.5%	0.23s	22.1	25.2

Cold prefill: ~650 tok/s (linear fit over 188–3032 token prompts)
Sustained plain-text decode: ~30 tok/s
Structured JSON / function-calling: 58–69 avg TPS (draft acceptance 84–87%)

Provenance

Stage	Detail
Draft init	`Qwen3.5-4B-DFlash`
Finetune	1607 steps, final loss ~0.19, token-acc ~0.93
Offline acceptance (val, teacher-forced)	MAT 13.12 (base 1.14)
Draft quantization	RTN INT8 per-channel (min/max), llmcompressor `pack-quantized`
Target	`Qwen3.5-4B-Autoround-Safetensors` (pre-quantized, W4A16)
Compile	`sima_lmm.host.compile_lmm`, `--speculative_method dflash`, `--max_num_tokens 4096`, `--speculative_block_size 16`, `--language_group_size 128`

License

Apache-2.0 (see individual model licenses for Qwen3.5 base terms).

Downloads last month: -; Downloads are not tracked for this model. How to track

Model tree for visteon/Sima-Qwen3.5-Dflash-Custom-4k-Modalix-Deploy

Base model

Qwen/Qwen3.5-4B-Base

Finetuned

Qwen/Qwen3.5-4B

Finetuned

(1173)

this model

visteon
/

Sima-Qwen3.5-Dflash-Custom-4k-Modalix-Deploy