tiny-d3-nano-for-testing

This is a modified, damaged copy of vllm-sr/d3-nano, made only to test software. Its output is gibberish. Do not use it for anything else.

It keeps the architecture, tokenizer, chat template, decision_config.json (codes, token ids, temperature, attention mode) and readout of d3-nano, so it runs the same code paths as the real model in llama.cpp (vllm-d3 decision type: non-causal attention, readout of 255 option codes stored as an LM head, entropy confidence, image input), but it is small enough for CI.

GGUF version: ggml-org-dev/tiny-d3-nano-for-testing-GGUF

Changes from the original model

Sliced from the safetensors with slice_qwen35_decision.py (python3 slice_qwen35_decision.py vllm-sr/d3-nano <out dir>), which follows the slice-model skill of llama.cpp:

  • Text model: 4 of 24 layers (0-3: 3 linear attention + 1 full attention), hidden size 2048 -> 32, FFN 6144 -> 64, heads 8 -> 2, KV heads 2 -> 1 (head_dim 256 kept), linear attention key heads 16 -> 2, value heads 16 -> 2 (head dims 128 kept)
  • Vision encoder: 2 of 24 blocks, hidden size 1024 -> 128, heads 16 -> 2 (head_dim 64 kept), FFN 4096 -> 256, output 2048 -> 32
  • readout.safetensors: 255 x 2048 -> 255 x 32 (first columns)
  • The Python code of the original repo (d3_*.py, modeling_d3.py, pipeline_d3.py), MODEL_MANIFEST.json, assets and eval results are not included

License

Derived from d3-nano by the vLLM Semantic Router Team (itself built on Qwen/Qwen3.5-2B), under the Apache License 2.0. See NOTICE.

Downloads last month
13
Safetensors
Model size
9.21M params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for ggml-org-dev/tiny-d3-nano-for-testing

Finetuned
Qwen/Qwen3.5-2B
Finetuned
vllm-sr/d3-nano
Finetuned
(7)
this model
Quantizations
1 model