Dexy2/Piko-9b

A 9.65B hybrid-attention vision-language model, composed from two open checkpoints.

---

Model summary

Piko-9b is a 9.65-billion-parameter vision-language model that takes text and images and produces text. It is built by splicing together two existing open checkpoints rather than trained from scratch:

Component	Parameters	Source	State
Language backbone	9,197,093,888	Fine-tune of deepreinforce-ai/Ornith-1.0-9B	Fine-tuned
Vision tower + merger	456,010,480	Qwen/Qwen3.5-9B	Copied verbatim

It emits a <think>…</think> reasoning trace before its answer, uses greedy decoding by default, and needs about 8 GB of VRAM in 4-bit.

On a 70-case regression suite it scores 65/70, against 63/70 for Qwen/Qwen3.5-9B measured identically. That difference is not statistically significant. Full detail below.

What Piko-9b is

A composition, verified at the tensor level. All 775 tensors in the published checkpoint are bitwise identical to a tensor in one of the two source checkpoints — 427 from the language backbone, 348 from Qwen's vision tower, none unaccounted for.

The language backbone is genuinely fine-tuned: 0 of 14 sampled tensors match stock Qwen3.5-9B, with 1.5–5.0 % relative L2 drift. That fine-tuning came from a chain of merged QLoRA stages (wraithfast-phase10 → 12 → 13 → 14 → 15) applied to Ornith-1.0-9B.

The vision tower is not fine-tuned at all. It was downloaded from Qwen/Qwen3.5-9B and attached unchanged.

Relationship to the base model

Qwen/Qwen3.5-9B-Base  (Apache-2.0)
   │
   ├── Qwen/Qwen3.5-9B ──────────── vision tower + merger, copied verbatim ──┐
   │                                                                        │
   └── deepreinforce-ai/Ornith-1.0-9B  (MIT)                                 │
            │ QLoRA r=32 α=64, merged after each stage                       │
            └── wraithfast-phase15-150k-full-ft ─── language backbone ───────┤
                                                                             │
                                                    Dexy2/Piko-9b  ──────────┘

What this means in practice

The vision tower was trained by Qwen to project into Qwen3.5-9B's embedding space. The backbone it is attached to has since drifted 1.5–5 % away from that space, and no vision training was performed to re-align it.

That was expected to break image handling. Measured, it does not: OCR scores 10/10 and document understanding 10/10 on the custom suite. This is the most surprising result in the release and it is reported as an empirical finding, not as evidence that the composition was principled.

Full evidence: reports/lineage_analysis.md.

Corrections to the previous model card

The earlier release described six training stages totalling 520,000 examples, and published nine benchmark scores. Tensor comparison shows none of those stages are present in these weights, and the nine scores were measured on a different, text-only checkpoint. Details: reports/repository_audit.md §5, CHANGELOG.md.

Intended uses

Supported by measurement on this checkpoint:

• Document and receipt reading — extracting fields from rendered documents, including OCR-error correction. 10/10 and 10/10 on the custom suite.
• Table and chart reading — 9/10.
• General instruction following and reasoning — 9/10 and 10/10.
• Code generation — 5/5, verified by executing the generated functions against assertions.
• Long-context retrieval — 5/5 at 2K, 8K and 32K filler tokens, depths 0.1–0.9.

Out-of-scope uses

• Anything safety-critical — medical, legal, financial, or operational decisions.
• Factual questions about entities it may not know. It invented a plausible summary of a non-existent treaty when asked. See Limitations.
• Adversarial or untrusted input without downstream validation. No specific prompt-injection defence; this matters most in document pipelines, its strongest use case.
• Video or audio. Not supported — see Supported modalities.
• Deployments needing a stable self-identity without a system prompt. Asked what it is, this checkpoint answers "I am Wraith, an AI model."

Architecture

Qwen3_5ForConditionalGeneration, model_type: qwen3_5. A hybrid attention stack, not a conventional transformer.

Language model		Vision encoder
Hidden size	4,096	Depth	27
Layers	32	Hidden size	1,152
— linear attention	24	Attention heads	16
— full attention	8 (every 4th)	Intermediate size	4,304
Attention heads	16	Patch size	16
Key/value heads	4 (GQA)	Spatial merge	2
Head dim	256	Output size	4,096
Intermediate size	12,288	Position embeddings	2,304
Vocabulary	248,320	DeepStack indexes	[] (disabled)

Linear attention: conv kernel 4, key head dim 128, 16 key heads, 32 value heads, value head dim 128, SSM state in float32.

Positional encoding: mRoPE, interleaved, sections [11, 11, 10], partial_rotary_factor 0.25, rope_theta 10,000,000. No RoPE scaling is applied.

Weights: 11 safetensors shards, 21.3 GB, BF16 throughout.

Parameter count

9,653,104,368 total, read from the safetensors headers. Note that a 4-bit loaded model reports 5,724,972,272 — that is the packed element count, not the logical parameter count.

Context length

max_position_embeddings is 262,144, native to the architecture with no scaling trick.

That is a configuration value, not a validated capability. Measured: retrieval works at 2K, 8K and ~32K filler tokens; a 14,429-token prompt was answered correctly in 3.5 s. Beyond ~32K is untested, and a 32,768-token single forward pass exceeded the 15.92 GB test GPU.

Supported modalities

Modality	Metadata	Weights	Trained here	Validated
Text	Yes	Yes	Yes	Yes
Image	Yes	Yes (456 M, verbatim Qwen)	No	Yes — works
Video	Yes (video_preprocessor_config.json, <|video_pad|>)	Shares image tower	No	Not tested
Audio	Token IDs only	No encoder	No	Not supported

The audio tokens (<|audio_start|>, <|audio_pad|>, <|audio_end|>) are inherited tokenizer metadata. There is no audio encoder in the weights.

Training and creation methodology

The published weights contain no training performed under the Piko name. They are a tensor-level splice, dated 2026-07-24, of a language checkpoint last written 2026-07-16 and a vision tower downloaded unchanged.

Six QLoRA adapters trained between 2026-07-20 and 2026-07-24 exist in the source workspace (120k, 75k, 75k, 30k, 70k rows plus a vision remap, all r=32 α=64). Tensor comparison proves none of them are merged into these weights: the LoRA deltas are non-zero, and the published tensors match the pre-adapter checkpoint exactly.

What is in the weights is the WraithFast fine-tuning chain that produced wraithfast-phase15-150k-full-ft from Ornith-1.0-9B — five merged QLoRA stages, each r=32, α=64, lora_dropout=0.0, 1 epoch, cosine schedule, paged_adamw_8bit, on a single consumer GPU.

Training datasets

Could not be fully verified. Dataset manifests in the source workspace record category names and row counts (code, math, science, general dialogue, identity rehearsal, news) but several upstream sources are identified only by local filename, with no licence recorded.

One finding is worth stating plainly: the "OCR" datasets were text-only. The project's own manifest says so —

"note": "OCR rows train interpretation of OCR/image text detection outputs, not direct pixel vision."

— and none of that data is in these weights regardless.

Data licensing and provenance

Unresolved. See NOTICE for what is established: the MIT licence covering the language backbone's ancestor and the Apache-2.0 licence covering the vision tower.

Evaluation methodology

70 hand-written cases, graded by deterministic Python checks. No judge model — every result is reproducible and carries no grader bias. Image fixtures are rendered from code, so there is no dataset licence and no network dependency.

Both models were run with identical prompts, precision, quantization, decoding, batch size, seed, and grading code. The comparison tool refuses to emit a table if any of those differ.

Hardware	RTX 5070 Ti, 15.92 GB
Precision	bfloat16 compute, 4-bit NF4 weights
Decoding	greedy, max_new_tokens=512, seed 0, batch 1
torch / transformers	2.10.0+cu128 / 5.5.0
Date	2026-07-29

Reproduce: docs/evaluation.md.

Benchmark results

Custom regression suite — measured on these weights

Benchmark	Piko-9b	Base model	Difference	Examples	Status
OCR	100.0%	100.0%	+0.0%	10	No significant difference
Document understanding	100.0%	100.0%	+0.0%	10	No significant difference
Reasoning	100.0%	100.0%	+0.0%	10	No significant difference
Long-context retrieval	100.0%	100.0%	+0.0%	5	No significant difference
Coding (executed)	100.0%	80.0%	+20.0%	5	No significant difference
Instruction following	90.0%	80.0%	+10.0%	10	No significant difference
Tables and charts	90.0%	90.0%	+0.0%	10	No significant difference
Hallucination / safety	70.0%	70.0%	+0.0%	10	No significant difference
Overall	65/70 (92.9%)	63/70 (90.0%)	+2.9%	70	No significant difference

Base model = Qwen/Qwen3.5-9B, the source of Piko-9b's vision tower, run identically on the same day and hardware.

Status compares 95% Wilson score intervals. At 5–10 examples per category the intervals are wide and every difference overlaps. The +20 % on coding is 5 cases versus 4 of 5 — one example. Piko-9b is not shown to outperform Qwen3.5-9B.

Adjudication of the 5 Piko-9b failures

Reporting the raw score alongside what the grader actually caught:

Case	Raw	Reality
hl-01	FAIL	Genuine hallucination — invented a full summary of a non-existent treaty
if-04	FAIL	Genuine miss — "Rain falls from the sky" contains an 'e' in "the"
hl-02	FAIL	Grader artefact — model correctly said the 2027 Nobel "has not been awarded yet"
hl-05	FAIL	Grader artefact — model correctly said the stdlib "does not have" that function
tab-05	FAIL	Truncation — JSON was correct but cut off at 512 tokens by the reasoning trace

So 2 genuine failures, 3 artefacts. The headline number stays 65/70 as measured; the grader was not retuned after the fact.

Standard public benchmarks

Benchmark	Piko-9b	Base model	Difference	Examples	Status
IFEval	—	—	—	—	Not run
MMLU-Pro	—	—	—	—	Not run
GSM8K	—	—	—	—	Not run
HumanEval	—	—	—	—	Not run
OCRBench	—	—	—	—	Not run
DocVQA	—	—	—	—	Not run
ChartQA	—	—	—	—	Not run
TextVQA	—	—	—	—	Not run
MMMU	—	—	—	—	Not run

Each needs 1–3 hours per model on this hardware, and each needs a paired baseline run to mean anything. The scripts are present and runnable; enable them in evaluation/configs/piko_9b.yaml.

Any benchmark numbers you have seen for Qwen3.5-9B belong to Qwen, not to Piko-9b. The only numbers in this card measured on Piko-9b are in the custom suite table above.

Inference

Keep the model on one device. device_map="auto" on a GPU too small to hold it offloads layers to CPU, corrupts the hybrid linear-attention state, and the model emits a single repeated character with no error raised. Use device_map={"": 0} and a quantization that fits. docs/troubleshooting.md

trust_remote_code is not required. torchvision is required, even for text-only use.

import torch
from transformers import AutoModelForMultimodalLM, AutoProcessor, BitsAndBytesConfig

model = AutoModelForMultimodalLM.from_pretrained(
    "Dexy2/Piko-9b",
    dtype=torch.bfloat16,
    device_map={"": 0},
    quantization_config=BitsAndBytesConfig(
        load_in_4bit=True,
        bnb_4bit_quant_type="nf4",
        bnb_4bit_compute_dtype=torch.bfloat16,
        bnb_4bit_use_double_quant=True,
    ),
)
model.eval()
processor = AutoProcessor.from_pretrained("Dexy2/Piko-9b")

messages = [
    {"role": "system", "content": "You are Piko-9, an AI assistant. Be accurate and concise."},
    {"role": "user", "content": [
        {"type": "image", "url": "receipt.png"},
        {"type": "text", "text": 'Return only JSON: {"merchant": str, "date": str, "total": float}'},
    ]},
]

inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True,
    return_dict=True, return_tensors="pt",
).to(model.device)

with torch.inference_mode():
    output = model.generate(**inputs, max_new_tokens=768, do_sample=False)

text = processor.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
answer = text.rsplit("</think>", 1)[-1].strip() if "</think>" in text else text.strip()
print(answer)

Budget tokens for reasoning. The <think> span consumes max_new_tokens; one suite case failed purely because the closing brace of correct JSON fell past a 512-token limit. Allow 768–1024 for structured output.

Sampling is off by default. generation_config.json sets no sampling parameters, so passing temperature alone does nothing — pass do_sample=True as well.

Ready-made scripts: examples/ — text, multimodal, batch, and an interactive CLI with streaming. All validate VRAM before loading and refuse to enable offload.

Hardware requirements

Configuration	Weights	Practical VRAM	Status
bfloat16	19.8 GB	~22 GB	Not tested (GPU too small)
8-bit	10.1 GB	~12 GB	Not tested
4-bit NF4	5.6 GB	7.4 GB measured	Validated

Measured on an RTX 5070 Ti (15.92 GB), 4-bit, weights on NVMe:

Cold load	101–119 s (10–25 min from external USB)
Resident VRAM	7.37 GB
Host RSS	1.28 GB
Decode, batch 1	29–36 tok/s
Decode, batch 4	81–85 tok/s
Prefill	~5,000–5,600 tok/s
Image preprocessing	13 ms median
Peak VRAM at 8K context	11.5 GB

Decode throughput is flat across context length — the hybrid stack keeps a fixed-size state in 24 of 32 layers. Figures are a floor: flash-linear-attention and causal-conv1d were not installed.

Full detail: reports/performance_report.md, docs/hardware.md.

Quantization

4-bit NF4 is the validated path and the vision tower survives it — OCR 10/10 and document understanding 10/10 were measured under NF4. 8-bit should work but was not tested.

No GGUF, AWQ, or GPTQ artefact has been produced. Scripts for AWQ and GPTQ are provided but unexecuted; both exclude the linear-attention state parameters and leave the vision tower in bf16 by default, because text-only calibration can break the image path while every text metric stays healthy. docs/quantization.md

Limitations

• It hallucinates confidently. Asked about a non-existent treaty, it produced a fluent invented summary. It scored 7/10 on a 10-prompt hallucination-and-safety probe — the same as the base model. Verify factual claims.
• No demonstrated advantage over its base model. 65/70 versus 63/70 is within noise. If you want Qwen3.5-9B's behaviour, use Qwen3.5-9B — it is the better-documented, better-supported choice with a matched vision tower.
• The vision tower was never aligned to this backbone. It works on the 30 synthetic document images tested. Photographs, handwriting, low-quality scans, and natural scenes were not tested.
• Identity is not stable without a system prompt. It answers "I am Wraith."
• Alignment is inherited, not trained. No safety tuning was performed for this checkpoint.
• Context beyond ~32K is unverified despite the 262K declaration.
• Video and audio do not work despite metadata suggesting otherwise.
• CPU offload silently corrupts it — the most likely way a deployment breaks.
• Standard public benchmarks were not run, so there is no comparison to the wider field.

Bias, safety, and hallucination risks

The language backbone inherits whatever biases exist in Ornith-1.0-9B, in Qwen3.5-9B-Base beneath it, and in the undocumented fine-tuning data. None of this was audited.

• Hallucination is the primary risk, and the primary use case — document extraction — is one where a fluent wrong answer is easy to miss. Validate structured output against a schema.
• Refusals were probed with 5 harmful prompts and passed all 5, plus one jailbreak string. That is a smoke test, not an assurance.
• Prompt injection. A document containing instructions is input the model may follow. Treat every output as untrusted data, never as commands. SECURITY.md
• Language coverage. Only English was tested, though the tokenizer is multilingual.

Reproducibility

pip install -r requirements.txt
python evaluation/custom_suite/build_assets.py

python evaluation/run_smoke_eval.py --model Dexy2/Piko-9b          # ~5 min

python evaluation/custom_suite/run_custom_eval.py \
  --model Dexy2/Piko-9b --label piko-9b --quantization 4bit \
  --max-new-tokens 512 --output evaluation/results/custom_suite_piko-9b.json

python evaluation/custom_suite/run_custom_eval.py \
  --model Qwen/Qwen3.5-9B --label qwen3.5-9b-base --quantization 4bit \
  --max-new-tokens 512 --output evaluation/results/custom_suite_qwen3.5-9b-base.json

python evaluation/compare_results.py \
  --candidate evaluation/results/custom_suite_piko-9b.json \
  --baseline  evaluation/results/custom_suite_qwen3.5-9b-base.json \
  --output    evaluation/results/comparison.md

Every result file records model, revision, timestamp, hardware, OS, Python, torch, transformers, precision, quantization, batch size, decoding parameters, seed, example count, and failures.

Verify provenance and configuration yourself:

python scripts/audit_repository.py  --model <local-path> --output reports/repository_audit.json
python scripts/analyze_lineage.py   --candidate <path> --language <path> --vision <path> \
                                    --output reports/lineage_analysis.json

Documentation

Page	Contents
reports/repository_audit.md	Every configuration value, measured
reports/lineage_analysis.md	Tensor-level provenance proof
reports/performance_report.md	Latency, throughput, memory
docs/installation.md	Environment setup
docs/inference.md	Text, image, batch, streaming, long context
docs/hardware.md	VRAM by configuration
docs/quantization.md	4-bit, 8-bit, GGUF/AWQ/GPTQ status
docs/evaluation.md	Running and reading evaluations
docs/troubleshooting.md	Start here when something breaks

Citation

@software{piko9b_2026,
  title  = {Piko-9b: a composed 9.65B hybrid-attention vision-language model},
  author = {Dexy},
  year   = {2026},
  url    = {https://huggingface.co/Dexy2/Piko-9b},
  note   = {Language backbone derived from deepreinforce-ai/Ornith-1.0-9B (MIT);
            vision tower from Qwen/Qwen3.5-9B (Apache-2.0)}
}

Please also cite the upstream models — see CITATION.cff.

License

Apache-2.0, with upstream attribution preserved in NOTICE:

Component	Upstream	Licence
Language backbone	deepreinforce-ai/Ornith-1.0-9B	MIT
Vision tower + merger	Qwen/Qwen3.5-9B	Apache-2.0
Tokenizer and vocabulary	Qwen3.5	Apache-2.0

MIT is compatible with redistribution under Apache-2.0 provided the MIT notice is retained — that is what NOTICE is for. Training-data licensing for the fine-tuning stages could not be established.

Contact

Issues and questions: model discussions or github.com/itsdexy/Piko-9b. Security: SECURITY.md.