KAT-Coder-V2.5-Dev-OrnithVision

A multimodal code model combining KAT-Coder-V2.5-Dev's language backbone with Ornith-1.5-35B-A3B's vision tower.

Background

The original Kwaipilot/KAT-Coder-V2.5-Dev does not ship with a vision tower or MTP layers — it is a pure language model. This model adds vision capability by transplanting the vision tower from Ornith-1.5-35B-A3B into the KAT language backbone.

Credits

This model is a fusion of two existing projects:

• Language backbone: KAT-Coder-V2.5-Dev by Kwaipilot
• Vision tower: Ornith-1.5-35B-A3B by ornith-ai
• Quantization: AWQ INT4 variant by cyankiwi

What changed

Component	Source
Language model (35B MoE, 3B active)	KAT-Coder-V2.5-Dev (unchanged)
Visual tower (27-layer ViT)	Ornith-1.5-35B-A3B (transplanted)
Quantization	AWQ INT4 (group size 32)
Total parameters	~35B
Active parameters per token	~3B

Technical details

• Architecture: Qwen3_5MoeForConditionalGeneration
• Vision encoder: Qwen3_VisionTransformer, 27 blocks, hidden_size=1152, 16 attention heads, patch_size=16
• Projector: visual.merger — 2 linear layers + 1 norm, output hidden_size=2048
• Token id: image token = 248056
• Dtype: float16
• Context length: up to 128K (with appropriate KV cache settings)

Usage

from transformers import AutoModelForCausalLM, AutoProcessor

model = AutoModelForCausalLM.from_pretrained(
    "KennyVo/KAT-Coder-V2.5-Dev-OrnithVision",
    torch_dtype="auto",
    device_map="auto"
)
processor = AutoProcessor.from_pretrained("KennyVo/KAT-Coder-V2.5-Dev-OrnithVision")

# Example: image + text
messages = [{"role": "user", "content": [
    {"type": "image"},
    {"type": "text", "text": "Describe this image."}
]}]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)

# vLLM serving
vllm serve KennyVo/KAT-Coder-V2.5-Dev-OrnithVision --dtype float16

Method

1. Loaded visual weights (model.visual.*) from both cyankiwi/KAT-Coder-V2.5-Dev-AWQ-INT4 and cyankiwi/Ornith-1.5-35B-A3B-AWQ-INT4 — 333 tensors, all shapes match
2. Verified all tensor names and shapes are identical between the two models
3. Replaced all 333 visual tensors in KAT with Ornith's weights
4. Preserved all 123,493 non-visual tensors (language model, MoE experts, MTP) unchanged
5. Saved as 5 safetensors shards (~23 GB total)

Reproduction

Want to reproduce this model? See REPRODUCE.md for step-by-step instructions, or grab the transplant script transplant_ornith_vision.py and run it against the two source models.

Note: The entire combining study, Python source code, forensic analysis, and this model card were generated by KAT-Coder-V2.5-Dev itself — a model doing its own repair.

License

Apache 2.0 (inherited from base models)