Koa AI v2 (vamazing/Koa-AI-v2-code-9B)

Koa AI v2 is an advanced, instruction-tuned language model engineered for agentic workflows, complex code synthesis, multi-turn tool interaction, and step-by-step technical reasoning. It is a fine-tuned 9B parameter language model built on the Qwen 3.5 9B architecture. It is optimized for lightweight text generation and coding tasks.

This repository provides both 16-bit merged weights (.safetensors) exported directly from checkpoint-270 (optimal loss: 0.5614).

---

🛠️ Model Overview & Specifications

Feature	Specification
Model Name	Koa AI v2 (Code)
Base Architecture	Qwen 3.5 9B
Parameters	9 Billion
Precision Formats	16-bit Merged (bf16)
Context Length	32,768 tokens native (Fine-tuned at 2,048 sequence cap)
Fine-Tuning Method	QLoRA (r = 16, alpha = 32, Dropout = 0.0)
Target Modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Primary Frameworks	Unsloth, PyTorch, Hugging Face Transformers, llama.cpp

---

📡 Modalities & Capabilities

Supported Modalities

• Text Input → Text/Code Output: Structured reasoning, code synthesis, documentation, and agentic trajectory logging.
• Tool & Function Calling: Formatted structured output for executing terminal/bash commands, tool calls, and API integrations.

Core Capabilities

• Agentic Coding & Execution: Fine-tuned on agentic interaction traces to analyze system states, execute terminal commands, write code, and autonomously debug execution errors.
• Qwen 3.5 9B Foundation: Leverages deep multi-step problem solving across complex multi-file codebases and algorithm challenges.
• Structured Reasoning: Native support for deep logic, architectural planning, and structured chain-of-thought processing.

---

Quickstart

Option 1: Python / Transformers (16-bit Safetensors)

pip install transformers torch accelerate unsloth

from unsloth import FastLanguageModel

# 1. Load the model and tokenizer
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "your-username/Koa-AI-v1",
    max_seq_length = 2048,
    load_in_4bit = True,
)
FastLanguageModel.for_inference(model)

# 2. Define prompt using ChatML template
messages = [
    {"role": "system", "content": "You are Koa AI v1, an expert coding agent."},
    {"role": "user", "content": "Write a Python script to monitor GPU VRAM usage."},
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize = True,
    add_generation_prompt = True,
    return_tensors = "pt"
).to("cuda")

# 3. Generate response
outputs = model.generate(input_ids = inputs, max_new_tokens = 512, use_cache = True)
print(tokenizer.decode(outputs[0]))

This qwen3_5 model was trained 2x faster with Unsloth and Huggingface's TRL library.