⚡ TripleTrouble-V3 ⚡

The Triad of Code, Tool Agency, and World Simulation

A unified sovereign agent model merging three apex Qwen 35B-A3B checkpoints via Decoupled Normalized Geodesic Consensus (NGC), Row-Wise Router Manifold Calibration, and Functional Depth Scheduling.

Model Details • The Triad • Merge Engineering • Serving with vLLM • Transformers

---

🌌 Overview

TripleTrouble-V3 is a 35B-class sparse Mixture-of-Experts (MoE) foundation model created by fusing three divergent, specialized post-trained models of the Qwen 35B-A3B architecture:

1. The Coder (KAT-Coder-V2.5-Dev): Autonomous codebase manipulation, SWE-bench refactoring, repository traversal, and concrete AST syntax trees.
2. The Agent (Ornith-1.5-35B-A3B): Multi-turn API calling, complex structured JSON emissions, mathematical deduction, and competitive programmatic reasoning.
3. The Simulator (Qwen-AgentWorld-35B-A3B): Large World Model (LWM) dynamics, environment state transition modeling (MCP, OS, Web, Android), and next-state observation forecasting.

By moving beyond naive weight averaging and avoiding the rank collapse of pseudo-base TIES pruning, TripleTrouble-V3 preserves the singular value spectrum of all 256 experts and maintains the spectral radius of Qwen's hybrid Gated DeltaNet linear recurrence dynamics.

---

🧬 The Triad

flowchart TD
    subgraph MERGED ["⚡ TripleTrouble-V3"]
        ROOT["<b>TripleTrouble-V3</b><br/>34.7B MoE • ~3.3B Active per token"]
    end

    ROOT -->|"40% Weight"| KAT["💻 <b>KAT-Coder-V2.5-Dev</b><br/>• Code & SWE-bench Refactoring<br/>• AST & Syntax Trees<br/>• Terminal Command Logic"]
    ROOT -->|"35% Weight"| ORN["🦅 <b>Ornith-1.5-35B-A3B</b><br/>• Multi-Turn Tool Calling<br/>• MCP Protocol Execution<br/>• Structured JSON & Math"]
    ROOT -->|"25% Weight"| AGW["🌐 <b>Qwen-AgentWorld-35B</b><br/>• Large World Model (LWM)<br/>• Environment Dynamics<br/>• State Observation Loops"]

    classDef default fill:#1e293b,stroke:#475569,stroke-width:1px,color:#f8fafc;
    classDef highlight fill:#4338ca,stroke:#818cf8,stroke-width:2px,color:#ffffff;
    class ROOT highlight;

---

🧮 Merge Mathematics

Traditional model mergers suffer from two major failure modes on 35B hybrid MoE models:

• Naive Linear Soups: Cause high-dimensional variance collapse ($\mathbb{E}[|\sum w_i \theta_i|] \ll \mathbb{E}[|\theta_i|]$), dampening activations across 40 layers.
• Pseudo-Base TIES/DARE: Truncate 70% of delta coordinates, destroying the singular value spectrum of small ($2048 \times 1024$) MoE experts and shattering the Householder contraction operator of Gated DeltaNet attention.

TripleTrouble-V3 was built using a continuous hyperspherical framework designed specifically for this architecture:

1. Decoupled Normalized Geodesic Consensus (NGC)

Directional consensus and magnitude scaling are strictly decoupled. Checkpoints with larger gradient norms cannot skew the consensus angle:

$${\widehat{W}}_i=\frac{W_i}{\mathrm{\parallel }{W}_i{\mathrm{\parallel }}_F}\backslash hat\{W\}\_i\; =\; \backslash frac\{W\_i\}\{\backslash |W\_i\backslash |\_F\}$$

$$D_{\text{consensus}}=\sum _{i=1}^3{w}_i(l){\widehat{W}}_i,\widehat{D}=\frac{D_{\text{consensus}}}{\mathrm{\parallel }{D}_{\text{consensus}}{\mathrm{\parallel }}_F}D\_\{\backslash text\{consensus\}\}\; =\; \backslash sum\_\{i=1\}^3\; w\_i(l)\; \backslash hat\{W\}\_i,\; \backslash quad\; \backslash hat\{D\}\; =\; \backslash frac\{D\_\{\backslash text\{consensus\}\}\}\{\backslash |D\_\{\backslash text\{consensus\}\}\backslash |\_F\}$$

$$W_{\text{final}}=\widehat{D}\times (\sum _{i=1}^3{w}_i(l)\mathrm{\parallel }{W}_i{\mathrm{\parallel }}_F)W\_\{\backslash text\{final\}\}\; =\; \backslash hat\{D\}\; \backslash times\; \backslash left(\; \backslash sum\_\{i=1\}^3\; w\_i(l)\; \backslash |W\_i\backslash |\_F\; \backslash right)$$

2. Row-Wise Router Manifold Calibration

In an MoE with 256 routed experts, the router gate $W_{\text{gate}} \in \mathbb{R}^{256 \times d_{\text{model}}}$ consists of 256 distinct hyperplanes $g_e$. Global matrix scaling alters individual expert activation probabilities. We calibrate every expert hyperplane row-by-row:

$$g_{e,\text{final}}=\frac{\sum _i{w}_i\frac{g_{e,i}}{\mathrm{\parallel }{g}_{e,i}{\mathrm{\parallel }}_2}}{{\parallel \sum _i{w}_i\frac{g_{e,i}}{\mathrm{\parallel }{g}_{e,i}{\mathrm{\parallel }}_2}\parallel }_2}\times \left(\sum _{i=1}^3{w}_i\mathrm{\parallel }{g}_{e,i}{\mathrm{\parallel }}_2\right)g\_\{e,\; \backslash text\{final\}\}\; =\; \backslash frac\{\backslash sum\_i\; w\_i\; \backslash frac\{g\_\{e,\; i\}\}\{\backslash |g\_\{e,\; i\}\backslash |\_2\}\}\{\backslash left\backslash |\; \backslash sum\_i\; w\_i\; \backslash frac\{g\_\{e,\; i\}\}\{\backslash |g\_\{e,\; i\}\backslash |\_2\}\; \backslash right\backslash |\_2\}\; \backslash times\; \backslash left(\; \backslash sum\_\{i=1\}^3\; w\_i\; \backslash |g\_\{e,\; i\}\backslash |\_2\; \backslash right)$$

This guarantees router logit sharpness, prevents entropy collapse, and keeps expert dispatch distributions intact.

3. Functional Depth-Aware Dynamic Scheduling

Layer weights dynamically modulate across the 40 layers to match functional network mechanics:

Depth Bracket	Target Specialization	Dominant Driver	Allocation Distribution
Layers 0 – 11 (Shallow)	Syntax & Token ASTs Code grammar, indentation, and subword abstractions	KAT-Coder ~53%	KAT 🟩🟩🟩🟩🟩 ORN 🟦🟦 AGW 🟨🟨
Layers 12 – 27 (Middle)	Latent World Simulation Environment state tracking and entity transition dynamics	AgentWorld ~35% (Peak)	KAT 🟩🟩🟩 ORN 🟦🟦🟦 AGW 🟨🟨🟨🟨
Layers 28 – 39 (Deep)	Action Policy & Tool Execution Function schemas, JSON formatting, and multi-turn planning	Ornith ~47%	KAT 🟩🟩🟩 ORN 🟦🟦🟦🟦🟦 AGW 🟨🟨

---

⚙️ Architectural Specifications

Parameter	Value
Total Parameters	34.7 Billion
Active Parameters per Token	~3.3 Billion
Layers	40
Routed Experts	256 (Top-8 active per token)
Shared Experts	1 (Always active)
Attention Mechanism	Hybrid Gated DeltaNet (Linear Attention) + GQA
Attention Heads	16 Query Heads / 2 Key-Value Heads
Hidden Dimension ($d_{\text{model}}$)	2048
Intermediate Dimension ($d_{\text{ffn}}$)	1024 (per routed expert)
Vocabulary Size	248,320
Context Window	131,072 tokens

---

🚀 Fast Inference with vLLM

Thanks to its sparse MoE architecture, TripleTrouble-V3 runs at the inference speed of a ~3.3B dense model while delivering 35B-scale reasoning.

Installation

pip install vllm>=0.6.0

Launch an OpenAI-Compatible API Server

vllm serve OliviaRossi/TripleTrouble-V3 \
  --tensor-parallel-size 2 \
  --max-model-len 32768 \
  --gpu-memory-utilization 0.90 \
  --trust-remote-code

(For a single 80GB GPU, run using FP8 or AWQ quantization: --quantization fp8).

---

💻 Quickstart: Transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "OliviaRossi/TripleTrouble-V3"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True
)

messages = [
    {
        "role": "system",
        "content": (
            "You are TripleTrouble, an expert agent combining deep codebase mastery, "
            "rigorous multi-turn tool planning, and environment simulation capabilities."
        )
    },
    {
        "role": "user",
        "content": "Analyze the concurrency bottlenecks in an asynchronous Python producer-consumer queue and write a resilient implementation using asyncio.Queue."
    }
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=2048,
    temperature=0.6,
    top_p=0.9,
    repetition_penalty=1.05
)

response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(response)

---

🛠️ Recommended Sampling Parameters

Workload	Temperature	Top-P	Repetition Penalty	Notes
Code Synthesis & Bug Fixing	0.2	0.85	1.02	Maximizes syntax precision and strict AST adherence.
Tool Calling & MCP Agents	0.3	0.90	1.00	Ensures strict valid JSON schemas and tool-call formatting.
General Problem Solving	0.6	0.92	1.05	Balances creative reasoning with grounded problem deduction.
World Simulation & Planning	0.7	0.95	1.08	Explores diverse trajectory states and action paths.

---

⚖️ License & Attribution

This model is released under the Apache 2.0 License.

Source Models:

• KAT-Coder-V2.5-Dev by Kwaipilot
• Ornith-1.5-35B-A3B by ornith-ai
• Qwen-AgentWorld-35B-A3B by Qwen / Alibaba Cloud

@misc{tripletrouble2026,
  author = {Olivia Rossi},
  title = {TripleTrouble-V3: A Geodesic Manifold Merge of Code, Tool Agency, and World Simulation},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/OliviaRossi/TripleTrouble-V3}}
}