OliviaRossi/DoubleTrouble-GGUF

⚡ DoubleTrouble-27B (GGUF Suite)

DoubleTrouble-27B is a specialized, surgically merged multimodal model based on the dense Qwen 3.8 27B architecture (64 layers, 5120 hidden dimension, hybrid Gated DeltaNet linear recurrence + Grouped-Query Attention, native vision encoder).

This repository provides official GGUF quantizations, including a full-fidelity BF16 release and an Importance Matrix (imatrix) calibrated IQ3_XXS quantization.

• Base Safetensors Repository: OliviaRossi/DoubleTrouble
• Pre-computed Importance Matrix: OliviaRossi/DoubleTrouble/imatrix.dat

---

🧬 Merge Lineage & Design Philosophy

DoubleTrouble merges two high-performance fine-tunes of the Qwen 3.8 27B dense foundation:

Model	Primary Focus	Role in Merge
DavidAU/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-L-Uncensored	Decensored via Heretic Arbitrary-Rank Ablation (ARA), deep multi-perspective reasoning, complex instruction following.	Uncensored Anchor & Residual Projection Dominant (75%)
Jackrong/Qwopus3.8-27B-Flash	Low-latency flash execution, reduced thinking bloat, strict agentic tool-calling trajectories.	Internal Representation & Speed Specialist (50%)

---

🛡️ Critical Architectural Safeguards

1. Excised Multi-Token Prediction (MTP) Speculative Head

Native Qwen 3.8 checkpoints include an auxiliary speculative draft module (layer 64 / mtp.* / nextn.*). When models are merged, mismatched MTP weights often cause runtime crashes or severe speculative rejection penalties in llama.cpp and vLLM.

• The Fix: The auxiliary 65th layer (model.layers.64.*), mtp.*, and fusion matrices (eh_proj, enorm, hnorm) were excised. config.json was cleaned (mtp_num_hidden_layers purged; architectures set to clean Qwen3_5ForConditionalGeneration).
• Result: 100% stable, crash-free execution across all standard runtimes.

2. Abliteration-Preserving Subspace Blending (APSB)

A standard 50/50 linear average of an abliterated model with an aligned model re-introduces 50% of the refusal vector into the residual stream, resulting in broken loops, stuttering apologies, and "zombie" states.

• The Fix: Transformer internal residual writes occur exclusively through o_proj and down_proj. DoubleTrouble weights o_proj and down_proj 75% in favor of DavidAU's uncensored anchor, while internal attention (q/k/v), MLP (gate/up), and DeltaNet linear attention projections remain balanced 50/50.
• Result: Zero refusal relapse with the speed, reasoning agility, and agentic precision of Qwopus-Flash.

---

📦 Available GGUF Files

File	Size	Split	Quant Method	Target VRAM / RAM	Recommended Use Case
DoubleTrouble-BF16-00001-of-00002.gguf DoubleTrouble-BF16-00002-of-00002.gguf	~52.0 GB (Total)	2 Shards	BF16 (Native)	$\ge 56\text{ GB}$ (A100, H100, Mac 64GB+)	Lossless reference, fine-tuning, high-precision evaluation.
DoubleTrouble-IQ3_XXS.gguf	~11.5 GB	Single	IQ3_XXS (imatrix)	$\le 16\text{ GB}$ (RTX 4080, 3090, 4090, T4, Apple M-Series)	Daily driver. Calibrated with domain importance matrix for high-coherence reasoning at sub-3-bit sizes.

Note on Split BF16: The BF16 release is split into two shards to respect Hugging Face CDN's 50.0 GB single-file limit. llama.cpp, vLLM, and Ollama automatically load all shards when you point them to DoubleTrouble-BF16-00001-of-00002.gguf.

---

🧠 Dynamic Reasoning & Prompting Guide

DoubleTrouble ships with a unified, multimodal chat template featuring dynamic reasoning modes, instant flash bypass, and standard JSON tool calling.

1. Dynamic Reasoning Trigger Tags

You can control the model's thinking depth by including a {REASON:...} tag anywhere in your prompt. The template strips this tag automatically before feeding the prompt to the model:

Trigger Tag	Mode	Behavior
{REASON:flash} or {REASON:none}	Flash Mode	Bypasses thinking entirely (<think>\n\n</think>). Yields immediate, low-latency responses for simple queries.
{REASON:low}	Brief Thinking	Minimal internal reasoning for routine tasks and fast coding.
{REASON:xhigh}	Deep Reasoning	Comprehensive, multi-step verification for complex mathematics, logic puzzles, and difficult debugging.
{REASON:einstein}	Brainstormer Persona	Activates an internal multi-agent panel seeking creative, divergent, and high-burstiness perspectives.
{REASON:spoon}	Deep Research Protocol	Activates the virtual expert research protocol with task framing, hypothesis testing, and structured trade-off evaluation.

Example:

{REASON:flash} Write a Python function to check if an integer is a prime number.

{REASON:einstein} Propose 5 novel product architectures for edge-device local inference.

---

🛠️ Tool Calling Format

DoubleTrouble utilizes the standard Qwen JSON tool-calling schema, ensuring compatibility with vLLM, Ollama, and Open WebUI:

<tools>
{"type": "function", "function": {"name": "get_current_weather", "description": "Get current weather in a location", "parameters": {"type": "object", "properties": {"location": {"type": "string"}}, "required": ["location"]}}}
</tools>

The model outputs tool calls in structured JSON:

<tool_call>
{"name": "get_current_weather", "arguments": {"location": "San Francisco, CA"}}
</tool_call>

Tool results are fed back using <tool_response>:

<tool_response>
{"temperature": "64°F", "condition": "Sunny"}
</tool_response>

---

🚀 Running the Models

1. llama.cpp (CLI)

Running IQ3_XXS:

./llama-cli \
  -m DoubleTrouble-IQ3_XXS.gguf \
  -p "<|im_start|>user\n{REASON:flash} Explain how Gated DeltaNet differs from Mamba.<|im_end|>\n<|im_start|>assistant\n" \
  -n 2048 \
  --temp 0.6 \
  -ngl 99

Running Split BF16:

Point to the first shard; llama.cpp handles the rest:

./llama-cli \
  -m DoubleTrouble-BF16-00001-of-00002.gguf \
  -p "<|im_start|>user\nHello! Who are you?<|im_end|>\n<|im_start|>assistant\n" \
  -ngl 99

---

2. Ollama (Modelfile)

Create a Modelfile:

FROM ./DoubleTrouble-IQ3_XXS.gguf

PARAMETER temperature 0.6
PARAMETER top_p 0.95
PARAMETER stop "<|im_end|>"
PARAMETER stop "<|endoftext|>"

TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
{{ .Response }}<|im_end|>"""

Build and run:

ollama create doubletrouble -f Modelfile
ollama run doubletrouble

---

3. vLLM (OpenAI-Compatible Server)

python -m vllm.entrypoints.openai.api_server \
  --model OliviaRossi/DoubleTrouble-GGUF \
  --model-format gguf \
  --filename DoubleTrouble-IQ3_XXS.gguf \
  --port 8000 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen_25 \
  --max-model-len 32768 \
  --gpu-memory-utilization 0.95

---

⚖️ License & Acknowledgements

• Base License: Apache 2.0 (aligned with Qwen foundation releases).
• Credits:
  • Alibaba Qwen Team for the Qwen3.8-27B dense multimodal foundation.
  • DavidAU for the uncensored, abliterated Fable-Cold-Fusion checkpoint and dynamic reasoning schemas.
  • Jackrong for the high-efficiency Qwopus-Flash reasoning checkpoint.