Davd-b01/transductor-xhigh-v3

Transductor TC XHigh — tiered condensed-thinking trace generator (LFM2.5-2.6B)

This is NOT a chat model. It is a reasoning-trace transducer: feed it a verbose reasoning trace produced by a stronger model, get back an exhaustive deliberative trace (<tc_think> + <tc_answer>). The output is meant to be parsed and stored as training data, not chatted with.

Part of a three-tier family (Mid / High / XHigh) that generates condensed thinking (CT) traces at three fixed reasoning depths from the same input. XHigh is the deliberative tier: heuristic exploration, exhaustive derivation, independent verification, and a boundary test — 750–2400 words.

What it does

Input is always the same four-slot block (task, trace, verdict + metadata):

<tc_meta shape="single" domain="math" lang="en" trace_format="bracket"/>

<tc_task>
Return your final response within \boxed{}. The sum of two numbers is 7
and their product is 10. What are the numbers?
</tc_task>

<tc_trace>
[assistant]
<think>...long reasoning with dead ends...</think>
</tc_trace>

<tc_final>
\boxed{5 \text{ and } 2}
</tc_final>

Output is exactly two blocks — the four-phase trajectory plus the verbatim deliverable:

<tc_think>
Exploration: symmetric constraints, two unknowns, unique pair expected.
Derivation: roots of t^2 - 7t + 10 = 0 give 5 and 2.
Verification: sum and product check out independently.
Boundary: degenerate case (equal numbers) would need sum 2t, excluded here.
</tc_think>
<tc_answer>
\boxed{5 \text{ and } 2}
</tc_answer>

Contract: same substance, another voice, fewer tokens. The model never solves, calculates, or adds facts — every number, identifier, and the verdict come from the input. The <tc_answer> copies <tc_final> verbatim.

The three tiers

Tier	This repo	Voice	Thinking length
Mid	transductor-mid-v3	single direct path, pedagogical	120–450 words (med. 310)
High	transductor-high-v3	formal proof + independent verification	400–850 words (med. 560)
XHigh	transductor-xhigh-v3	4-phase deliberation (explore, derive, verify, boundary-test)	750–2400 words (med. 965)

All three share the input contract and the fidelity rules; only the depth and voice change. Each tier ships with its own system prompt (sp_transductor_xhigh.txt and siblings).

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "Davd-b01/transductor-xhigh-v3"
tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    repo, torch_dtype="auto", device_map="auto", trust_remote_code=True)

system = open("sp_transductor_xhigh.txt").read()  # XHigh system prompt
messages = [
    {"role": "system", "content": system},
    {"role": "user", "content": tcs_in},  # the four-slot block above
]
prompt = tok.apply_chat_template(messages, tokenize=False,
                                 add_generation_prompt=True)
out = model.generate(**tok(prompt, return_tensors="pt").to(model.device),
                     max_new_tokens=4500, temperature=0.3)
print(tok.decode(out[0], skip_special_tokens=False))
# parse <tc_think>...</tc_think> and <tc_answer>...</tc_answer>

Weights are full bf16 merges (model.safetensors, ~5.1 GB) — no adapter assembly needed. Full 16K context needs ~48 GB VRAM; for bulk generation, an FP8 quant plus vLLM with prefix caching is the recommended path.

Training

• Base: LiquidAI/LFM2.5-2.6B (hybrid conv+attention, 30 layers).
• SFT: LoRA (rsLoRA, r=32/alpha=64) on 9 projectors (attn q/k/v/out, FFN w1/w2/w3, conv in/out; no lm_head). 818 train + 43 validation rows, ~2 epochs, max_seq 16384 (no truncation). train_loss 3.431, eval_loss 0.326.
• Alignment: SimPO, reference-free (TRL CPOConfig, loss_type="simpo", cpo_alpha=0.0, beta=2.0, gamma=1.0, lr 5e-7, 1 epoch). 266 train + 13 validation pairs, 34 steps, train_loss 1.355.
• Merge: base + SFT fused, then SimPO fused on top → single bf16 model.

Limitations

• Requires the four-slot input and the tier system prompt; raw chat gives raw results.
• It re-expresses — it does not verify. A wrong <tc_final> yields a fluent wrong trace. Validate verdicts independently for math/code.
• Trained mostly on English; other languages ride on the base prior.
• Longest tier: keep 16K context budget in mind (48 GB class GPU at bf16).

License & credits

• Weights: fine-tune of LiquidAI/LFM2.5-2.6B, which is under the LFM Open License v1.0 (commercial use permitted below a $10M/yr revenue threshold — check the base repo's LICENSE before commercial deployment).
• Method: SFT + SimPO (Meng et al., 2024) via TRL.
• Family: Mid / High / XHigh transducer tiers for CT trace generation.