⚡ FlashModel-Qwen-3.6-35B-A3B-Maxed

A reasoning / coding / agentic / bilingual fine-tune of Qwen/Qwen3.6-35B-A3B.

Key capabilities:

  • Code: Rust, TypeScript, JavaScript, Python, QML, HTML, CSS, Bash
  • Controllable reasoning effort: put reasoning_effort=low|medium|high|xhigh|max in the system prompt to change reasoning depth — real control, not an artificial length cap
  • Agentic tool use: file read/write, shell, git operations
  • Bilingual EN/RU dialogue (≈22% RU in training)
  • MTP head preserved: mtp.* tensors kept from the original checkpoint so speculative decoding works

GGUF quants: flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF

Training details

  • Method: LoRA (peft), merged into the base weights in this repo
  • LoRA: r=64, alpha=128, dropout=0.05
  • Targets (310 modules): attention q/k/v/o_proj, linear-attention in_proj_a/b/qkv/z + out_proj, shared-expert gate/up/down_proj (routed experts untouched — 85% of weights)
  • Data: 14.4k examples stratified:
    • Reasoning: 6k (OpenR1-Math + simplescaling/s1K, 5 effort levels × ~1200 each)
    • Code (local corpus): 4.9k (Rust 1101, QML 900, JS 900, Py 900, TS 711, CSS 250, HTML 66, Bash 100)
    • Code (instruction): 800 (Magicoder-Evol-Instruct-110K, capped per language)
    • Tool use: 1.5k (NousResearch Hermes + glaive)
    • Bilingual chat: 1.2k (OpenAssistant oasst2, 21% RU)
  • Loss target: ~1.4 to avoid memorization; stopped at step 1500 of 2436 (train loss 0.43)
  • Context length in training: 16k tokens (longer examples were dropped, not truncated)
  • Effort data: reasoning traces regenerated at 5 budget levels so the marker maps to real behaviour
  • Hardware: single A100-80GB, 4-bit base + bf16 LoRA, ~4 h

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="bfloat16", device_map="auto")

msgs = [
    {"role": "system", "content": "You are a coding and reasoning assistant.\nreasoning_effort=high\nAdjust the depth of your reasoning to the level given above."},
    {"role": "user", "content": "Напиши функцию бинарного поиска на Python."},
]
enc = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt", return_dict=True)
out = model.generate(enc["input_ids"].to(model.device), max_new_tokens=800)
print(tok.decode(out[0][enc["input_ids"].shape[1]:], skip_special_tokens=True))

Reasoning effort control

The effort level is read from the system prompt:

<system>
You are a coding and reasoning assistant.
reasoning_effort=medium
Adjust the depth of your reasoning to the level given above.
</system>
Value Behaviour
low 1–2 sentences, minimal reasoning
medium short paragraph
high multi-paragraph reasoning
xhigh long trace, explores alternatives
max full chain-of-thought

This was trained with matched-pairs data (same question at 5 effort levels), so the mapping is behavioural rather than a truncation of the same text.

Known eval results (baseline → finetuned)

Metric Baseline Finetuned
Tool Use (8 scenarios) 6/8 7/8
Agentic (5 tasks) 5/5 5/5
Effort monotonic (low→max) ❌ ✅
RU Perplexity — 5.41

Full benchmark suite (HumanEval, MBPP, GSM8K, MMLU, SWE-bench) is planned.

Licence

Apache-2.0, inherited from Qwen/Qwen3.6-35B-A3B.

Downloads last month
81
Safetensors
Model size
35B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed

Finetuned
(378)
this model
Quantizations
3 models

Datasets used to train flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed