TylerMayfield/qwen3.5-9b-cascade-tooluse-adapter

Qwen3.5-9B Cascade Tool-Use Adapter (v2)

LoRA adapter that fine-tunes mlx-community/Qwen3.5-9B-MLX-4bit for the Hermes flat-JSON tool-calling format used in Cascade. Designed for Apple Silicon training via mlx-lm; usable for inference on any MLX-compatible runtime (mlx-lm, vLLM-MLX, etc.).

What this is

A LoRA adapter (rank 16, alpha 32, 16 target layers) trained on 552 records of Hermes-format tool-use data: 30 hand-written seed conversations plus 522 records generated by Qwen3.6-35B-A3B-6bit as a teacher through an omlx-server endpoint.

The adapter teaches the base model to emit tool calls in the flat-JSON Hermes schema:

{
  "name": "get_weather",
  "arguments": {"location": "Vancouver, WA", "units": "imperial"}
}

wrapped in ` tags, rather than the nested-XML format Qwen3.5 emits by default.

Intended use

Plug into Cascade or any OpenAI-compatible tool-calling pipeline that wants Qwen3.5-9B-level reasoning with reliable Hermes-format tool emission. Fits 71-tool libraries; trained on a representative mix of network/sysadmin/productivity/generic tools.

Out of scope: vision, audio, code completion without tool use, and multi-turn dialogue without a tool schema in the system prompt.

How to use

The adapter is a mlx-lm LoRA, not a fused model. You need the base model separately.

from mlx_lm import load, generate

base_model = "mlx-community/Qwen3.5-9B-MLX-4bit"
adapter = "TylerMayfield/qwen3.5-9b-cascade-tooluse-adapter"

model, tokenizer = load(base_model, adapter_path=adapter)

prompt = """<|im_start|>system
You are a function calling AI model. You are provided with function
signatures within <tools></tools> XML tags. ...
<|im_end|>
<|im_start|>user
What's the weather in Portland, OR?
<|im_end|>
<|im_start|>assistant
"""

print(generate(model, tokenizer, prompt=prompt, max_tokens=200))

To produce a single fused model (one safetensors file, no separate adapter load), fuse with mlx_lm.fuse:

mlx_lm.fuse \
  --model mlx-community/Qwen3.5-9B-MLX-4bit \
  --adapter-path ./downloaded-adapter \
  --save-path ./qwen35-9b-tooluse-fused

Training data

Source	Records	Notes
Seed (hand-written)	30	Authored for Cascade scenarios
Synthetic (v1)	522	Generated via Qwen3.6-35B-A3B-6bit teacher
Combined (v2)	552	Seed + synthetic, used for this adapter

Categories: simple_single, parallel_calls, multi_step, error_recovery, no_tool_needed, clarification, irrelevant_tools, multi_turn_context, complex_args, tool_selection. Per-category distribution is documented in the Cascade training repo's PROGRESS.md.

The data was generated with chat_template_kwargs={"enable_thinking": False} on the teacher — reasoning models emit a CoT block by default, which breaks downstream JSON parsing.

Eval results

Evaluated on a held-out set of 30 hand-written Cascade test conversations (70 turns across 10 categories). The base model scores 64.3% turn-exact; this adapter lifts it to 74.3%.

Metric	Base	v1 (552 records)	v2 (this, 552 records)
turn_exact	64.3%	68.6%	74.3%
args_exact	35.3%	44.1%	52.9%
format_ok	98.6%	100.0%	98.6%
schema_valid	97.1%	97.1%	97.1%
hallucinated (n)	3	3	2
missed (n)	2	0	1

Per-category turn-exact:

Category	Base	v1	v2
simple_single	100.0%	100.0%	100.0%
no_tool_needed	100.0%	100.0%	100.0%
parallel_calls	100.0%	100.0%	83.3%
irrelevant_tools	100.0%	100.0%	100.0%
multi_turn_context	83.3%	83.3%	91.7%
clarification	55.6%	55.6%	77.8%
multi_step	44.4%	44.4%	55.6%
complex_args	42.9%	42.9%	42.9%
error_recovery	33.3%	55.6%	66.7%
tool_selection	33.3%	50.0%	50.0%

Limitations of these numbers:

• The eval set is 30 conversations I wrote. It is not an industry standard benchmark (no MMLU, no BFCL ToolBench). Real-world performance will vary.
• N=70 turns is small. The ±10pp swings on complex_args and parallel_calls reflect 1-2 individual turns, not robust category performance.
• Eval prompt format is chatml with no chat_template_kwargs. Other chat templates will produce different results.

What the adapter does NOT do

• It does not add new tools to the model's knowledge. Tools must be described in the system prompt at inference time.
• It does not improve general reasoning. v2 scores worse than the base model on no_tool_needed turns where the model is asked general knowledge questions and gives prose answers.
• It does not change the base model's refusal behavior, identity, or safety training.

Training details

• Base model: mlx-community/Qwen3.5-9B-MLX-4bit (5.6 GB, 4-bit)
• Adapter: LoRA, rank 16, alpha 32, 16 target layers (10.82M trainable params, 0.121% of base)
• Sequence length: 1024 tokens (the M5 Pro 48GB + mlx-lm 0.31.3 ceiling — see "Hardware constraints" below)
• Iterations: 1500 (best val at iter 750, val loss 0.267)
• Learning rate: 1e-4 with linear decay
• Batch size: 1, grad accumulation 1
• Gradient checkpointing: enabled (required for Qwen3.5 hybrid attention + Apple Silicon at this size)
• Mask-prompt: DISABLED. mlx-lm 0.31.3 has a divide-by-zero bug when --mask-prompt is combined with truncation; loss stays on the full sequence instead of assistant tokens only.

Hardware constraints

Trained on Apple M5 Pro, 48 GB unified memory. The seq-length ceiling on this hardware is 1024 (1280 OOMs). See the Cascade training repo's references/mask-prompt-and-mem-ceiling-2026-06.md for full details.

Known follow-ups (not addressed in this adapter)

1. complex_args stays at 43% across v1, v2, and the v3 attempt (v3 was not released). More synthetic data of the same shape did not move it; addressing this needs a structural audit of the 7-test complex_args failures vs the 65+ training examples.
2. parallel_calls regressed 100% → 83% in v2 due to one example where the model over-called a follow-up tool. Recovered in the v3 attempt but v3 regressed other categories; v2 was kept as the production model.
3. The base model emits empty blocks before responses. This is the Qwen3.5 chat template behavior with thinking-mode on; passchat_template_kwargs={"enable_thinking": False}` to suppress.

Provenance

• Training code, generator scripts, eval scripts, and full loss curves are in the Cascade training repo (private).
• Synthetic data was generated locally; no API costs were incurred.
• Base model is mlx-community/Qwen3.5-9B-MLX-4bit from HuggingFace, originally Qwen3.5-9B released under Apache 2.0.

License

Apache 2.0, matching the base model. See LICENSE in this repo.