depwire-slm

LoRA adapter for Qwen3.6-35B-A3B (MoE, 4-bit MLX), fine-tuned to reason about codebase structure (dependency graphs, hotspots, blast radius, health) rather than just code text.

Status: v0.1 — proof of concept. This adapter was trained on template-generated (prompt, completion) pairs derived from graph snapshots. It demonstrates that a small model can be steered toward codebase-structure vocabulary, but its benchmark is keyword-based and does not measure grounded reasoning.

v0.2 (in development) replaces this with a facts-in / reasoning-out design: the depwire-cli graph engine computes the facts deterministically, the model reasons over them, and every answer is verified against the same engine (deterministic oracle eval on real repositories) — not keyword matching.

Trained on graph snapshots produced by the Depwire SDK across 989 open-source repositories (5 languages, permissive licenses). Adapter only — the base model is not redistributed here.

Results (v0.1)

Model	Benchmark (20 Qs)*	Val loss
Qwen3.6-35B-A3B base	25%	—
+ depwire-slm adapter	55% (+30pp)	0.212

* keyword-coverage on a 20-question template benchmark, not a grounded reasoning score

Best checkpoint: iter 2700 out of 3000. Trained with MLX-LM on Apple Silicon (M1 Ultra, 128 GB). 17,556 train / 925 val pairs across 17 question types (impact analysis, hotspot reasoning, blast radius, health scoring, dead code, refactor planning, cross-language edges, change safety, etc.).

Usage

With MLX (Apple Silicon)

pip install mlx-lm
mkdir -p lora-adapters
huggingface-cli download atefataya/depwire-slm adapters.safetensors \
  --local-dir lora-adapters

mlx_lm.generate \
  --model mlx-community/Qwen3.6-35B-A3B-4bit \
  --adapter-path lora-adapters \
  --prompt "Symbol AuthService has 15 dependents. What is the risk of modifying it?"

Training Data

Structural pairs generated from Depwire SDK graph snapshots. Each repo yields a JSON snapshot (health score, hotspots, dead-code candidates, dependency edges) which is expanded into 17 question types about the graph.

• Repos: 989 (filtered from 2,847 processed)
• Snapshots: 989 valid graphs (≥10 nodes)
• Training pairs: 17,556
• Validation pairs: 925
• Languages parsed: TypeScript, JavaScript, Python, Go, Rust

Limitations

• Answers are grounded in graph statistics, not source code semantics. The model will not read your code — it reasons about numbers you feed it (dependents, in-degree, health score, etc.).
• Non-English prompts are out of scope.
• Java, Ruby, C++, Swift, Kotlin were excluded from training data due to tree-sitter WASM parser crashes; the adapter has not seen those graphs.

Repository

Pipeline, training scripts, and benchmark are open source: https://github.com/atef-ataya/depwire-slm (tag v0.1-final)

License

Apache 2.0. Base model (mlx-community/Qwen3.6-35B-A3B-4bit) is subject to its own license — check the Qwen model card before use.