Qxern v6 — int4 deployment bundle
Latent inter-model communication for code: model A (Qwen2.5-Coder-1.5B) encodes a code snippet into 32 latent tokens via a trained Q-Former adapter; a frozen decoder (Qwen3.5-0.8B) answers questions about the code from that latent packet alone — no code text is transmitted. A tiny deterministic AST sidecar (pure ast, no LLM, microseconds) restores exact symbols (function names, arity, literals), and an adaptive router picks latent-only / latent+sidecar / text relay per question type.
Full write-up, benchmarks, notebook and honest limitations: https://github.com/bezvka/Qxern
Contents
| Path | Description |
|---|---|
model_a_int4/ |
Qwen2.5-Coder-1.5B encoder, int4-quantized |
model_b_int4/ |
Qwen3.5-0.8B frozen decoder, int4-quantized |
adapters/qxern_adapter_32tok.pt |
Main v5 Q-Former adapter (32 latent tokens) |
adapters/qxern_adapter_struct_32tok.pt |
v6 adapter retrained with AST-exact targets |
sidecar_router.py |
Standalone AST sidecar + adaptive router (CPU, no LLM) |
qxern_config.json |
Bundle configuration |
teacher_answers.json |
Cached teacher generations (for reproduction) |
| |
Key results
n=30 held-out functions for AST facts, n=50 for SemSim/latency; CodeSearchNet, repo-level split. Computed from the raw per-example run data (results/qxern_v6_results.json in the GitHub repo). Accuracy/SemSim are means, latency is p50.
| System | func name | param count | returns | SemSim | p50 ms |
|---|---|---|---|---|---|
| relay (strengthened text baseline) | 0.70 | 0.63 | 0.83 | 0.43 | 1222 |
| qxern_v5 (pure latents) | 0.00 | 0.13 | 0.90 | 0.24 | 202 |
| hybrid (latents + sidecar, no retrain) | 0.87 | 0.93 | 0.90 | 0.44 | 511 |
| hybrid_struct (latents + sidecar + struct retrain) | 0.87 | 0.73 | 0.90 | 0.42 | 420 |
| |
Guard gates passed for both hybrids (returns ≥ 0.88, SemSim drop vs pure latents ≤ 0.01, names ≥ 0.65, params ≥ 0.60, speedup vs relay ≥ 2×). Retraining without the sidecar fails the gates (names 0.20): architecture, not training, restores exact symbols.
Paired bootstrap vs relay (10,000 resamples, 95% CI): the zero-shot hybrid is significantly better on param count (+0.30 [+0.10, +0.50]) and statistically indistinguishable on names, returns and SemSim, at 2.39× lower p50 latency; hybrid_struct is indistinguishable from relay on all accuracy metrics at 2.91× lower latency.
Why the sidecar is necessary and not a crutch: contrastive probing shows code differing only in identifier names collapses to almost one point in latent space (cos 0.985 renamed vs 0.930 different function) — identifiers physically never reach the decoder. Exact symbols must travel through a symbolic channel.
Usage
from huggingface_hub import snapshot_download
path = snapshot_download("aximi/qxern-v6-deploy-int4")
Then follow the reproduction guide in the GitHub repo.
Limitations
- n=30 eval functions (50 for SemSim/latency), one GPU (RTX 5090), one seed, small models — directional evidence, not a paper-grade eval.
- The names gain (+0.17 vs relay) is not statistically significant at n=30; the significant win is param count.
- Retraining buys speed (420 vs 511 ms), not accuracy: it drops param count to 0.73 vs 0.93 for the zero-shot hybrid.
- SemSim is computed against teacher answers, which favors the relay baseline.
- Latency numbers are hardware-specific.