Accio-Lab

/

occamy-1.0

Like 123

Follow

Accio 68

@suszon how is the model, better than qwen 3.6 or benchmaxxed?

We release the ckpts whose formats match the hf requirement. We do not treat speculative decoding as a necessary component, and it might also affect model performance.

@suszon We’ve now published a separate Occamy MTP head, with loading instructions and validation results. It was initialized from the Qwen head and adapted to the frozen Occamy trunk for 512 steps; this is a newly adapted head, not one preserved from the original training run. The main checkpoint is unchanged.

In our bounded SGLang MTP1 tests, heldout acceptance was 87.6% on BF16 and 85.9% on NVFP4. Native three-step MTP also passed our H200 BF16 token/logprob consistency checks, but currently needs the included runtime patch. We haven’t measured speed after the final fix, and haven’t validated this head with vLLM or your int4 stack.

Would love to hear your acceptance and throughput results on the 5090 if you try it. Thanks for offering to test!

Thanks for shipping the head, and for the fast turnaround.

We tested it on our RTX 5090 stack (int4 experts, fp8 dense, our own engine) against the untrained transplant of the Qwen3.6 head onto the Occamy trunk,
same session, same prompts:

So on this stack the head is worth about +17 % over no-MTP, and the trained version ties the transplant. That matches the weights: attention and experts are
identical to the Qwen head, only fc and the two norms moved, and TRAINING.json shows 512 steps at batch 1 on 256 sessions.

If you decide to run it further, three things would likely move acceptance where the projection fit could not:

1. Unfreeze the drafting layers (self-attention and the shared expert first, LoRA is enough), not just fc.
2. Target the trunk's full next-token distribution (KL), not top-1 agreement. Acceptance under sampling depends on the whole distribution.
3. Train on long contexts. Our prompts are 3K to 100K tokens of agent history; 512-token windows are far from that regime.

Happy to rerun acceptance and throughput on the 5090 for any new head you publish.

@suszon Thanks for the same-session comparison, and for offering to test another head. The trained head and transplant are effectively tied here; we shouldn't present this as a meaningful gain from adaptation. The improvement over no-MTP is useful to see on your stack.

Your suggestions on drafting-layer adaptation, a distribution-level objective and longer contexts give us a much better next experiment than simply extending the current run. Before setting that up, could you share the draft depth, acceptance denominator (accepted/proposed draft tokens?), and approximate request count? A few public or redacted representative prompts would also help us keep a held-out comparison aligned with your workload. No need to share private agent histories.

Happy to. Details of the comparison:

Setup

• Engine: our own Rust engine on one RTX 5090. Occamy trunk with int4 (Hadamard-rotated) routed experts, fp8 dense/attention/KV, fp8 lm_head. Head tensors
  loaded from your mtp-trained.safetensors converted to fp8 with block scales, same layout as the Qwen3.6 head.
• Draft depth: 1 token per step (MTP1; our verify graph is M=2: one draft + one verify position).
• Acceptance denominator: accepted draft tokens / proposed draft tokens, summed over all requests in the arm.
• One caveat: our decode runs a hybrid where a context-lookup drafter (prompt-lookup style, n-gram match against the prompt/history) takes a step whenever
  it has a candidate, and the MTP head drafts the remaining steps. So the MTP numbers below are measured on the tokens the lookup drafter did not cover, which
  biases MTP acceptance down relative to a pure MTP1 run. The two arms share this exactly, so the comparison is fair, but the absolute number is not
  comparable to your 87.6 %.

Per arm (same session, same prompts, same binary; only the head file differs)

Prompts

• Greedy: HumanEval problems 0–19 (prompt = the function stub, "complete this function" wrapper) and GSM8K test items 0–19 ("answer with the number on the
  last line"), deterministic decode, max 16K tokens.
• Sampled (8, T=0.6), verbatim:
  1. Write a Python function that parses ISO-8601 durations into seconds, with tests.
  2. Explain in ~300 words why GDN linear attention keeps state in fp32.
  3. Write a bash script that rotates logs in /var/log/myapp keeping 7 days, with comments.
  4. Draft a short design doc for a prefix cache keyed by token ids and image hashes.
  5. Implement quicksort in Rust with a property test.
  6. Summarize the trade-offs between MoE and dense models for single-GPU inference.
  7. Write a SQL query and explain it: top 5 customers by revenue per region last quarter.
  8. Write a haiku sequence (5 haiku) about compression.

Our real workload is agent sessions (tool calls, code edits, 3K to 100K tokens of history) where live acceptance runs around 0.72 to 0.77 for both heads;
those histories are private, but the sampled prompts above are the closest public stand-in for the code/prose mix. If you publish a new head we will run the
identical 48-request battery and report accepted/proposed with and without the lookup drafter so the pure-MTP1 number is directly comparable to yours.

@suszon Thanks for the detailed breakdown and public prompts! The lookup-drafter distinction is especially helpful. Really appreciate you taking the time to test both heads.