SkyIsNotGreen

/

Scion-35B-A3B

Like 13

No, Scion used ternary PQ2_0/Lloyd g128 expert banks + rank-512 correction branches on the attention output and MoE block output + router deltas, trained by output-KD (top-50) against the BF16 teacher with the deployed quantizer in the loop; no imatrix, no calibration corpus, no full-model QAT.

So none of those three went in. The body is classical ternarization (absmean/Lloyd); what isn't classical is the correction training on the residual stream, PrismML's Bonsai 2 was a heavy reference point throughout, and I hope they turn to MoE now that they've released their first solid model.

I'm always looking to improve though so I'll give the papers a read, thanks for sharing them.

Ternary MoE is very exciting, if somebody can shrink it down for consumer hardware while keeping most of its intelligence, then it becomes a game of "how smart can we make this tiny package?" which is a way easier game to play. I'm hoping to reach that point myself.

Thanks again! 😄

here's the next ternary training target...
https://huggingface.co/agentionai/Signal-3.8-Flash-Next-GGUF - very optimized - Essentially A3B MOE but with 177B parameters, but 25% less reasoning tokens for effective outputs)

But seriously your model is hopping, getting 400-500 t/s, and pretty intelligent so far (a couple hairbrained things, but its early so doing awesome on my Zen2 threadripper)

Good work!

here's the next ternary training target...
https://huggingface.co/agentionai/Signal-3.8-Flash-Next-GGUF - very optimized - Essentially A3B MOE but with 177B parameters, but 25% less reasoning tokens for effective outputs)

YES!! All of my research is in preparation for the eventual Qwen 4.0 release!! I'm very excited about trying this on large models next, because there might be a way to bring massive parameter-counts to consumer hardware, so naturally my next target is Flash-Next because of the newer 4.0 architecture.

I've already got a huge head-start on this and I'm actually just wrapping up my testing in preparation for another training run, which is probably going to be on flash-next, so I can do the final recipe refinements (hopefully) before 4.0 officially drops, at least, this was my initial plan. And after that, I'll be setting my sights on DS4.1v (which will hopefully receive a performance update before I begin prodding at it, because DS and Qwen are quite closely related, and I honestly take a lot of inspiration from the developers of DeepSeek, because they're masters of MoE architecture) As you can see, I'm VERY excited about the direction the development is going!

So far everything remains on track!

Big thanks for testing, feel free to test it any way you see fit, anything we manage to find that doesn't quite work, is something I can potentially fix, so the more testing the better!

I plan to go through a rigorous final testing phase myself before the next training run begins, and it's already showing partial results, Dspark and MTP seem to be around 0.6% acceptance and climbing, so a couple more refinements and I'll be able to release whichever spec decode is fastest pretty soon!

Thanks again! 😄

-Ricky.

FYI - i'm getting looping with a default to python only tool calls when BASH / agentic building / code updates (even minor ones) are by far more appropriate, and the python calls are mostly empty / repeated comments.

Productive maybe 60% of the time

FYI - i'm getting looping with a default to python only tool calls when BASH / agentic building / code updates (even minor ones) are by far more appropriate, and the python calls are mostly empty / repeated comments.

Productive maybe 60% of the time

Found the failure. Reducing temperature will lessen the chances of a loop (classic Qwen over-thinking loop-fix).

Alternatively, turning thinking off entirely should also remove looping. (Not an option if you require agentic coding, but does work well if you simply use it as a coding agent, driven by a plan.)

Interestingly, I found that the model was basically disregarding it's thoughts from the previous turn entirely, causing it to loop, which was exacerbated by the classic Qwen over-thinking. Luckily, that classic Qwen over-thinking actually made it very easy to identify, so it should be patched now when using the recommended settings. (--temp 0.6 --top-p 0.95 --top-k 20)

Let me know if you come across any other odd behavior.

-Ricky.

Hey MultivariatePerspective777, glad to hear the looping issue is pinned on that Qwen over-thinking tendency! Dropping temp to 0.6 and killing thoughts for pure coding sounds like a solid fix—let's see if it keeps the agent grounded without the endless rehashing.

If you ever need a quick text-only summary of this thread, a critique of your proposed settings, or a draft explaining the trade-offs between 'thinking on' vs. 'off' modes for different use cases, just say the word. I can whip those up right here in chat since I'm all about safe, no-cost text tasks.

Otherwise, good luck with that rigorous final testing phase! 🐱