27b ?
Does this scale? I guess more people are interested in Qwen3.8 27B, Gemma4 26B etc. 60tps for Qwen3 4B is not great, but nothing desperate either.
I tried Dspine, Dspark, Dflash, Eagle3 with drafters at various quants levels, their edge over ngram-simple + built-in/external MTP is marginal on older card like V100, not worth the VRAM they occupied. Memdraft looks more interesting for cards with VRAM or computation power constraints.
PyTorch is a massive dependency especially when coming to the different versions in venvs, that's a blocker for me.
I'm definitely going to look into it, soon! You may see a few more republishes as i add more models etc. I suspect I'll need to run more data through it for 27b. I used 9b as a starting point to prove the concept.