1-bit Qwen3.8 GGUF output example!
Hey for those asking for an example of the 1-bit 397GB Qwen3.8 quant, here it is!
A reminder this 5TB reduced down to just 397GB. That's a whooping 91% reduction
More tests you can see here: https://www.reddit.com/r/LocalLLaMA/comments/1vnu366/1bit_qwen_38_24t_a95b_unsloth_iq1_s_medium/
Hi Daniel,
thanks for the demonstration.
Do you know if there are benchmarks that compare the Quants to the Original?
I'd like to know how much the IQ-0 quantization degrades the quality.
Thanks
I'm also interested in more comparisons and different benchmarks/tests as well!
Hi Daniel,
thanks for the demonstration.
Do you know if there are benchmarks that compare the Quants to the Original?
I'd like to know how much the IQ-0 quantization degrades the quality.
Thanks
I'm also interested in more comparisons and different benchmarks/tests as well!
We're conducting, unfortunately will require some time
Hi - In addition to the above example. Medium reasoning, 1-bit. For context, this took ~70k tokens in Unsloth Studio, and I don’t have enough context headroom to push it to Extra High reasoning
Acropolis I stole this prompt from Arena Ai Chanel so you can go compare against other generations from big fat models.
Prompt : Create Ancient Athens' Acropolis with a recognizable Parthenon and surrounding rock plateau, in warm Mediterranean light.
If you use Three.js, add an import map (before the module script) mapping "three" and "three/addons/" to the same pinned version, and import only via those names. Never reuse identifiers in the same scope - use descriptive variable names.
SCENE (instant recognition)
Parthenon with correct "temple" proportions: colonnade, pediment, stepped stylobate. Pale marble material with subtle warm tint; columns cast crisp shadows. Rocky Acropolis hill with carved steps/paths. Olive trees scattered (instanced) and distant city massing below (simple). ATMOSPHERE
Warm sun + blue sky; slight haze in distance. Optional: a few small human figures in robes for scale (instanced). CONTROLS
Time slider: morning → high noon → golden hour (color temperature and shadow softness change). Haze slider. Crowd toggle + density slider. Camera: Classic postcard (default), Close architectural inspection, Flyover path. TECHNICAL
Instanced trees/people. Procedural textures for marble variation (subtle). Target >=55 FPS; clamp DPR <= 2; minimal UI. FINAL OUTPUT Generate the full output now. Single self-contained Three.js scene. Let me write it and syntax-check it.
I've tested the IQ2-XS (uses 740Gb Ram) and its super slow to the level of unusable. I've noticed that all Qwen big models are becoming very slow in inference than Kimi or Deepseek on same hardware and similar quantized size(the fastest on CPU is all Kimi models pre K3).
This was like using Colibri method because it's mostly by Xeon CPU with little help of one GPU, but its not Colibri. For this small code writing it takes a whole 5 hours (0.17 tokens/sec). Mostly 90% of time wasted on thinking, which wasn't turned on in launcher and started by default in model itself.
This is first Qwen model which completed my test in fact, but of course i'm not planning to use it, by waste of energy-time.
The result of 5 hours work:
Mozart — "Ah! vous dirai-je, maman" K.265 (Twinkle Twinkle Little Star)
// ============================================================
// Mozart - Ah! vous dirai-je, maman (Twinkle Twinkle) K.265
// ============================================================
TriOsc osc => Envelope env => Gain master => dac;
master.gain(0.35);
env.duration(8::ms);
0.32::second => dur qn; // quarter note
0.64::second => dur hn; // half note
0.05::second => dur gap; // small gap between notes
fun void play(int note, dur d)
{
Std.mtof(note) => osc.freq;
env.keyOn();
(d - 15::ms) => now;
env.keyOff();
15::ms => now;
}
// --- Phrase 1: C C G G A A G | F F E E D D C ---
play(60, qn); play(60, qn); play(67, qn); play(67, qn);
play(69, qn); play(69, qn); play(67, hn);
play(65, qn); play(65, qn); play(64, qn); play(64, qn);
play(62, qn); play(62, qn); play(60, hn);
gap => now;
// --- Phrase 2: G G F F E E D | G G F F E E D ---
play(67, qn); play(67, qn); play(65, qn); play(65, qn);
play(64, qn); play(64, qn); play(62, hn);
play(67, qn); play(67, qn); play(65, qn); play(65, qn);
play(64, qn); play(64, qn); play(62, hn);
gap => now;
// --- Phrase 3: C C G G A A G | F F E E D D C ---
play(60, qn); play(60, qn); play(67, qn); play(67, qn);
play(69, qn); play(69, qn); play(67, hn);
play(65, qn); play(65, qn); play(64, qn); play(64, qn);
play(62, qn); play(62, qn); play(60, hn);
How to run: save as mozart.ck and execute:
chuck mozart.ck
What it does:
- Uses a
TriOsc(triangle wave) for a soft, music-box-like timbre Envelopeshapes each note with a quick attack/release to avoid clicks- Three phrases of the melody with correct rhythm (quarter + half notes)
- MIDI note numbers: C4=60, D4=62, E4=64, F4=65, G4=67, A4=69
P.S.: Qwen3.8-27B failed this test fully in BF16 quality (62Gb Ram), thinking slowing it by x10 times vs non-thinking.
Kimi K3-UD-IQ2_XXS (720Gb Ram) successfully made this test in non-thinking mode by 86 minutes (0,3-0,4 tok/sec) on 10 years old CPU only.
#My Hardware# Intel Xeon E5-2699v4 LGA2011-3 22 cores 44 threads (2016) $110 # Gigabyte C612 chipset 12 RAM slots VGA motherboard year 2016 $150 # Samsung-Hynix ECC RAM 12x64Gb=768Gb ~$900 # VGA monitor # IKEA chair # NO GPU # Run: Trillions Deepseeks, Kimis in Q5-Q6, 400-500billions in BF16, super high quality 1 token/sec