Even Big Lebowski doesn’t know where the money went: Meta’s "Superintelligence" is a sub-$5 basement-tier Frankenstein
Let’s talk about the Elephant in the room. Mark Zuckerberg is all over the news right now, loudly claiming that Meta is building a "personal superintelligence" for every human and demanding hundreds of thousands of H100s/B200s to "train the future."But when you actually open the repository of your latest "state-of-the-art agentic masterpiece" Muse-Glimmer-30B and look at the configs, it feels like this entire model was slapped together by a drunk intern over a weekend to fit a quarterly KPI. This isn't AI engineering; this is corporate resource laundering.Let’s break down this structural circus piece by piece:1. The Triple Bottleneck / Projector NightmareYour vision encoder is a tiny, generic ViT-G/14 with a hidden size of 1536. Your language backbone expects a hidden_size of 6656. Instead of natively training a multimodal model like any competent lab in 2026 (look at Qwen or Kimi 3), you guys just bought a handful of cheap adapters from AliExpress.First, you compress the 1536 vision tokens down to a bottleneck of "projector_hidden_size": 4096.Then you linearly upscale it to "out_hidden_size": 6144 (probably because you copy-pasted the projector head from an older Llama 3.2 artifact and were too lazy to retrain it).And finally, you forcibly pad or stretch it from 6144 to 6656 right before feeding it into the LLM!Are you serious? You are passing low-dimensional visual data through a sequence of non-linear (GELU) and linear interpolations into a highly non-linear causal probability space. You are injecting flat, upscaled noise into a 6656-dimensional semantic manifold. No wonder this model suffers from cortical blindness and starts hallucinating the moment it looks at a basic UI screenshot or small terminal logs.2. The Tokenizer Config: Copy-Paste CrimeThe tokenizer_config.json is a work of pure comedy.Why are there over 2000 reserved special tokens (<|reserved_special_token_2|> to 2047) just bloating the embedding matrix and wasting VRAM during initialization? Did someone forget to delete their scrap vectors?Why is the dictionary filled with video tokens (<|vid_start|>, <|vid_frame_separator|>) when this specific 30B model does not support video? You literally didn't even bother to clean up the vocabulary from your internal Muse Spark/Llama 4 test runs.And the crown jewel: embedding raw regular expressions inside the tokenizer config to parse XML-like tags (atem:parameter). Forcing an agentic LLM to generate pseudo-HTML and relying on rigid regex strings for tool calling is a structural design flaw. If the model misses a single space, the regex breaks, and the agent freezes. Is this 2018?3. Slided-Attention Leaks: Recycled GimmicksUsing a [Local, Local, Local, Global] hybrid pattern with a 2048 sliding window is not "innovation." It’s a recycled, desperate bandage from the Gemma 2/Mistral era. We all know how these hybrid architectures behave under real pressure. Sure, it looks nice on paper with a "132k context," but on long context chains, the information travels through the network in delayed hops. The model completely loses track of instructions placed in the middle ("Lost in the Middle" phenomenon), and the KV-cache management on local layers creates massive throughput degradation.4. DFlash: A Hardware-Locked Cop-OutInstead of training a proper, lightweight speculative 2B auto-regressive draft model (like you did for Llama 3), you came up with this Block-Diffusion DFlash monster. A 5GB diffusion draft head that requires massive parallel matrix multiplications (GEMM) just to guess packages of tokens. It’s completely useless on consumer hardware like Mac or mid-tier GPUs because it triggers OOM or massive offloading latency. It only achieves your bloated "233 tokens/sec" benchmark on top-tier Nvidia rigs (RTX 5090 / server clusters). This is not democratizing AI; this is optimizing for your own internal data centers.Serious Question: Where is the Budget Going?Meta has an army of thousands of elite Ph.D. researchers and billions of dollars in computational budget. So why does your open-source release look like a stitched-together Frankenstein monster made of incompatible, recycled weights held together by blue duct tape?Stop shouting about "AGI for every primate" from the stage when your actual deployment architecture is a cascade of lazy engineering compromises. We deserve native multimodal architectures, clean vocabularies, and real engineering—not this corporate gaslighting.Bravo!!!
I don't disagree with all of this, but how about taking the narrative out of it and formatting with paragraphs, bullet points and lists so we can actually read what you're saying. On a M5 Pro it performs reasonably (better than Qwen 3.6 or Gemini 4 models of equivalent sizes on some tasks and worse on others, which is exactly what the release stats say.
If you wanted to ignore the structural facts, why even reply? Just to type some empty corporate buzzwords?Let’s talk about your "reasonable performance" on that M5 Pro. You do realize that from an architecture and hardware perspective, the M5 Pro is a massive downgrade compared to even the old M1 Max when it comes to memory bandwidth, right? Your narrow memory bus completely chokes on long contexts, and since ExecuTorch doesn’t have native, hardware-optimized block-diffusion compilation for Apple Metal, your 5GB DFlash draft head is running as a bloated, sub-optimal sequence of matrix operations. You are literally burning your Unified Memory throughput on a hardware-locked cop-out, getting maybe a miserable 1.5x speedup, and calling it "reasonable."And your only defense is "well, it matches the release stats on some tasks"? Seriously? Are benchmarks the only thing that keeps your hype-worshipping tech bubble alive? Anyone with half a brain in AI engineering knows how Meta cooked those stats by forcing Qwen and Gemini to run on their rigid ATEM XML-prompting templates to artificially tank their scores.If you love being fed low-quality engineering on a silver platter just because it has a Meta logo on it, go ahead and make a direct financial donation to Mark Zuckerberg. But unless you have a serious, mathematical argument to counter the triple-upsampling bottleneck from 1536 to 6656 or the memory leaks of their recycled hybrid sliding-attention, don't just post here "to say something."Keep believing the marketing fairy tales while downloading half a terabyte of unoptimized garbage.Read some books-it will help you))
i was about to download the model but checked the reviews first. this guy is the hero gotham deserves but doesn't have because nobody really understands these things, they're now as easy to use that anybody could so marketing hype is the driving factor. i don't care if he writes without spaces, he said the truth. anyone better be prepared to answer in the merit/substance because the AI Police guy is on fire.
i was about to download the model but checked the reviews first. this guy is the hero gotham deserves but doesn't have because nobody really understands these things, they're now as easy to use that anybody could so marketing hype is the driving factor. i don't care if he writes without spaces, he said the truth. anyone better be prepared to answer in the merit/substance because the AI Police guy is on fire.
Sorry to break it to you, but Adikkkk is most likely just an abliterated/Heretic-uncensored version of Qwen 3.6...
I think the best way to evaluate a model is to try it in your own agents.
Most people don’t care about how a model was trained, or even how it works under the hood. They, including me, simply try it and compare it with other models like Qwen and Gemma. It’s like using an iPhone—you may know nothing about how it was built or how it works internally, but you can still judge whether it’s a good product based on your experience using it.
Why are Qwen models so popular?
The simple reason is that the quality is great.
You may be an LLM engineer, so you might notice things that are not as good. I hope this is useful feedback and helps Meta improve their models.
Now, I’m looking forward to the Qwen3.8 27B model, which is going to be released this week, and seeing how it compares with this model.
I launched this model unsloth muse glimmer 30b Q8_0 with draft-dflash 3 using the latest llama.cpp with rocm 7.14.0 dual 9060 XT 16 GB. Everything seems to be working fine: output performance is about 28-35 t/s. Unsloth qwen 3.6 27b Q8_0 on the same setup works 35-40 t/s output (it is a bit more performant). However, it works in a comparable way to Qwen 27b. Maybe MTP would be better, but dflash is fine too, it is better than nothing.
I am also waiting for a new models from Qwen, however a competitor from Meta is fine, just let it be. From my experience I will still use Qwen 3.6 27b, but Muse Glimmer 30b will be an additional validator (like Gemma 4 31b). Do not criticize this model too much.
Wow, I didn't realize that simply being able to open a text file, look at matrix dimensions, and do basic arithmetic makes me a "Heretic-uncensored version of Qwen 3.6" in your eyes. Is critical thinking so dead in this community that anyone who doesn't praise corporate garbage is assumed to be a rogue Chinese LLM? This is a supreme testament to the digital cargo-cult you guys live in. To you, a neural network is not mathematics and silicon; it’s literal voodoo magic, and the tech billionaires are your high priests.Shoutout to @gbaromhunterisback for being the only user here who actually understands how reality works. Glad I saved you some disk space from downloading this marketing hype.As for the "I use it like an iPhone, if it works, it works" crowd (@Duonglv and @TenStrip ) — congratulations, you are the ideal target audience for corporate gaslighting. If you weld a fifth square wheel to a car, the car will technically still move forward if you push it with enough horsepower. That is exactly what Meta did here. They took a tiny 1536 vision vector, bloated it 4.3 times through a cascade of linear interpolations just to fit a 6656-token text body, and you guys are clapping because "the screen lights up and it does something."And to @puchuu — thank you for proving my point with your benchmarks. Your dual RX 9060 XT setup literally shows that this model, even with its heavy, hardware-locked 5GB DFlash diffusion crutch, is slower than a properly trained, native Qwen 3.6 27B model operating without any speculative draft companions. You are burning extra VRAM and power just to run a slower, heavily censored validator that spends half its compute checking if its answers are "politically correct."Keep enjoying the meal from Mark Zuckerberg's silver platter. He is a poor, struggling billionaire who desperately needs you to validate his quarterly KPIs by treating his duct-taped Frankenstein as the next step toward AGI.The rest of us will stick to actual engineering.brothers -keep going being resource for corporations)))
this guy hates 2 things. this model and formatting.
agree
good enough for 30B params. not an innovation or masterpiece engineering.
report this bot
I don't even have to use pangram to tell this is ai slop writing. Either a troll or an openai intern lol.
I don't even have to use pangram to tell this is ai slop writing. Either a troll or an openai intern lol.
you are to stupid to use some one even your brain) that why primates like you love eat some sht for meta)
I mean taking the tone away and looking at the arguments, on paper it does, look terrible. In actual testing:
- Vision - quite strong, better than the vast majority of VL models I've used, old or not it works fine. No detected difference between Qwen3.6 27B and Glimmer vision.
- Attention Recall - Glimmer is outscoring 27B, both at BF16 model weights and BF16 KV, same hardware, same build. Code Needle expanded to use 300k token inputs and asked to retrieve functions, line for line. Glimmer scored a 100% accuracy across Rust, JS, TS, Python, CPP, SQL where prompt range it must recall from ranges from 100k to 300k tokens. These are not fed in each request, it gets fed the codebase once, then in multiple turns is asked to reproduce a function. Qwen 3.6 27B scores ~99.8, usually dropping only a line or two across the whole test section.
- Tokenizer, no excuse, they left cruft in from their other models.
- Tool calling: take away the gate keeping neck beard tone and your argument is possibly valid, but not represented at actual run time. Over 1000 tool calls in my evaluation suite, strictly enforced, validated, and rejected if incorrect. 0 failures in normal tool calling, and peer capability in adversarial tool scenarios to the competitors.
- Throughput... yes its reliant on your serve configuration having enough compute to get the DFLASH under the roofline of the system. That said, it does not take a 5090. RDNA4 can do it just fine (not in llama, but then nothing is fast in llama if you care about prefill at all). Seeing > 230 tps decode using RDNA 4. So if you're the type that accepts quantization damage on models, two 9070xts can get you going at the 200+ decode speed with high concurrency capability and 2k+ prefile. ~1400 bucks, well within the window of "personal AI server" at this point.
- Context extension: 131072 extended out to 524288 via simple config change, no scaling, no rope or whatever, and it has zero detectable behavioral regression. I am positive I could find a regression if I extracted the run time numerics, but they're not significant enough to perturb the outputs, e.g. no one is going to notice.
- Output quality, still under evaluation but all things suggest it is a complementary options to the Qwen 27B, having both available covers most bases. So far each excel in different areas, so unbunch your undies and be happy a new option landed that does certain things better than the equivalent sized alternatives.
PS: Muse Glimmer is almost identical speed to Qwen3.6 27B at prefill and slightly behind in decode when dialed in right. Having a wider hidden rather than a deeper seems to be the correct choice and the model is demonstrating some nuance dropped by both 27B and Gemma 31B in certain areas.
The failure mode is quite different than the similar sized dense competitors too, omission vs confident falsehood is a repeatedly observed failure, arguable a much better and easier to detect LLM failure mode.
I mean taking the tone away and looking at the arguments, on paper it does, look terrible. In actual testing:
- Vision - quite strong, better than the vast majority of VL models I've used, old or not it works fine. No detected difference between Qwen3.6 27B and Glimmer vision.
- Attention Recall - Glimmer is outscoring 27B, both at BF16 model weights and BF16 KV, same hardware, same build. Code Needle expanded to use 300k token inputs and asked to retrieve functions, line for line. Glimmer scored a 100% accuracy across Rust, JS, TS, Python, CPP, SQL where prompt range it must recall from ranges from 100k to 300k tokens. These are not fed in each request, it gets fed the codebase once, then in multiple turns is asked to reproduce a function. Qwen 3.6 27B scores ~99.8, usually dropping only a line or two across the whole test section.
- Tokenizer, no excuse, they left cruft in from their other models.
- Tool calling: take away the gate keeping neck beard tone and your argument is possibly valid, but not represented at actual run time. Over 1000 tool calls in my evaluation suite, strictly enforced, validated, and rejected if incorrect. 0 failures in normal tool calling, and peer capability in adversarial tool scenarios to the competitors.
- Throughput... yes its reliant on your serve configuration having enough compute to get the DFLASH under the roofline of the system. That said, it does not take a 5090. RDNA4 can do it just fine (not in llama, but then nothing is fast in llama if you care about prefill at all). Seeing > 230 tps decode using RDNA 4. So if you're the type that accepts quantization damage on models, two 9070xts can get you going at the 200+ decode speed with high concurrency capability and 2k+ prefile. ~1400 bucks, well within the window of "personal AI server" at this point.
- Context extension: 131072 extended out to 524288 via simple config change, no scaling, no rope or whatever, and it has zero detectable behavioral regression. I am positive I could find a regression if I extracted the run time numerics, but they're not significant enough to perturb the outputs, e.g. no one is going to notice.
- Output quality, still under evaluation but all things suggest it is a complementary options to the Qwen 27B, having both available covers most bases. So far each excel in different areas, so unbunch your undies and be happy a new option landed that does certain things better than the equivalent sized alternatives.
PS: Muse Glimmer is almost identical speed to Qwen3.6 27B at prefill and slightly behind in decode when dialed in right. Having a wider hidden rather than a deeper seems to be the correct choice and the model is demonstrating some nuance dropped by both 27B and Gemma 31B in certain areas.
The failure mode is quite different than the similar sized dense competitors too, omission vs confident falsehood is a repeatedly observed failure, arguable a much better and easier to detect LLM failure mode.
Thank you for honestly admitting in your first sentence that "on paper, it looks terrible." That is exactly my point. The architecture is an engineering crime held together by corporate duct tape.Now, let's dismantle your "actual testing" fairy tales:524k context with zero RoPE scaling? Are you kidding me? Muse-Glimmer uses a hybrid [Local, Local, Local, Global] attention pattern where local layers are locked into a strict 2048 sliding window. If you pump 500k tokens into it without any rotary position embedding (RoPE) scaling or interpolation, the phase shifts in the global layers accumulate catastrophic positional noise. Claiming "131k extended to 524k with zero behavioral regression" proves that either your evaluation suite is shallow, or you don't understand how multi-turn context-hopping accumulates attention drift."Omission is better than falsehood." This is the ultimate cope. You are literally confirming that the model has amnesia and drops core functions or instructions from the middle of the context because of the hybrid attention bottlenecks. Forgetting data because the network physically cannot pass information efficiently through 52 layers with a 16:1 GQA ratio is not a "better failure mode"—it's a structural defect.The $1400 "Personal Server" Paradox. Your solution to make their hardware-locked DFlash run at 230 tps is to tell consumer users to go out and buy two RX 9070 XT GPUs? You consider a multi-GPU array running a proprietary, non-standard framework just to execute a heavily quantized 4-bit model with a bloated tokenizer "accessible"? Qwen 3.6 27B achieves superior density and honest native vision out of the box without forcing users into specific hardware configurations or multi-GPU setups just to overcome a heavy diffusion-draft crutch.You can dry your tears and try to defend this Frankenstein all you want, but you cannot change the math. A triple-upsampled vision projection (1536 -> 4096 -> 6144 -> 6656) is a lossy, noisy joke. A tokenizer filled with video-trash and regex is pure laziness.But hey, if you enjoy spending $1400 to watch a model elegantly omit your codebase functions while chanting "Amazing!", that's your financial right. The rest of us prefer clean, un-adulterated native engineering.
You realize the full attention layers are Nope right? Like you claim to have read and looked into the model, but you didn't catch this?! 😀
https://sebastianraschka.com/llm-architecture-gallery/nope/
--max-model-len 524288
--hf-overrides '{"text_config": {"max_position_embeddings": 524288}}'
The only things needing to be set for 512k to work. Directyly from the code Mr. "Knows what he's talking about":
"layer_rope_theta": [
500000.0, // layer 0 sliding
500000.0, // layer 1 sliding
500000.0, // layer 2 sliding
0, // layer 3 ← full attention, NoPE
500000.0, // layer 4 sliding
500000.0, // layer 5 sliding
500000.0, // layer 6 sliding
0, // layer 7 ← full attention, NoPE
... // same 4-layer pattern through layer 51
]
"Omission is better than falsehood." This is the ultimate cope. - Omission can be validated in a harness very easily, it is much harder to verify a non-quantifiable than an omission. It's not omission through lost in the middle, its omission as in the model didn't make a claim it did something it didn't or the code doesn't. This is a know and frequently observed Qwen 27B failure mode, overclaiming task performance in narrative.
Glimmer:
Qwen 27B:
When you've setup and tested them like for like at full precision, I'd love to see your results....
You realize the full attention layers are Nope right? Like you claim to have read and looked into the model, but you didn't catch this?! 😀
https://sebastianraschka.com/llm-architecture-gallery/nope/--max-model-len 524288 --hf-overrides '{"text_config": {"max_position_embeddings": 524288}}'The only things needing to be set for 512k to work. Directyly from the code Mr. "Knows what he's talking about":
"layer_rope_theta": [
500000.0, // layer 0 sliding
500000.0, // layer 1 sliding
500000.0, // layer 2 sliding
0, // layer 3 ← full attention, NoPE
500000.0, // layer 4 sliding
500000.0, // layer 5 sliding
500000.0, // layer 6 sliding
0, // layer 7 ← full attention, NoPE
... // same 4-layer pattern through layer 51
]"Omission is better than falsehood." This is the ultimate cope. - Omission can be validated in a harness very easily, it is much harder to verify a non-quantifiable than an omission. It's not omission through lost in the middle, its omission as in the model didn't make a claim it did something it didn't or the code doesn't. This is a know and frequently observed Qwen 27B failure mode, overclaiming task performance in narrative.
Glimmer:
Qwen 27B:
When you've setup and tested them like for like at full precision, I'd love to see your results....
Oh wow. This is absolutely embarrassing for you. You linked Sebastian Raschka’s blog post about NoPE and thought you just did something smart? 😀You just publicly proved that you have zero understanding of how non-linear token manifolds actually operate under the hood. Let me educate you on the math behind your own copy-pasted config:The NoPE Illusion: NoPE (No Position Embeddings) means the model doesn't use explicit sinusoidal or rotary positional biases on those layers. Instead, the model is forced to learn implicit relative positions directly through the weights of the Q, K, and V matrices during pre-training. If the core weights were trained and aligned within a 132k context envelope, the implicit geometric distances between keys and queries are mathematically bounded by that scale.The 512k Crash: When you force --max-model-len 524288 without any scaling framework, you are feeding the NoPE layers sequence lengths that are completely outside their training distribution. The dot-product attention matrix scores for tokens at distance 400k collapse into out-of-distribution thermodynamic noise. The implicit positional signals blur into absolute mush.The Hybrid Collapse: Because this is a hybrid architecture, your layers 0, 1, and 2 are still strictly bound to a 2048 sliding window using RoPE with a fixed theta. They can never see past 2048 tokens. The only way information travels across your 512k context is by hopping through the global NoPE layers (every 4th layer). But since your global NoPE layers are now drowning in out-of-distribution noise at 512k, the structural bridge collapses. Your network completely fragments, and the layers can no longer communicate long-range semantics.The "Omission" Hype: Passing a primitive, synthetic evaluation harness doesn't mean your model functions in production. When a model suffers from structural omission (dropping code blocks or skipping logic), it’s not being "cautious"—it is experiencing severe attention-sink degradation caused by your broken 16:1 GQA ratio and fragmented hybrid layers.Congratulations, you copy-pasted a JSON config snippet from the repo, completely misunderstood the underlying linear algebra of implicit position learning, and exposed your own lack of expertise.Go back to drinking corporate smoothies and staring at cooked evaluation benchmarks. The math doesn't care about your hype.