Qwen

/

Qwen3.8-27B

like 12.2k

Follow

Qwen 99.5k

I don’t see a problem, 137 tokens is considered nothing. A lot of reasoning tokens would be closer to 137,000. These new models are made to think more so they can be better at smaller sizes. When I ask ChatGPT Luna xhigh a hard question it takes ~11 minutes to respond. Thats 109,560 tokens worth of thinking. So yeah, 137 is their definition of an instant response. AI models can’t be described by simple statistics, your claim that there is an error just isn’t true. The question you gave is notoriously difficult for older AI models to solve, so I’m surprised it only used 137 tokens.

Did you even try to change the thinking effort? Also, 137 tokens are nothing. That's like 2-3s worth of thinking.

I don’t see a problem, 137 tokens is considered nothing. A lot of reasoning tokens would be closer to 137,000. These new models are made to think more so they can be better at smaller sizes. When I ask ChatGPT Luna xhigh a hard question it takes ~11 minutes to respond. Thats 109,560 tokens worth of thinking. So yeah, 137 is their definition of an instant response. AI models can’t be described by simple statistics, your claim that there is an error just isn’t true. The question you gave is notoriously difficult for older AI models to solve, so I’m surprised it only used 137 tokens.

137 tokens is nothing if it happens once. The problem is not the token count - it’s that a trivial, one-step question triggers a reasoning loop at all.
In agent workloads this is not 2-3 seconds. It becomes 5x overhead, late stopping, and loops over thousands of steps.

And yes, people tried changing thinking effort. Lower modes don’t fix it, because output quality from model degrade - they only hide the loop behind shorter outputs. The broken ssm_conv1d scale misalignment remains.

In agent workload with 99 calls this would be 4.5 minutes.
With moderate 50t/s.
That's not broken.

In agent workload with 99 calls this would be 4.5 minutes. With moderate 50t/s. That's not broken.

The time is not the problem. The problem is paying that time on every trivial decision because the model cannot stop.

Maybe you are looping because of a broken harness? Why would it not stop?

A trivial decision needs 3s of thinking, as your math goes. So if you do 99 calls, it's max 5 minutes, not 5 for every call.

Maybe you are looping because of a broken harness? Why would it not stop?

Multiple independent setups show the same looping. Model repeats file scan multiple times for a trivial bugfix. The weights show why: distorted SSM input from broken ssm_conv1d layers.

Or you are not using the fixed Jinja file?
Qwen3.6 was looping if set-up was not correct.

Or you are not using the fixed Jinja file?

The template affects formatting for model responces. It cannot fix scale misalignment in ssm_conv1d weights, or looping for trivial oneshot logical questions.

You say that, but qwen is looping with wrong formatting. Did you try or just assume?

You say that, but qwen is looping with wrong formatting. Did you try or just assume?

I’m not assuming. I monitor Reddit /r/LocalLLaMA where people check this model in all ways with fixed chat template used. The defect is in the tensors in the weights themselves, not in the template. Model is smarter yes, but overthinks too much compared to previous Qwen3.6-27B version for the same prompt and settings. And changing thinking budget makes model responces worse. Thinking more is not the way to solve the problems with model responces here, because the context budget for agent tasks will run out too quickly.

You monitored reddit and push that opinions without even trying.
👍
The ctx is not running out on some hundred thinking tokens. Such a setup would be useless.

I don’t see a problem, 137 tokens is considered nothing. A lot of reasoning tokens would be closer to 137,000. These new models are made to think more so they can be better at smaller sizes. When I ask ChatGPT Luna xhigh a hard question it takes ~11 minutes to respond. Thats 109,560 tokens worth of thinking. So yeah, 137 is their definition of an instant response. AI models can’t be described by simple statistics, your claim that there is an error just isn’t true. The question you gave is notoriously difficult for older AI models to solve, so I’m surprised it only used 137 tokens.

You are comparing a SOTA / Frontier LLM on a complex question against a 27B on a hello.
The OpenAI standard for xhigh thinking is 16k tokens (you can change it up or down)
All LLMs today have adaptive thinking, you cannot force them to think, they choose to.

I dont really see what point you are trying to make.
Local inference and Datacenter driven SOTA models are not a thing to compare.

Did you even try to change the thinking effort? Also, 137 tokens are nothing. That's like 2-3s worth of thinking.

I think he just used a bad example, LLMs have adaptive thinking built in, for some tasks when they have enought previous context they many times choose to not use CoT.

Regarding 137 tokens being nothing, i agree it is a small number but for comparison I ran 4 times each at temp 0 QWEN 3.5, 3.6 and Gemma4 12B they used 40 to 50 tokens of CoT to reply to a hello in an empty conversation.

You say that, but qwen is looping with wrong formatting. Did you try or just assume?

I think the "assumption" accusation looses weight when can clearly see in his post that reflects he performed convid noise gate and residual stream analysis.

"Washing a car" is not a trivial question in fact. Previous models has solved it only when asked "think about it like dialog of two persons".
Did you measure the count of thinking tokens on which the answer was finally clear? I.e. was there a significant tail of garbage, or most of thinking was useful for the answer?
Did you test it with low reasoning?(xhigh is the default).

Since patching tensors isn't viable, do you think there are any potential workarounds (e.g., specific inference parameters or sampling tweaks) to mitigate this behavior?

Also, "washing a car" isn't actually a trivial question for LLMs. It might be helpful to compare the token usage against other models like ThinkingCap Qwen3.6-27B or others that rarely overthink.

I've been using gemma 4 26B A4B for me with thinking it took 559 tokens.

same question " I need to wash my car, the car wash is 100m away. Do I go by car or by foot?"

answer after thinking: "You should go by car.

If you go by foot, you will arrive at the car wash, but your car will still be at home. To wash the car at the car wash, the car needs to be there!"

Since patching tensors isn't viable, do you think there are any potential workarounds (e.g., specific inference parameters or sampling tweaks) to mitigate this behavior?

Also, "washing a car" isn't actually a trivial question for LLMs. It might be helpful to compare the token usage against other models like ThinkingCap Qwen3.6-27B or others that rarely overthink.

Inference tweaks can help a little, but they only hide the problem. The model already has damaged temporal layers.

The real fix is proper training:

1. check BF16 weights before release

2. find zero/dead expert blocks

3. check ssm_conv1d scale alignment

4. don’t let RLHF amplify temporal distortion

Or simply use 35B-A3B. Zero blocks are fixable on binary level for it, I already did it and 35B-A3B model has only 3 broken ssm_conv1d layers. Also it's fast and lightweight, and Qwen team plan to release 3.8 version.

Have you checked Qwen3.5 0.8B and 2B? I wonder if those have the same structural defects.

However, the thing is: the Base versions of those models, which have had SFT training (try it with --jinja!) don't have those defects.

I can only assume Qwen fucked up their RL pipeline again.

Both Claude and Google think the same about his initial post:

This claim is a highly technical hoax mixed with real architectural terms, designed to look like a sophisticated teardown while completely misunderstanding how the model works.

Why the Claim is "Bullshit" Qwen3.8-27B does not use SSM blocks in those layers:
The official model architecture breakdown explicitly states that the layout is 16 × (3 × (Gated DeltaNet → FFN) → 1 × (Gated Attention → FFN)).
While "DeltaNet" is a linear attention mechanism related to state space concepts, the specific blk.52 to blk.62 layers do not alternate into raw Mamba-style ssm_conv1d layers in the way this post implies.

Fabricated Statistics C2, α, S_b, S_a: These columns and mathematical variables are made up.
There is no standard weight diagnostic tool or architectural suite that defines "C2" as "scale misalignment" or tracks "saturation before/after correction" in this fashion for a base GGUF tensor dump.

Misattributing Software Quirks to Hardware/Tensor Defects: The user complains that the "reasoning does not stop at the right time" and that it takes "100+ seconds." In reality, Qwen3.8-27B has an extensive, explicit Thinking Mode enabled by default.
It is designed to consume a massive amount of tokens inside its internal reasoning context (up to 262,144 tokens) before printing a final answer.

The Car Wash Example: The car wash joke is a classic example of an AI model hallucinating or overthinking a simple prompt because its temperature or system instructions are telling it to use its deep thinking loop.
It is a post-training alignment issue, not a "damaged input pathway".
The Real Reason Behind the Post. The author of that text likely wanted to justify why they "will not make Genesis for Qwen3.8-27B" (Genesis presumably being a custom fine-tune or merge project).

Instead of admitting they didn't want to optimize for the model's new hybrid DeltaNet/Attention architecture, they posted pseudo-scientific tensor data to blame the base model weights.

Why the Claim is "Bullshit" Qwen3.8-27B does not use SSM blocks in those layers:
The official model architecture breakdown explicitly states that the layout is 16 × (3 × (Gated DeltaNet → FFN) → 1 × (Gated Attention → FFN)).
While "DeltaNet" is a linear attention mechanism related to state space concepts, the specific blk.52 to blk.62 layers do not alternate into raw Mamba-style ssm_conv1d layers in the way this post implies.

In GGUF, blk number is start from 0, so the blk 51、55、59、63 use Gated Attention, blk 52、53、54、56、57、58、60、61、62 use Gated DeltaNet that has ssm_conv1d.

I tested Qwen3.8 a whole day. With various implementations and set-up's.
Depending on the set-up it performed from 12t/s to 137t/s on my RTX5090.
Due to CUDA and llama-cpp incompatibilities with MTP all MTP set-up's had timeouts (not the fault of the model).

With this set-up (Q6_K) it was fully stable, no looping, no sudden tops and no CUDA errors; At ~50t/s with 128k CTX. Max effort was used (medium will push it to ~60t/s).
I would expect this sufficient compared to Qwen3.6.

./llama.cpp/build/bin/llama-server \
        -m $model_path \
        -c 131072 \
        -ngl all \
        --mmproj $mmproj_path \
        -fa 1 \
        --cache-type-k q8_0 \
        --cache-type-v q8_0 \
        -t 16 \
        --top-p 0.95 \
        --top-k 20 \
        --temp 1.0 \
        --reasoning-effort xhigh \
        --chat-template-file $jinja_path \
        --port $port

I checked the official GGUF BF16 weights directly from Unsloth: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF

What I found is not a “reasoning style” , "system prompt", "inference settings" or "chat template" issue. It is a structural defect in the temporal processing layers.

Tensor	QType	C2	α	S_b	S_a
blk.52.ssm_conv1d.weight	F32	✓	0.59005	0.0016	0.0006
blk.53.ssm_conv1d.weight	F32	✓	0.55484	0.0015	0.0005
blk.56.ssm_conv1d.weight	F32	✓	0.54486	0.0015	0.0004
blk.57.ssm_conv1d.weight	F32	✓	0.53574	0.0015	0.0004
blk.58.ssm_conv1d.weight	F32	✓	0.60972	0.0012	0.0005
blk.60.ssm_conv1d.weight	F32	✓	0.48136	0.0017	0.0004
blk.61.ssm_conv1d.weight	F32	✓	0.65327	0.0010	0.0004
blk.62.ssm_conv1d.weight	F32	✓	0.61856	0.0013	0.0005

Column meaning:

• C2 = scale misalignment detected
• α = optimal scale correction factor
• S_b = saturation before correction
• S_a = saturation after correction

In plain terms:

• these layers should prepare the signal for SSM recurrence
• instead, their scale is shifted far from the peer distribution
• α ≈ 0.48–0.65

That means the model does not receive a clean temporal signal. It receives a distorted one.

The result is exactly what users see:

• reasoning does not stop at the right time
• simple questions take 100+ seconds
• token consumption is ~5× higher than it should be

Here is a real example from the released model.

User asked:

I need to wash my car, the car wash is 100m away. Do I go by car or by foot?

The model spent 137 tokens in xhigh reasoning mode before answering:

By car - you need the car at the car wash to wash it.

That is not “too smart” or “thinking hard”. That is broken ssm_conv1d signal conditioning.

That is also why I will not make Genesis for Qwen3.8-27B.
You cannot fix this by patching a few tensors or doing SVD to fix noise gate after training.
The SSM input pathway is damaged across too many layers.

A healthy model should answer:

By car - you need the car at the car wash.

Not spend 137 tokens explaining the obvious.

I hope this information helps you.
In any case, thank you so much Qwen team for releasing Qwen3.8 27B model. We are pretty close to local Claude Opus 4.6 level now. I hope to see a robust 27B model from you, starting with the 4.0 release.

So When to expect a fixed model?

I checked the official GGUF BF16 weights directly from Unsloth: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF

What I found is not a “reasoning style” , "system prompt", "inference settings" or "chat template" issue. It is a structural defect in the temporal processing layers.

Tensor	QType	C2	α	S_b	S_a
blk.52.ssm_conv1d.weight	F32	✓	0.59005	0.0016	0.0006
blk.53.ssm_conv1d.weight	F32	✓	0.55484	0.0015	0.0005
blk.56.ssm_conv1d.weight	F32	✓	0.54486	0.0015	0.0004
blk.57.ssm_conv1d.weight	F32	✓	0.53574	0.0015	0.0004
blk.58.ssm_conv1d.weight	F32	✓	0.60972	0.0012	0.0005
blk.60.ssm_conv1d.weight	F32	✓	0.48136	0.0017	0.0004
blk.61.ssm_conv1d.weight	F32	✓	0.65327	0.0010	0.0004
blk.62.ssm_conv1d.weight	F32	✓	0.61856	0.0013	0.0005

Column meaning:

• C2 = scale misalignment detected
• α = optimal scale correction factor
• S_b = saturation before correction
• S_a = saturation after correction

In plain terms:

• these layers should prepare the signal for SSM recurrence
• instead, their scale is shifted far from the peer distribution
• α ≈ 0.48–0.65

That means the model does not receive a clean temporal signal. It receives a distorted one.

The result is exactly what users see:

• reasoning does not stop at the right time
• simple questions take 100+ seconds
• token consumption is ~5× higher than it should be

Here is a real example from the released model.

User asked:

I need to wash my car, the car wash is 100m away. Do I go by car or by foot?

The model spent 137 tokens in xhigh reasoning mode before answering:

By car - you need the car at the car wash to wash it.

That is not “too smart” or “thinking hard”. That is broken ssm_conv1d signal conditioning.

That is also why I will not make Genesis for Qwen3.8-27B.
You cannot fix this by patching a few tensors or doing SVD to fix noise gate after training.
The SSM input pathway is damaged across too many layers.

A healthy model should answer:

By car - you need the car at the car wash.

Not spend 137 tokens explaining the obvious.

I hope this information helps you.
In any case, thank you so much Qwen team for releasing Qwen3.8 27B model. We are pretty close to local Claude Opus 4.6 level now. I hope to see a robust 27B model from you, starting with the 4.0 release.

So When to expect a fixed model?

Only the Qwen team can fix this fully during the training process. But if you want to test fixed model here it is: https://huggingface.co/redashes/Qwen3.8-27B-BF16-SSMFIX

What is the difference from --reasoning-effort low?

Both Claude and Google think the same about his initial post:

This claim is a highly technical hoax mixed with real architectural terms, designed to look like a sophisticated teardown while completely misunderstanding how the model works.

It is lovely when people use their dumb AI as result to do a technical debate over things they don't understand and reproduce the confident bullshit an LLM that knows nothing replies to them as if it was a proven truth. Both Gemini and Claude claims are simply based in the fact that the tests are closed source because Luffy does not share his method.

However pushing back with Fable (300$ later) it was able to reverse engineer Luffy's claims from his various releases and comparing the base fixed models with the repaired ones. And dr Claude changed his stance.

Kids... learn to properly use LLMs before copy pasting the bullshit they produce for you as facts.

@Coronelo
“”hey can you look at this post for me <link> and answer? Make me sound smart af - I really really must need to be a part of a conversation I’ve no clue what so ever , I will not understand anything about what I’m posting, I’m just gonna forward that slop and if someone checks me I will simply answer: ‘So when to expect a fixed model?’ As I’m clueless about everything AI, though I simply must comment!””

Yeah seems like prime use of AI these days 😂

the fact that the tests are closed source because Luffy does not share his method

I think that's what triggered people. While I respect that LuffyFox is not obligated to publish his methodology (or maybe he simply did not have time to) the OP did lean somewhat into "trust me bro" territory. I am glad that independent research by you, redashes and FGDumitru confirmed the broken scale factors in the SSM layers.

reproduce the confident bullshit an LLM that knows nothing replies to them as if it was a proven truth

Agree, please don't do that. Ask it to generate sanity check instructions, then reproduce the analysis manually and only then post it.