A crazy thinking model
This is a preview model i think, it always thinking and thinking, can it stop?
This is a preview model i think, it always thinking and thinking, can it stop?
"--reasoning-budget N",
N equals to thinking tokens
This is a preview model i think, it always thinking and thinking, can it stop?
there are three thinking levels for a reason. Try to change them
The default shouldn't be an unusable amount of thinking.
This was clearly chosen for test maxing because without it Qwen 3.8 isn't broadly better than Qwen 3.6. If anything, it's broadly worse. Alibaba, like the rest of the industry, is masking the fact that they're hit a ceiling. They've been trading broad abilities for marginal gains in select domains, especially coding and agentic work, for more than a year. And now they're adding an unusable amount of latency (>10x longer) to squeeze out a tiny bit more performance in only a handful of domains.
The default shouldn't be an unusable amount of thinking.
This was clearly chosen for test maxing because without it Qwen 3.8 isn't broadly better than Qwen 3.6. If anything, it's broadly worse. Alibaba, like the rest of the industry, is masking the fact that they're hit a ceiling. They've been trading broad abilities for marginal gains in select domains, especially coding and agentic work, for more than a year. And now they're adding an unusable amount of latency (>10x longer) to squeeze out a tiny bit more performance in only a handful of domains.
idk man. I am only using the xhigh level now and it's been pretty good so far. Not using gguf quants though, but exl3
@vonamakitsune That's good to hear. A lot of people are saying Qwen 3.8 is doing better at long coding tasks. But I only test models without thinking, and most are regressing. Even with thinking on Arena across the various domains model families are regressing. Such as Opus 5 doing worse than Opus 4.7, which is doing worse than 4.6. Or Grok 4.6 doing worse than 4.5, which is doing worse than 4.2. And OpenAI's GPT series is stagnated (e.g. GPT 4.5/4.6). And Qwen 3.6 is broadly worse than Qwen 3.5.
The reason why is clear. The underlying LLMs are the same (e.g. same information density), as is the training. The primary change is grossly overtraining for agentic and long context coding work, so once they're done the weights have been scrambled more, making the models generally inferior (e.g. more factual hallucinations and lower quality creative writing with more contradictions).
In short, the industry seems to be giving up making AI models generally stronger, and are instead trying to maximize performance in a handful of select domains, which includes throwing a lot more tokens at them.
This is a preview model i think, it always thinking and thinking, can it stop?
there are three thinking levels for a reason. Try to change them
I have set to low, but nothing happend, still thinking without stop.
Load with chat-template-kwargs "{\"preserve-thinking\": true,\"reasoning_effort\": \"medium\"}" is better.
Load with chat-template-kwargs "{\"preserve-thinking\": true,\"reasoning_effort\": \"medium\"}" is better.
sadly even on medium it hits the wall exactly in a same way. Even exceeding the max output thinking length... its crazy annoying. Hope for some fine tunes of the model. the base on is not impressive when it comes to efficiency sofar
...still thinking
The default shouldn't be an unusable amount of thinking.
This was clearly chosen for test maxing because without it Qwen 3.8 isn't broadly better than Qwen 3.6. If anything, it's broadly worse. Alibaba, like the rest of the industry, is masking the fact that they're hit a ceiling. They've been trading broad abilities for marginal gains in select domains, especially coding and agentic work, for more than a year. And now they're adding an unusable amount of latency (>10x longer) to squeeze out a tiny bit more performance in only a handful of domains.
I beg to differ. It has been a huge improvement for me compared to Qwen3.6.
@hermanbrood Many people are sharing your opinion that Qwen 3.8 is a huge improvement over Qwen 3.6 on large coding projects. But how is it at everything else?
Qwen 3.5 was generally stronger than Qwen 3.6 (more broad knowledge, better stories, fewer boneheaded mistakes, and so on). This is almost certainly why Qwen 3.6's score was never added to the Arena. And my guess is Qwen 3.8 is performing generally much worse than Qwen 3.6, and its score will also never be added to Arena.
It seem Alibaba is pushing coding and agentic really hard, and at the expense of broad knowledge and abilities. Plus if it takes ~10x longer to respond with thinking than without it's less than useless in real-time scenarios, such as customer service. This model really appears to be a tool (agentic coding) rather than a general purpose AI model.
This model really appears to be a tool (agentic coding) rather than a general purpose AI model.
Yeah, exactly. As i always understood, it was never intended to be a general-purpose model. And my guess - if it gained so much in agentic/coding benchmarks compared to the 3.6 version, it certainly got worse in other fields. But tbh that was the expectation for many people (probably developers). We want something that can reason strongly, knows how to work with info and understand all the right patterns rather than having up-to-date knowledge or being creative.
Out of curiosity, have you tried gemma 4 31b or any fine-tune of it? I found it extremely better in terms of creative tasks and all of that
@vonamakitsune Thanks for the recommendation. Yes, I've tried Gemma 4 and it's notably better at most things, including creative writing. For example, Qwen's stories are more repetitive, include more boneheaded mistakes, like contradictions to itself and the user's prompt, and so on.
But since Qwen 3.8 max is balanced and broadly capable you'd think Alibaba would make a balanced general purpose Qwen 3.8 27b and a separate Qwen 3.8 27b Coder for maximum coding/agentic performance.
Alibaba focusing on coding/agentic tasks wouldn't be so bad if most other companies weren't doing the same, including Nvidia and Meta, likely because >90% of the users of small OS AI models are coders so the entire ecosystem is moving away from small general purpose AI models in favor of coding/agentic tools.
if you actually have enough KV to give it a full 256k context window the thinking pays off. It eventually does stop and starts to execute, and because it thought so much it does an excellent job.
you'd think Alibaba would make a balanced general purpose Qwen 3.8 27b and a separate Qwen 3.8 27b Coder for maximum coding/agentic performance.
You know, you got me thinking about making just a very strong base and adding LORA heads to it. That'd be really cool. Your base model is always in VRAM. And then you just switching heads depending on a task. Fast switches, strong capabilities. And i know it was already kinda tried by the HG community, but i never heard about it in terms of a big corporate environment, if you know what i mean. Like actually proved and tested performance, not someone's pet project.
Try to disable preserve_thinking. Seems KV cache wall is more distant in this way.
Before this model I used ThinkingCap-Qwen3.6-27B extensively for programming and agentic flows, and I was really impressed with that finetune. It felt really snappy and responsive and didn't need to think much.
I was very hyped for Qwen3.8-27B, but wasn't prepared for how much slower it would be in practice. Even with effort set to low, it thinks A LOT. I gave it a very simple task of fixing a bug in my project. It managed to identify the problem within a minute or two, but then it went off on a tangent where it read lots of unrelated files and wasted more than half an hour doing something completely unrelated to the task at hand before I had to stop it. This is an issue I've never experienced with Qwen3.6 27B. The latter never derails like that. It gets straight to work and it doesn't waste my time.
I feel like a lot of the positive news about this model is overblown. It does appear to be better at invoking skills in Claude Code automatically though, which Qwen3.6 27B wasn't good at, but aside from that it's just too slow to be useable on my RTX 5090.
@29sushi since you have 32gb of vram i strongly suggest you to try an exl3 quant (you can use tabbyapi for inference, but pay attention!! it is GPU only, no ram offloading). It is stronger than ggufs for the same size and you will probably not have such a problem with 30 minutes thinking. 4 or 5 bits would be enough to feel the power .
If you try it, let me know how it goes. I use exl3 since the release of qwen 3.6 27b and never been happier
In case you need exact params to run it, here is what i use right now (modify it for you gpu):
#!/bin/bash
/home/<YOUR_USER>/tabbyAPI/start.sh \
--model-name Qwen3.8-27B-exl3-5.0bpw \
--max-seq-len 131072 \
--cache-size 262144 \
--cache-mode "6,6" \
--override-preset qwen38 \
--vision true \
--tool-format qwen3_coder \
--max-batch-size 2 \
--tensor-parallel true \
--draft-mode mtp
Also pay attention to the override-preset parameter. It is just a file where you specify sample params like temp, top-p and so on. I put numbers from this official repo