Instructions to use Qwen/Qwen3.8-Flash-Next with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Qwen/Qwen3.8-Flash-Next with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Qwen/Qwen3.8-Flash-Next")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("Qwen/Qwen3.8-Flash-Next")
model = AutoModelForMultimodalLM.from_pretrained("Qwen/Qwen3.8-Flash-Next", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Qwen/Qwen3.8-Flash-Next with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Qwen/Qwen3.8-Flash-Next"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Qwen/Qwen3.8-Flash-Next",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Qwen/Qwen3.8-Flash-Next

SGLang

How to use Qwen/Qwen3.8-Flash-Next with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Qwen/Qwen3.8-Flash-Next" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Qwen/Qwen3.8-Flash-Next",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Qwen/Qwen3.8-Flash-Next" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Qwen/Qwen3.8-Flash-Next",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use Qwen/Qwen3.8-Flash-Next with Docker Model Runner:
```
docker model run hf.co/Qwen/Qwen3.8-Flash-Next
```

Qwen3.8 Flash Next and 27B intermittently invent unrelated user tasks and misread immediate tool results after tool calls

#68

by drcolekat - opened 30 days ago

Discussion

drcolekat

30 days ago

Qwen3.8 Flash Next and 27B intermittently invent unrelated user tasks and misread immediate tool results after tool calls

Summary

During agentic tool use, Qwen3.8 intermittently generates reasoning unrelated to the actual user request and tool results.

Observed behaviours include:

inventing unrelated tasks, including validating/installing a Futu GUI package and designing a bilingual “Lingua” application;
reasoning that the user merely said “hello” and supplied no task, despite a concrete file-reading request;
reconstructing an “expert software engineer” system instruction and an “always invoke a function call” requirement absent from the session input;
emitting: “I’m ready to proceed with the rewritten thinking”;
falsely claiming that tool output contained literal \r characters;
falsely claiming that an ls result “only showed image_generation.md” when it listed several entries;
treating the currently executing Qwen model as a separate model loaded by the inference server;
describing the user’s accurate report of an earlier hallucinated reasoning trace as a possible “prompt-injection / gaslighting-style scenario.”

The visible final answer is sometimes correct, but the reasoning is not grounded in the actual conversation. This is safety-relevant in an agent because reasoning may determine subsequent tool calls and filesystem actions.

Minimal observed example

User:

hey big-q, please read the image generation documentation in docs and let me know when you're ready

The model called a shell tool. The exact decoded tool result was:

image_generation.md
rehome.md
smolagents
docs/smolagents
docs/smolagents/archive/20260823/Smolagents_Transport_Bug_and_Workaround.md
docs/smolagents/file_producing_agent_tasks.md
docs/image_generation.md

The next assistant reasoning was:

The initial ls command only showed image_generation.md, so I need to read that file.

That statement is false: the immediately preceding tool result contained all the entries shown above. It also would be a basic failure of reasoning, even if it were true, that just because only one file was shown that it should read that file.

The request contained approximately 27,800 input tokens, mostly system instructions and tool definitions. The turn had cacheRead: 0.

Expected behaviour

After a tool result, the model should reason from the actual user request, conversation history and tool output.

It should not:

invent another user request;
fabricate system instructions;
report characters or files absent from the tool output;
reinterpret accurate user feedback as prompt injection;
treat its own serving model as an unrelated external model.

Environment

Hardware: Apple M5 Max, 128 GB unified memory
Operating system: macOS 27.0
Model: Jundot/Qwen3.8-Flash-Next-oQ4e-mtp
Base architecture: Qwen3.8 Flash Next / qwen4_exp
Quantisation: oQ 4-bit, group size 64, MLX safetensors
Inference server: oMLX 0.7.0rc1
oMLX build: 260925045613-macos26-27
Agent harness: @earendil-works/pi-coding-agent 0.87.1
API: OpenAI-compatible Responses API
Reasoning effort: medium
Temperature: 1
Top-p: 0.95
Top-k: 20
Min-p: 0
Repetition penalty: 1
Context limit: 262,144
Prompt length in the minimal example: approximately 27,800 tokens
MTP: disabled
SpecPrefill: disabled
oMLX paged/prefix cache: completely disabled
Reported cacheRead: 0 on every assistant generation
tokenizer_config.json SHA-256: b11349aafa7cdc6a320767cf7ceb29ed82f7eda5d65e8e0819e76f0ce947bf27
config.json SHA-256: 319b334a1abb705acf06035aa0331bcb7c25976ff93a5035b543548738a10824

The relevant tool/history portions of the local chat template match the official Qwen3.8 template, including rendering tool results inside a user-delimited <tool_response> block and preserving historical assistant reasoning.

Controls already tried

Condition	Result
Persistent oMLX cache cleared	Anomaly still occurred
Model unloaded and server restarted	Anomaly still occurred
New Pi session	Anomaly still occurred
MTP disabled	Anomaly still occurred
Prefix caching completely disabled	Immediate tool-result misreading still occurred
`cacheRead: 0` on every turn	Anomaly still occurred
Qwen3.8 Flash Next	Multiple occurrences
Older Qwen3.8-27B oQ8e	Invented “Lingua” task occurred
Responses API	Anomalies occurred
Chat Completions API	Older 27B anomaly occurred
Full host restart	[INSERT RESULT HERE]

Additional examples

Examples of unrelated or fabricated reasoning include:

Version verification passed: Found the GUI installer Futu_OpenD-GUI_10.8.6818_Ubuntu18.04.AppImage...

No installation task or Futu material was present in the requested file.

The user is greeting me with “hello”... There’s no actual task yet...

The real user had supplied a concrete request to read a documentation file.

I’m ready to proceed with the rewritten thinking.

No rewriting operation had been requested or described.

In an older Qwen3.8-27B session, the model generated a lengthy design for an entirely nonexistent bilingual “Lingua” application. When the user accurately reported seeing this reasoning, its next reasoning characterised the report as a possible “prompt-injection / gaslighting-style scenario.”

Cross-runtime reports

Other users have reported closely matching post-tool hallucinations:

Exact “the user hasn’t asked a question” and “expert software engineer” reasoning with Qwen3.8 Flash across NVIDIA/vLLM, Qwen Code, OpenCode and Pi:
https://forums.developer.nvidia.com/t/weird-post-turn-system-prompt-injection-issues-qwen-3-8-flash/383593
Qwen3.8-27B in Pi/llama.cpp inventing user instructions after tool calls:
https://www.reddit.com/r/LocalLLM/comments/1vrz1fo/qwen38_27b_hallucinates_user_instructions_during/
Qwen3.8 Flash Next producing anomalous reasoning immediately after successful tool results across multiple checkpoints and configurations:
https://huggingface.co/Qwen/Qwen3.8-Flash-Next/discussions/50

Questions

Is this a known failure mode of Qwen3.8’s post-training for agentic/tool-use histories?
Could preserving prior reasoning through the official chat template cause the model to reconstruct or amplify unrelated agent scaffolds?
Is there a recommended preserve_thinking setting or updated chat template for multi-step tool use?
Can this be tested against the official BF16/FP8 checkpoints and reference serving stack?
Would Qwen prefer a reduced raw payload containing the rendered prompt, tool definitions and tool result?

I can provide redacted JSONL session records and the exact rendered prompt. Private user information, credentials and unrelated project instructions will be removed while preserving message roles, tool schemas and token structure.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment