Repeated-token runaway generation in Qwen3.8-Flash-Next-NVFP4 with DeepSeek Harness sub-agents

Summary

I am running RadixArk/Qwen3.8-Flash-Next-NVFP4 with SGLang in a two-node tensor-parallel setup across two NVIDIA DGX Spark / GB10 systems. When used as a sub-agent backend by DeepSeek Harness, the model can enter an unrecoverable reasoning loop and emit the same character repeatedly, most notably !, for thousands of tokens.

The issue is reproducible with multiple independent sub-agents in the same Harness session. It is not limited to one prompt or one sub-agent. The coordinator may continue normally while the sub-agents become stuck.

Observed symptom

The affected sub-agents were processing PDF/music-sheet verification tasks. After several normal tool calls, the model entered a repetition loop inside the reasoning stream:

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
...

In one earlier run, a corrupted output also contained a very long sequence such as:

G!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

The repeated character is not a valid final answer and the stream does not terminate by itself. The request must be aborted externally.

Reproduction context

• Model: RadixArk/Qwen3.8-Flash-Next-NVFP4
• Architecture reported by SGLang: Qwen4ExpForConditionalGeneration
• Quantization: ModelOpt NVFP4
• Runtime: SGLang, two-node TP=2
• Context length: 262144
• SGLang port: 30010
• SGLang image: lmsysorg/sglang:qwen38flashnext
• --quantization modelopt_fp4
• --fp4-gemm-backend flashinfer_cutlass
• --attention-backend triton
• --tp-size 2
• --nnodes 2
• --max-running-requests 4
• --context-length 262144
• --speculative-algorithm NEXTN
• --speculative-num-steps 3
• --speculative-eagle-topk 1
• --speculative-num-draft-tokens 2
• --reasoning-parser auto
• --tool-call-parser auto

SGLang resolved the parsers as follows:

reasoning_parser: qwen3
tool_call_parser: qwen3_coder

The Harness uses the OpenAI-compatible /v1/chat/completions endpoint and runs multiple tool-using sub-agents over the same model endpoint.

Harness workload

The parent agent launched three sub-agents to inspect and repair hymn data against PDF sheet music. Each sub-agent:

1. Read a shared technical brief.
2. Inspected songs.json.
3. Ran PDF extraction and alignment scripts.
4. Rendered or inspected PDF evidence when the text layer was corrupted.
5. Prepared a JSON verdict.

The PDF task was intentionally difficult because some PDFs contain broken glyph/text-layer extraction. However, the model initially handled multiple tool calls correctly before entering the repetition loop.

Log evidence

The affected session was exported to:

/home/dangnsh/Downloads/dsh-session-session-4590be1f-6982-457c-bd36-19b5d814dec8

It contains one coordinator session and three sub-agent session.jsonl files.

The three affected sub-agents were:

• df3f992c-fcd8-4e1c-be73-531e3f118e14
• 805698dc-c310-47ff-9b9a-5c880553ac0f
• af6dfd83-7846-4da5-8faf-d509ef9ea19f

All three had the following request configuration in their request headers:

provider: qwen-flash-next
model: Qwen3.8-Flash-Next-NVFP4
maxTokens: 65536
reasoningEffort: xhigh

The first clear repetition events occurred after normal tool use and PDF analysis. The sub-agent logs contain thousands of consecutive ! characters in the reasoning stream. The sessions were eventually aborted by the user.

The coordinator did not show the same character-level repetition. Its final activity was mainly checking sub-agent status.

Runtime impact

While the loop was active:

• Multiple requests remained active on the SGLang scheduler.
• GPU utilization stayed high.
• The scheduler continued decoding instead of terminating the response.
• The model could consume a large part of the configured output budget.
• In one observation, three requests were running concurrently.
• The SGLang server itself remained healthy; this was not a process crash or HTTP failure.

The SGLang metrics showed the server was still alive, but the active request could continue decoding indefinitely from the application point of view.

Mitigations already attempted

1. External request abort

I used SGLang's request-abort endpoint without restarting the model:

curl -X POST http://127.0.0.1:30010/abort_request \
  -H 'Content-Type: application/json' \
  -d '{"rid":"","abort_all":true}'

This returned HTTP 200 and reduced the active request count to zero. The model remained loaded in GPU memory, so recovery did not require reloading the checkpoint.

2. Sampling penalty adjustment

I added the following to the model's generation_config.json on both nodes:

"repetition_penalty": 1.05,
"presence_penalty": 0.1

SGLang confirmed that it loaded repetition_penalty: 1.05 along with the model defaults:

Using default chat sampling params from model generation config:
{'repetition_penalty': 1.05, 'temperature': 1.0, 'top_k': 20, 'top_p': 0.95}

The loop still occurred in a subsequent Harness session. The presence_penalty value was not shown in the SGLang startup log, so it is unclear whether it is accepted as a model default from generation_config.json or must be supplied per request.

3. Explicit parser verification

SGLang's automatic parser selection was verified to resolve to:

reasoning_parser: qwen3
tool_call_parser: qwen3_coder

The launch configuration was already using the correct effective parsers. Making the parser flags explicit is planned as a cleanup, but parser auto-detection did not appear to be the direct cause of the repetition.

4. Lower reasoning level for sub-agents

The affected run used reasoningEffort: xhigh and maxTokens: 65536 for all sub-agents. I changed the DeepSeek Harness configuration so that sub-agents use a dedicated provider route with reasoning: medium, rather than inheriting the coordinator's xhigh setting.

The new route is:

qwen-flash-next-medium:
  baseURL: http://127.0.0.1:30010/v1
  reasoning: medium

Both subagent and subagent_deep are now configured to use this route. This change has not yet been validated by a clean end-to-end reproduction test, so it should be considered a mitigation under test, not a confirmed fix.

5. Keeping the model alive after abort

Instead of killing SGLang and reloading approximately 50 GiB per node, I abort the active requests through /abort_request. This successfully stops the current task but does not prevent a future request from entering the same loop.

What has not fixed the problem

The following have not eliminated the issue so far:

• Running the model with the correct Qwen reasoning parser.
• Running the model with the correct Qwen coder tool parser.
• Adding a mild repetition_penalty of 1.05.
• Adding a mild presence_penalty of 0.1.
• Aborting and restarting Harness tasks while keeping the model loaded.
• Limiting the server to four concurrent requests.
• Running the affected work as separate sub-agents rather than as one large coordinator response.

Current hypotheses

The issue may involve one or more of the following:

1. A model/runtime bug in the NVFP4 checkpoint or its sampling path.
2. An interaction between Qwen's reasoning format and the OpenAI-compatible streaming/tool-call path.
3. An interaction between NEXTN speculative decoding and the reasoning stream.
4. A failure mode triggered by corrupted PDF text/glyph content, although the identical repetition behavior across three independent sub-agents suggests a broader runtime/model issue.
5. Harness sub-agent requests inheriting an excessive reasoning budget (xhigh, 65,536 output tokens).
6. A sampling parameter compatibility issue where presence_penalty or repetition controls are not applied as expected to the default request path.

Requested guidance

Could the model/runtime maintainers advise on the following?

1. Is this a known issue with RadixArk/Qwen3.8-Flash-Next-NVFP4, SGLang, NEXTN/EAGLE speculative decoding, or the Qwen3/Qwen4 reasoning parser?
2. Are repetition_penalty and presence_penalty supported for this checkpoint and this SGLang backend? Should they be supplied per /v1/chat/completions request rather than through generation_config.json?
3. Is there a recommended sampling_defaults or generation_config.json for this NVFP4 checkpoint?
4. Is --speculative-use-rejection-sampling recommended for this model when using NEXTN?
5. Would maintainers recommend testing with speculative decoding disabled to isolate the issue?
6. Are there known restrictions on long reasoning/tool-call sessions with --reasoning-parser qwen3 and --tool-call-parser qwen3_coder?