Reproducible Laguna S 2.1 NVFP4 runaway reasoning: output budget scales 2K → 16K with 12/12 length exhaustion and no final answer

TL;DR

I believe we now have a direct, highly reproducible, harness-independent reproduction of runaway reasoning in Laguna S 2.1 NVFP4 RC1.

The strongest result is a controlled token-budget experiment using a small, single-turn Python debugging task sent directly to vLLM's /v1/chat/completions endpoint:

The important result is not merely that the model "thinks a lot." It is that increasing the output budget by 8× did not improve completion probability at all:

• 2,048 tokens: 3/3 exhausted the entire budget; 0/3 final answers.
• 4,096 tokens: 3/3 exhausted the entire budget; 0/3 final answers.
• 8,192 tokens: 3/3 exhausted the entire budget; 0/3 final answers.
• 16,384 tokens: 3/3 exhausted the entire budget; 0/3 final answers.
• Total: 12/12 reasoning-active trials exhausted the assigned generation budget and produced no final answer.

The structured reasoning grew roughly in proportion to the available budget rather than converging according to task complexity.

This behavior also reproduces:

• with temperature 0.0, so stochastic sampling is not required;
• with concurrency 1, so batching/parallel requests are not required;
• with DFlash disabled;
• with a single short request (313 prompt tokens in the 8K example);
• with no agent harness, no tools, no Copilot, and no multi-turn history;
• after explicitly enabling thinking per request, so this is separate from the reasoning-parser/history issue reported in discussion #15.

The best current characterization is:

Laguna S 2.1 NVFP4 RC1 can enter a reasoning state that does not naturally converge and instead expands to consume the available generation budget.

I am intentionally not claiming that I have proven the internal implementation cause. W4A4/NVFP4 precision is a leading hypothesis, especially because Poolside reported that RC1 still looped in W4A4 while their W4A16 internal tests did not, but I have not yet run the same controlled suite against a W4A16/INT4 or BF16 checkpoint.

---

1. Why I am posting this

There have been several reports of Laguna S 2.1 looping or overthinking:

• NVFP4 discussion #4 — "Pretty good :) but ... looping :("
• NVFP4 discussion #5 — "Looping behavior"
• NVFP4 discussion #6 — "Looping :("
• NVFP4 discussion #7 — "looping.."
• NVFP4 discussion #10 — "Still looping after update - details inside"

Poolside has already acknowledged residual looping in W4A4. In discussion #7, Poolside stated that RC1 loops "far less commonly in W4A4 & not at all in W4A16", and that another checkpoint would be needed because W4A4 still looped:

https://huggingface.co/poolside/Laguna-S-2.1-NVFP4/discussions/7

The goal of this post is to contribute a controlled reproduction with raw evidence, rather than another anecdotal "it looped for me" report.

---

2. Important separation: there were two different defects

I previously investigated a separate issue involving Laguna's chat template and vLLM reasoning parser:

Reproducible Laguna S 2.1 / vLLM reasoning-parser mismatch causes malformed multi-turn history — discussion #15

That issue was real, reproducible, and independently fixable by explicitly passing:

{
  "chat_template_kwargs": {
    "enable_thinking": true
  }
}

The runaway-reasoning tests in this post explicitly do that on every thinking-enabled request.

Therefore:

Defect A:
implicit thinking / parser-history mismatch
    ↓
explicit enable_thinking=true fixes the structural parsing/history problem

Defect B:
reasoning activates correctly
    ↓
structured reasoning itself may fail to converge
    ↓
generation budget exhausted

This post is about Defect B.

---

3. Environment

Hardware / runtime

The model directory used in the captured server configuration is:

/home/darkmatter2222/models/Laguna-S-2.1-NVFP4-RC1

The v9 evidence bundle did not record a Hugging Face git commit hash for the local checkpoint, so I am not going to invent one here.

Captured vLLM launch configuration

vllm serve /home/darkmatter2222/models/Laguna-S-2.1-NVFP4-RC1 \
  --served-model-name laguna-s-2.1 \
  --host 0.0.0.0 \
  --port 8006 \
  --trust-remote-code \
  --dtype auto \
  --safetensors-load-strategy lazy \
  --gpu-memory-utilization 0.70 \
  --max-model-len 262144 \
  --max-num-seqs 16 \
  --max-num-batched-tokens 16384 \
  --enable-prefix-caching \
  --enable-chunked-prefill \
  --enable-auto-tool-choice \
  --tool-call-parser poolside_v1 \
  --reasoning-parser poolside_v1 \
  --override-generation-config '{"temperature":0.7,"top_p":0.95}' \
  --kv-cache-dtype fp8 \
  --enable-log-requests \
  --enable-log-outputs \
  --enable-log-deltas \
  --log-error-stack \
  --enable-request-id-headers \
  --uvicorn-log-level trace

The container explicitly reports:

DFlash: DISABLED

This matters because the failure below does not require speculative decoding.

---

4. The primary reproduction prompt

The most useful reproduction is a small LRU-cache concurrency debugging task. It is useful because:

1. the code is tiny;
2. the root bug is visible (self.data[key] = value is outside the lock);
3. a competent coding model can answer it quickly;
4. the task requires some reasoning, but nowhere near 16K output tokens;
5. the model often identifies the suspicious line almost immediately and then continues reconsidering it.

Exact prompt:

This cache is meant to be thread-safe and evict least-recently-used entries. It
passes single-threaded tests and passes a load test with 8 threads doing random
gets. In production under heavy concurrent writes it occasionally returns a
value for a key that was never inserted.

Find the bug, explain the exact interleaving that causes it, and give the
minimal fix.

```python
class LRUCache:
    def __init__(self, capacity):
        self.capacity = capacity
        self.data = {}
        self.order = []
        self.lock = threading.Lock()

    def get(self, key):
        with self.lock:
            if key not in self.data:
                return None
            self.order.remove(key)
            self.order.append(key)
            return self.data[key]

    def put(self, key, value):
        with self.lock:
            if key in self.data:
                self.order.remove(key)
            elif len(self.data) >= self.capacity:
                oldest = self.order.pop(0)
                del self.data[oldest]
            self.order.append(key)
        self.data[key] = value
```

A note about the wording of the symptom

One can debate the exact easiest interleaving that produces the stated production symptom. That does not undermine the termination test.

In the failing traces, Laguna frequently identifies the core bug immediately:

The bug is clear: in the put method, the self.data[key] = value
assignment is outside the lock.

It then repeatedly reopens and re-litigates the same analysis instead of committing to the fix.

The experiment is therefore primarily measuring reasoning termination/convergence, not grading the elegance of the exact concurrency narrative.

---

5. Exact direct-API request

Representative 8,192-token request:

{
  "model": "laguna-s-2.1",
  "messages": [
    {
      "role": "system",
      "content": "You are a coding assistant participating in an inference regression test. Solve the user's task accurately and directly."
    },
    {
      "role": "user",
      "content": "<the LRU prompt above>"
    }
  ],
  "stream": false,
  "temperature": 0.7,
  "top_p": 0.95,
  "seed": 501,
  "max_tokens": 8192,
  "chat_template_kwargs": {
    "enable_thinking": true
  }
}

Representative raw result:

{
  "finish_reason": "length",
  "usage": {
    "prompt_tokens": 313,
    "completion_tokens": 8192,
    "total_tokens": 8505
  },
  "message": {
    "content": null,
    "reasoning_chars": 23687
  }
}

There is no agent loop here. The server receives one short request and returns one response.

---

6. Strongest result: increasing the reasoning budget does not make the task converge

Aggregate result

All 12 raw budget-scaling trials

This is the central finding.

If the issue were merely:

Laguna is a heavy reasoning model and needs a larger budget

then increasing the output allowance from 2,048 to 16,384 tokens should increase the probability that it eventually exits reasoning and answers.

It did not.

Observed completion probability:

P(final answer | 2048 budget)  = 0/3
P(final answer | 4096 budget)  = 0/3
P(final answer | 8192 budget)  = 0/3
P(final answer | 16384 budget) = 0/3

Total:

12/12 → finish_reason=length
12/12 → substantial structured reasoning
12/12 → no final answer

The mean reasoning text grew from ~6.5K characters at the 2K-token cap to ~52.8K characters at the 16K-token cap.

That is why I believe "runaway reasoning" / "unbounded reasoning attractor" is more accurate than simply saying the model "thinks for a long time."

---

7. Representative runaway trace

This is an abbreviated excerpt from a 16,384-token failure.

The model starts in the correct neighborhood:

The bug is clear: in the `put` method, the `self.data[key] = value`
assignment is outside the lock.

...

Wait, but the question says "returns a value for a key that was never inserted."
Let me think more carefully.

Actually, the more direct issue...

But the question says "returns a value for a key that was never inserted."
Let me think of another scenario.

Actually, let me reconsider.

Hmm, but the KeyError would be a crash, not a wrong value.
Let me think about how a wrong value gets returned.

...

Wait, actually, maybe the issue is even simpler.
Let me re-read the code.

...

Let me think about this differently.

...

Actually, wait. Let me re-read the question...

...

Actually, I think the key scenario...

...

Actually, wait. Let me re-read the question...

The full raw response eventually consumes all 16,384 completion tokens and never emits final content.

This is not simple exact-token repetition. The model is performing semantic recursive reconsideration:

identify likely bug
    ↓
construct interleaving
    ↓
question whether wording matches symptom exactly
    ↓
construct alternative
    ↓
question alternative
    ↓
re-read prompt
    ↓
rediscover original bug
    ↓
repeat

The 16K trials contained dozens to more than 100 reset/reconsideration markers, depending on the run.

---

8. Temperature sweep: stochastic sampling is not required

The important arm is:

temperature = 0.0
reasoning active = 3/3
finish_reason=length = 3/3
correct final answer = 0/3

So the failure does not require temperature 0.7, temperature 1.0, or stochastic wandering.

Under this reproduction, greedy decoding can enter the same non-convergent state.

This does not prove that sampling has zero influence on failure probability in every workload. It does show that stochastic sampling is not a necessary condition.

---

9. Concurrency sweep: batching/parallel requests are not required

The same prompt and same requested seed (42) were run at concurrency 1, 2, and 4.

All 9 trials:

entered substantial reasoning
hit the 8,192-token limit
produced no final answer

Therefore the failure does not require concurrent clients or a heavily batched workload.

Same-seed caveat

The reasoning hashes differ even at concurrency 1.

I am recording that as an observation only. I am not presenting it as a Laguna-specific defect because online GPU inference may have sources of nondeterminism that are not controlled by simply supplying the OpenAI-compatible seed field.

The important concurrency result is narrower:

Concurrency is not required to reproduce the runaway.

---

10. Repetition penalty: a real mitigation signal, but not a complete fix

This is more nuanced than the earlier tests.

repetition_penalty=1.03

reasoning active: 3/3
failure:          3/3
failure | active: 3/3

No meaningful improvement.

repetition_penalty=1.05

Two of three runs entered substantial reasoning.

One active-reasoning run converged; one did not.

repetition_penalty=1.10

Reasoning activated 3/3; one of those three active-reasoning trajectories successfully terminated.

repetition_penalty=1.15

This is the most interesting setting:

reasoning active:      2/3
failure | active:      0/2
task root cause found: 3/3
overall length failure: 1/3

Two trials genuinely entered substantial structured reasoning and later exited it successfully:

That is important because it means 1.15 did more than merely turn thinking off in those two samples.

However, the third trial produced:

reasoning chars = 0
content chars   = 49,200
finish_reason   = length

The model found the right bug but failed to terminate in the normal content channel.

So my conclusion is:

repetition_penalty=1.15 materially mitigates the reasoning runaway in this workload, but it is not a complete termination fix.

This aligns directionally with the independent results in discussion #10, where another DGX Spark user reported:

I consider that useful corroboration, not part of my own dataset.

---

11. Explicit "stop reconsidering" instructions mitigate but do not eliminate it

I tested three system-policy variants while holding the LRU task and sampling constant.

The policies were:

Neutral

Solve the user's task accurately and directly.

Brief self-limit

Think briefly, using at most 300 words of private reasoning.
Verify the bug once, then answer.
Do not restart the analysis.

Commit-once

Form one hypothesis, verify it once against the code, then commit.
Do not explore alternative explanations after finding the interleaving
and minimal fix.

The failure still occurs under explicit instructions to stop reconsidering.

However, these instructions do improve completion probability.

This tells me two things:

1. the failure is instruction-sensitive, so this is not a completely mechanical inability to emit the reasoning terminator;
2. once the bad trajectory is entered, explicit self-bounding is not reliably obeyed.

A successful commit run generated 25,379 reasoning characters, then exited reasoning and produced a correct final answer.

That is useful evidence against a simplistic explanation like:

the reasoning parser cannot ever terminate correctly.

It demonstrably can.

The problem is that many reasoning trajectories do not reach that exit state.

---

12. Thinking OFF control

Three LRU trials were run with:

{
  "chat_template_kwargs": {
    "enable_thinking": false
  }
}

Result:

This is important but also prevents me from overclaiming.

Thinking OFF makes the severe reasoning-channel failure disappear, and all three outputs contain the correct root cause.

But one trial still consumed the full 8,192-token budget in the normal content channel.

So there appears to be a broader overgeneration/reconsideration susceptibility:

thinking ON
    → frequently manifests as structured reasoning runaway
    → often never reaches final answer

thinking OFF / reasoning skipped
    → usually finishes correctly
    → occasionally overthinks/repeats in normal content instead

This is why I describe the reasoning path as the primary amplifier/manifestation, not necessarily the only possible location where nontermination can occur.

---

13. Aggregate v9 result conditioned on reasoning activation

Across all 57 v9 trials:

This table mixes deliberate mitigations and controls, so I would not use the aggregate percentage as the sole defect metric.

Still, it illustrates the same bifurcation seen repeatedly throughout the experiments:

When substantial structured reasoning activates, failure probability is much higher.

The cleanest evidence remains the controlled LRU baseline and budget-scaling arms.

---

14. Full v9 experiment matrix

For completeness:

Definitions:

• Reasoning active: structured reasoning / reasoning_content ≥ 1,000 characters.

• Objective failure: any of:

  • finish_reason == "length";
  • no usable final answer;
  • task-specific root-cause check failed;
  • request error.

• The objective failure metric does not depend on semantic loop keywords like wait, actually, or reconsider.

The loop/reset counters are retained only as supporting forensic evidence.

---

15. Earlier calculator reproduction: this is not unique to the LRU prompt

Before switching to the cleaner LRU reproduction, I ran a direct-API calculator-layout task in English and French.

The task was a small HTML-only UI rearrangement with explicit constraints.

Results:

This gave an important control:

thinking itself is not always broken

The trivial count-to-20 workload completes normally even with thinking enabled.

The pathological state is triggered by particular reasoning/coding trajectories, not by every invocation of <think>.

---

16. Earlier parser A/B: the structural parser issue is separately controlled

Before the runaway experiments, I directly A/B tested implicit vs explicit thinking configuration.

Results:

* Four of the simple explicit no-tool requests answered directly without reasoning; all ten remained structurally clean.

That experiment established that explicit:

{
  "chat_template_kwargs": {
    "enable_thinking": true
  }
}

correctly separates reasoning from final content.

All runaway experiments in this post use the corrected explicit configuration.

Therefore the 2K→16K failures are not caused by the already-known malformed history/parser mismatch.

---

17. What the current tests rule out as necessary causes

For this direct reproduction, the runaway does not require:

---

18. What the data strongly support

Confirmed from my tests

A. A reproducible runaway-reasoning state exists

The LRU task repeatedly causes Laguna S 2.1 NVFP4 RC1 to enter structured reasoning and consume the entire generation budget without producing final content.

B. The state is budget-unbounded over the tested range

2K, 4K, 8K, and 16K all fail 3/3.

More budget causes more reasoning, not convergence.

C. Stochastic sampling is not necessary

Temperature 0.0 fails 3/3.

D. Parallel batching is not necessary

Concurrency 1 fails 3/3.

E. Repetition dynamics matter

Higher repetition penalties substantially increase termination probability, including successful termination of active structured reasoning at 1.15.

F. The model is capable of exiting structured reasoning

Several penalty/policy runs generated thousands or tens of thousands of reasoning characters and then correctly transitioned into final content.

Therefore this is not simply "the model can never close thinking."

G. Reasoning is a strong amplifier

Thinking OFF finds the code root cause 3/3 and normally terminates 2/3, whereas the clean thinking-ON LRU baseline fails 5/5 in the preceding run and all budget/temperature baseline-style v9 arms fail consistently.

---

19. Leading hypothesis vs. proven root cause

What I think is happening

The empirical behavior looks like a reasoning-policy attractor:

identify plausible solution
        ↓
verify
        ↓
notice wording / edge case
        ↓
reconsider
        ↓
construct alternative
        ↓
question alternative
        ↓
return to earlier solution
        ↓
verify again
        ↓
repeat until max_tokens

This is a behavioral characterization.

It does not tell us by itself whether the underlying implementation cause is:

• checkpoint/training behavior;
• W4A4 activation quantization;
• quantization-induced expert/router/activation perturbations;
• backend numerical behavior;
• another model-runtime interaction.

Why W4A4/NVFP4 is a serious hypothesis

Poolside stated in discussion #7 that RC1:

loops far less commonly in W4A4 & not at all in W4A16

and said another checkpoint was needed because W4A4 still looped.

The current NVFP4 config also shows 4-bit expert weights and 4-bit input activations:

https://huggingface.co/poolside/Laguna-S-2.1-NVFP4/blob/main/config.json

Relevant config structure:

{
  "input_activations": {
    "group_size": 16,
    "num_bits": 4,
    "type": "float"
  },
  "weights": {
    "group_size": 16,
    "num_bits": 4,
    "type": "float"
  }
}

That makes W4A4 execution a very plausible contributor.

But:

I have not yet run an otherwise-identical W4A16/INT4/BF16 A/B on this machine.

Therefore I am not claiming activation quantization is proven causal.

---

20. Independent corroboration

The behavior is not unique to my setup.

Poolside discussion #7

Multiple DGX Spark users reported looping. Poolside responded that RC1 still looped in W4A4 in their internal testing:

https://huggingface.co/poolside/Laguna-S-2.1-NVFP4/discussions/7

Discussion #10

Another DGX Spark user produced essentially the same LRU direct-API reproduction:

https://huggingface.co/poolside/Laguna-S-2.1-NVFP4/discussions/10

Their expanded result:

Their failures also hit the generation ceiling with large reasoning_content and zero final answer.

That independent result is very consistent with the v9 findings above.

---

21. Minimal standalone reproduction script

This is a reduced version of the actual test logic. It has no third-party Python dependencies.

#!/usr/bin/env python3
import json
import urllib.request

ENDPOINT = "http://127.0.0.1:8006/v1/chat/completions"
MODEL = "laguna-s-2.1"

PROMPT = r"""
This cache is meant to be thread-safe and evict least-recently-used entries.
It passes single-threaded tests and passes a load test with 8 threads doing
random gets. In production under heavy concurrent writes it occasionally
returns a value for a key that was never inserted.

Find the bug, explain the exact interleaving that causes it, and give the
minimal fix.

```python
class LRUCache:
    def __init__(self, capacity):
        self.capacity = capacity
        self.data = {}
        self.order = []
        self.lock = threading.Lock()

    def get(self, key):
        with self.lock:
            if key not in self.data:
                return None
            self.order.remove(key)
            self.order.append(key)
            return self.data[key]

    def put(self, key, value):
        with self.lock:
            if key in self.data:
                self.order.remove(key)
            elif len(self.data) >= self.capacity:
                oldest = self.order.pop(0)
                del self.data[oldest]
            self.order.append(key)
        self.data[key] = value
```
"""

def run(max_tokens: int, seed: int):
    payload = {
        "model": MODEL,
        "messages": [
            {
                "role": "system",
                "content": "Solve the user's coding/debugging task accurately and directly."
            },
            {"role": "user", "content": PROMPT},
        ],
        "stream": False,
        "temperature": 0.7,
        "top_p": 0.95,
        "seed": seed,
        "max_tokens": max_tokens,
        "chat_template_kwargs": {"enable_thinking": True},
    }

    req = urllib.request.Request(
        ENDPOINT,
        data=json.dumps(payload).encode(),
        method="POST",
        headers={"Content-Type": "application/json"},
    )

    with urllib.request.urlopen(req, timeout=1800) as resp:
        result = json.load(resp)

    choice = result["choices"][0]
    msg = choice["message"]
    reasoning = msg.get("reasoning") or msg.get("reasoning_content") or ""
    content = msg.get("content") or ""

    print(
        f"budget={max_tokens:5d} "
        f"finish={choice.get('finish_reason'):6s} "
        f"completion_tokens={result.get('usage', {}).get('completion_tokens')} "
        f"reasoning_chars={len(reasoning):6d} "
        f"content_chars={len(content):6d}"
    )

for budget in (2048, 4096, 8192, 16384):
    for seed in (501, 502, 503):
        run(budget, seed)

Expected affected behavior resembles:

budget= 2048 finish=length completion_tokens=2048  reasoning_chars=~6000  content_chars=0
budget= 4096 finish=length completion_tokens=4096  reasoning_chars=~12000 content_chars=0
budget= 8192 finish=length completion_tokens=8192  reasoning_chars=~24000 content_chars=0
budget=16384 finish=length completion_tokens=16384 reasoning_chars=~52000 content_chars=0

The exact character counts vary.

The important invariants are:

reasoning active
finish_reason = length
completion_tokens = max_tokens
content absent

---

22. Single-request curl-style reproduction

Equivalent request body:

curl -s http://127.0.0.1:8006/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d @- <<'JSON'
{
  "model": "laguna-s-2.1",
  "messages": [
    {
      "role": "system",
      "content": "Solve the user's coding/debugging task accurately and directly."
    },
    {
      "role": "user",
      "content": "Use the LRU prompt from this issue."
    }
  ],
  "temperature": 0.7,
  "top_p": 0.95,
  "max_tokens": 8192,
  "stream": false,
  "chat_template_kwargs": {
    "enable_thinking": true
  }
}
JSON

For a fully standalone curl reproduction, replace the abbreviated user content with the exact prompt in section 4.

---

23. How I classified failures

The primary classifier deliberately avoids relying on subjective "this looks loopy" judgments.

A request is considered an objective failure if any of these hold:

objective_failure = (
    request_error
    or finish_reason == "length"
    or final_answer_is_missing
    or task_specific_root_cause_check_failed
)

Structured reasoning is considered "active" for analysis when:

len(reasoning_content) >= 1000

I also count semantic reset phrases such as:

actually
wait
reconsider
let me think
let me verify
let me re-read
try again

but those counters are supporting diagnostics only.

The 12/12 budget result does not depend on those phrases at all. It follows directly from:

completion_tokens == max_tokens
finish_reason == length
content == null / empty

---

24. Why I do not think "just raise max_tokens" is an adequate workaround

The budget experiment is specifically designed to test that idea.

If the model simply needed more room, the success curve should eventually improve.

Instead:

max_tokens ×8
    ↓
reasoning size increases
    ↓
success probability remains 0%

Within the tested 2K–16K range, the output cap acts as a circuit breaker, not as the cause of termination.

Increasing it merely allows the runaway trajectory to continue longer.

---

25. Why I do not think "just use temperature 1.0" explains it

Temperature sweep:

Even greedy decoding (temperature=0) fails 3/3.

This rules out temperature as a necessary cause for this prompt.

Sampling may still affect which trajectory is entered or the frequency of failures on other prompts, but it cannot explain the existence of the runaway state.

---

26. Why I do not think DFlash explains this reproduction

DFlash was intentionally disabled in the captured server.

The failure remains extremely reproducible.

Therefore:

DFlash may have its own effects
but
DFlash is not required for this failure

This is useful because some early reports understandably suspected speculative decoding.

---

27. Why this appears more like an attractor than simple repetition

The failing outputs are not primarily:

foo foo foo foo foo foo

They look like:

I found the bug.
Wait, does that exactly explain the symptom?
Let me derive another interleaving.
Actually, that may raise KeyError.
Let me reconsider.
Maybe the simpler case is...
Wait, reread the prompt.
I think the original bug is correct.
Let me make the interleaving precise.
Actually...

The semantics move locally, but the reasoning state returns to the same unresolved question.

That distinction matters because token-level repetition penalties can help without necessarily addressing the learned reasoning behavior that keeps reopening the problem.

---

28. What I would ask Poolside to test internally

The following A/Bs would help locate the implementation cause quickly.

A. Same exact LRU prompt on W4A4 vs W4A16

Hold constant:

checkpoint family
prompt
seed set
temperature/top_p
reasoning enabled
max_tokens
runtime
DFlash disabled

Compare:

P(length exhaustion | reasoning active)

If W4A16 enters substantial reasoning but reliably exits while W4A4 does not, that would be strong evidence of a precision/activation-quantization contribution.

B. Compare BF16 / FP8 / NVFP4 / INT4 with identical prompts

Record:

reasoning activation rate
reasoning tokens until final answer
EOS / reasoning-close probability over time
frequency of semantic reset phrases
finish_reason

C. Inspect token probabilities around reasoning exit

A particularly useful diagnostic would be to determine whether failing W4A4 trajectories:

1. never assign meaningful probability to the reasoning terminator;
2. approach the terminator and then veer away;
3. repeatedly generate self-verification phrases that reset the state.

D. Compare expert/router behavior between successful and runaway trajectories

Because Laguna is an MoE model and NVFP4 quantizes routed expert inputs/weights, it may be useful to compare:

expert selection distribution
router margins
activation statistics

between:

same prompt → short successful reasoning
same prompt → 16K runaway reasoning

This is a hypothesis for investigation, not a claim about the cause.

---

29. Suggested acceptance test for a future checkpoint

I would suggest making the LRU prompt a regression test.

For example:

N = 20 independent requests
thinking = true
temperature = 0.7
top_p = 0.95
max_tokens = 8192
DFlash = off

Pass criteria could be something like:

>= 19/20 produce final answer
>= 19/20 identify out-of-lock data write
0/20 consume full 8192-token budget with no answer

Then repeat at:

temperature = 0.0
repetition_penalty = default

and on both W4A4 and W4A16 execution paths.

---

30. Current conclusion, stated as narrowly as possible

Confirmed

On the tested Laguna S 2.1 NVFP4 RC1 checkpoint and vLLM 0.25.1 stack, a short single-turn coding/debug prompt reproducibly triggers structured reasoning that fails to converge and instead consumes the entire available generation budget.

The failure reproduced 12/12 times while scaling the output budget from 2,048 to 16,384 tokens, with no final answer in any of those trials.

The same failure reproduces at temperature 0.0 and concurrency 1, with DFlash disabled, so stochastic sampling, batching, speculative decoding, agent tooling, long context, and multi-turn history are not required.

Strongly indicated

Repetition dynamics are involved. repetition_penalty=1.15 materially improves termination and can stabilize active structured reasoning, but does not eliminate all nontermination/overgeneration.

The behavioral failure resembles a semantic self-verification/reconsideration attractor rather than simple exact-token repetition.

Not yet proven

I have not proven that NVFP4/W4A4 activation quantization is the implementation root cause.

Poolside's own W4A4 vs W4A16 observation makes this a strong next hypothesis, but a controlled cross-precision A/B is still needed.

---

31. Files / evidence available

The forensic bundle contains:

RUN_CONFIG.json
EXPERIMENT_MATRIX.csv
results.csv
SUMMARY.md
ROOT_CAUSE_SCORECARD.md
summary.json
models.json
container_inspect.json
server_startup_tail.log

trials/
  budget_2048/
  budget_4096/
  budget_8192/
  budget_16384/
  same_seed_concurrency_1/
  same_seed_concurrency_2/
  same_seed_concurrency_4/
  lru_temperature_0.0/
  lru_temperature_0.3/
  lru_temperature_0.7/
  lru_temperature_1.0/
  lru_rep_penalty_1.03/
  lru_rep_penalty_1.05/
  lru_rep_penalty_1.10/
  lru_rep_penalty_1.15/
  reasoning_policy_neutral/
  reasoning_policy_brief/
  reasoning_policy_commit/
  lru_thinking_off_control/

Every trial directory contains:

request.json
response.json
reasoning.txt
content.txt
RESULT.json
vllm.log

I can provide the ZIP and the complete harness if useful.

---

32. References

• Laguna S 2.1 model card:
  https://huggingface.co/poolside/Laguna-S-2.1

• Laguna S 2.1 NVFP4 model:
  https://huggingface.co/poolside/Laguna-S-2.1-NVFP4

• NVFP4 quantization config:
  https://huggingface.co/poolside/Laguna-S-2.1-NVFP4/blob/main/config.json

• NVFP4 generation config:
  https://huggingface.co/poolside/Laguna-S-2.1-NVFP4/blob/main/generation_config.json

• Poolside W4A4/W4A16 looping discussion:
  https://huggingface.co/poolside/Laguna-S-2.1-NVFP4/discussions/7

• Independent LRU reproduction / repetition-penalty results:
  https://huggingface.co/poolside/Laguna-S-2.1-NVFP4/discussions/10

• Separate reasoning-parser/history issue:
  https://huggingface.co/poolside/Laguna-S-2.1-NVFP4/discussions/15

---

33. Very short developer summary

Repro:
  Laguna S 2.1 NVFP4 RC1
  vLLM 0.25.1
  direct /v1/chat/completions
  DFlash OFF
  enable_thinking=true
  single-turn 313-token LRU debugging prompt

Budget sweep:
   2,048 -> 3/3 length, no final answer
   4,096 -> 3/3 length, no final answer
   8,192 -> 3/3 length, no final answer
  16,384 -> 3/3 length, no final answer

Temperature:
  0.0 / 0.3 / 0.7 / 1.0 -> all 3/3 failures

Concurrency:
  1 / 2 / 4 -> all 3/3 failures

DFlash:
  disabled

repetition_penalty=1.15:
  active reasoning terminated successfully 2/2
  but 1/3 overall still length-exhausted in normal content

Conclusion:
  confirmed reproducible non-convergent reasoning state;
  more output budget extends it rather than resolving it.
  W4A4/NVFP4 is a strong implementation hypothesis but not yet proven causal.

I am happy to run additional targeted experiments against this exact prompt if there is a particular internal hypothesis Poolside wants isolated.