poolside

/

Laguna-S-2.1-NVFP4

like 126

Follow

Poolside 1.03k

Aside from the looping issue, the model does do a lot of thinking. From a speed and efficiency standpoint, would it make sense to have a reasoning budget flag implemented?

I've been using vLLM with --reasoning-config={"reasoning_start_str":"<think>","reasoning_end_str":"</think>"}, --default-chat-template-kwargs={"enable_thinking":true} and it's been effective. This is how I tested it:

curl -s http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "poolside/Laguna-S-2.1-NVFP4",
    "messages": [
      { "role": "user", "content": "9.11 and 9.8, which is greater?" }
    ],
    "thinking_token_budget": 10
  }'

And the thinking cuts off before it even finishes a sentence.

Aside from the looping issue, the model does do a lot of thinking. From a speed and efficiency standpoint, would it make sense to have a reasoning budget flag implemented?

I've been using vLLM with --reasoning-config={"reasoning_start_str":"<think>","reasoning_end_str":"</think>"}, --default-chat-template-kwargs={"enable_thinking":true} and it's been effective. This is how I tested it:

curl -s http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "poolside/Laguna-S-2.1-NVFP4",
    "messages": [
      { "role": "user", "content": "9.11 and 9.8, which is greater?" }
    ],
    "thinking_token_budget": 10
  }'

And the thinking cuts off before it even finishes a sentence.

I don't want to get it to stop thinking. I just want to get it to stop thinking for 30k tokens. Is it respecting the "thinking_token_budget" parameter?

Just confirmed that using the thinking token budget parameter doesn't return an error, but also doesn't constrain thinking in any way. Had one test where it didn't think, and then two more where it exhausted 32k tokens.

I don't want to get it to stop thinking.

The example I showed did not get it to stop thinking. The thought I got is "Okay, so I have this problem here: "", then it outputs the answer: "9.8 is greater than 9.11. ..."