valendra/sherry-35b-a3b-0.1-preview

Sherry 35B-A3B 0.1 Preview

Controllable reasoning. Less unnecessary thinking. Same model, different effort.

Sherry 35B-A3B 0.1 is an experimental reasoning-tuned version of Qwen/Qwen3.6-35B-A3B designed around one idea:

not every problem should require the same amount of reasoning.

Instead of treating reasoning length as an accidental property of generation, Sherry exposes reasoning effort as an explicit control.

The model supports three project-specific effort levels:

• low
• medium
• high

This allows applications to trade latency and token consumption for additional reasoning depth depending on the task.

The preview was trained using SFT followed by GRPO, with rewards combining correctness, output structure, and effort-aware length shaping.

Highlights

• Explicit reasoning-effort control through dedicated tokenizer tokens and the chat template.
• 97.1% math_verify on MATH-500 in our controlled evaluation.
• 78.3% exact match on GPQA-Diamond, compared with 57.1% for the base model under the same harness.
• 97.5% pass@1 on our 40-task basic Python suite.
• Substantially lower completion-token usage on GPQA-Diamond than the base model.
• Built on a 35B-A3B MoE architecture, retaining the deployment characteristics of Qwen3.6-35B-A3B.
• Fully merged weights — no PEFT adapter required at inference time.
• Tested through the vLLM serving path used during evaluation.

The result is not simply a model trained to "think more". The objective is to make reasoning more deliberate and more controllable.

📖 Full training write-up:
How we trained Sherry and added controllable reasoning effort

---

Why reasoning effort?

Reasoning models often spend a large number of tokens on problems that do not require them.

For production systems, this matters.

More reasoning tokens can mean:

• higher latency,
• lower throughput,
• larger KV-cache requirements,
• higher inference cost,
• and worse user experience for simple requests.

Sherry explores a different interface: let the caller choose how much reasoning budget a request deserves.

For example:

low     → favor concise reasoning for straightforward tasks
medium  → balance reasoning depth and efficiency
high    → allow substantially more reasoning for difficult problems

The effort level becomes part of the inference request rather than something applications must approximate indirectly with max_tokens or prompting.

---

Evaluation

This preview includes a controlled comparison against the original Qwen3.6-35B-A3B base model.

Both models were executed using:

• vLLM 0.29.0
• BF16
• temperature=0.0
• top_p=1.0
• fixed seed
• one generation per problem
• identical prompts
• identical completion budgets

Sherry was evaluated with:

reasoning_effort=high

Results

Benchmark	Metric	Qwen3.6-35B-A3B	Sherry	Paired delta (95% CI)
MATH-500 (500)	math_verify	88.3%	97.1%	+9.4 (+4.7 to +14.7)
MATH-500 (500)	exact_match	25.2%	56.2%	+31.0 (+26.2 to +35.8)
GPQA-Diamond (198)	exact_match	57.1%	78.3%	+21.2 (+14.1 to +28.3)
Basic Python suite (40)	pass@1	75.0%	97.5%	+22.5 (+10.0 to +35.0)
AIME 2025 (30)	exact_match	63.3%	50.0%	−13.3 (−33.3 to +6.7)
AIME 2025 (30)	math_verify (shared problems)	100% (n=19)	51.7% (n=29)	−36.8 (−57.9 to −15.8)

The preview shows strong gains on MATH-500, GPQA-Diamond and our Python suite.

AIME 2025 result is under review

The current AIME 2025 numbers should be considered preliminary.

During post-evaluation review, we found enough inconsistencies in the AIME run to warrant repeating the benchmark before treating the apparent regression as representative of the model. In particular, the interaction between answer extraction, completion truncation, shared-problem accounting and the evaluation harness requires further validation.

The base model also reached the 32,768-token completion budget on 33.3% of AIME problems, which makes direct interpretation of the current comparison more difficult.

We are therefore publishing the current result for transparency, but we do not consider the AIME 2025 comparison final. A clean rerun with a reviewed evaluation path will replace or confirm these numbers in a future update.

We intentionally keep the current measurements visible rather than removing an unfavorable result while it is being investigated.

---

Reasoning efficiency

One of the most interesting results was not just accuracy, but how many tokens the model needed to reach an answer.

On GPQA-Diamond:

Model	Average completion tokens	Hit 32,768-token limit
Qwen3.6-35B-A3B	20.0k	27.8%
Sherry	7.5k	3.5%

Despite using the maximum Sherry reasoning-effort level, the preview generated approximately 62% fewer completion tokens on average on GPQA-Diamond in this evaluation.

That distinction matters for real inference systems.

Reasoning quality is useful.

Reasoning quality with less unnecessary computation is considerably more useful.

---

Evaluation protocol

• MATH-500 and AIME 2025 use the lm-evaluation-harness prompt formats.
• GPQA-Diamond uses the public hendrydong/gpqa_diamond_mc mirror containing 198 rows because the official Idavidrein/gpqa repository is gated.
• Completion budget is 32,768 tokens for math and GPQA tasks.
• Completion budget is 16,384 tokens for the code suite.
• MATH-500 and AIME 2025 exact_match use a heuristic extractor for unboxed answers.
• math_verify is the more robust metric for mathematical answers and is therefore reported alongside exact match.
• The code suite executes hidden unit tests inside an isolated subprocess.

Evaluation caveats

• Training data derives from NuminaMath, so MATH-500 and AIME 2025 may be contaminated. Treat those results as an upper bound.
• The evaluation uses a single greedy generation per problem.
• AIME 2025 contains only 30 problems, producing wide confidence intervals.
• The base model is completion-budget limited on several tasks.
• The current AIME 2025 evaluation is under review and will be rerun before its observed delta is treated as representative.
• Preliminary inspection suggests the AIME evaluation path may have been affected by answer extraction, truncation or harness-level accounting issues; this has not yet been conclusively isolated.
• Published numbers on the original Qwen model card use a different sampling protocol and should not be directly compared with the numbers in this table.

The evaluation should therefore be treated as controlled evidence about this preview, not as a definitive ranking of model capability.

---

Training pipeline

Sherry 0.1 was produced in two stages.

Stage 1 — Supervised fine-tuning

The supervised reasoning-effort adapter is published separately:

valendra/sherry-35b-a3b-0.1-sft-preview

This stage introduces the reasoning-effort behavior and the project's dedicated control tokens.

Stage 2 — GRPO

The SFT model was then continued using Group Relative Policy Optimization with an effort-aware reward function.

The final model published in this repository corresponds to:

checkpoint-200

The GRPO LoRA was merged into the original Qwen3.6-35B-A3B weights for this release.

For a detailed explanation of the training design, dataset construction, reward function and reasoning-effort mechanism, see:

👉 Sherry — Training a model to control its own reasoning effort

---

Training and merge details

• Base model: Qwen/Qwen3.6-35B-A3B
• Training method: GRPO with QLoRA/LoRA
• Adapter source: qwen36-reasoning-effort-grpo-0.1-qlora-200-12k/checkpoint-200
• Training steps: 200
• LoRA rank: 32
• LoRA alpha: 64
• LoRA dropout: 0.0
• Target modules: q_proj, k_proj, v_proj, o_proj, shared_expert.gate_proj, shared_expert.up_proj, shared_expert.down_proj
• Trainable token IDs: 248077, 248078, 248079
• Base precision during merge: bfloat16
• Published weights: safetensors
• Weight shards: 15
• Merge operation: PEFT merge_and_unload()

The original adapter artifacts are preserved separately from the merged release.

The published safetensors use the canonical Qwen 3.6 packed-MoE layout required by current inference runtimes.

---

GRPO reward

Checkpoint-200 was optimized using a composite reward over four generations per problem and effort level.

Correctness is the dominant signal.

Length shaping is deliberately bounded so that the model is encouraged to use an appropriate amount of reasoning without being rewarded for shortening an incorrect answer.

Component	Weight	Rule
Correctness	+2.0	Exact numeric match against the hidden ground truth using format-tolerant numeric comparison.
Partial correctness	0.0 to +0.05	Numeric near miss: 0.05 / (1 + relative_error) where relative_error = abs(predicted - truth) / (abs(truth) + 1).
Structure	±0.20	+0.20 when the thinking block is closed and a final answer is parseable; −0.20 when the boundary is missing, the answer is missing, or the completion reaches the global budget.
Length — low	−0.20 to +0.20	+0.20 at or below 1,024 thinking tokens; continuous penalty for overruns.
Length — medium	−0.20 to +0.20	+0.20 between 1,024 and 4,096 thinking tokens; linear decay below 1,024 and continuous penalty above.
Length — high	0.0	No length bonus or penalty. Target allows up to 32,768 thinking tokens inside a global 65,536-token completion budget.

Additional GRPO settings:

generations per group: 4
loss_type: dr_grpo
beta: 0.0
learning_rate: 5e-6
temperature: 0.7
top_p: 0.95
optimizer steps: 200
LoRA rank: 32
checkpoint interval: 50 steps

beta=0.0 means no reference-model KL penalty was used.

The checkpoint-200 logged reward of 2.2144 is a training signal and must not be interpreted as an independent benchmark or capability score.

---

Reasoning-effort interface

Sherry introduces three dedicated special tokens:

<|reasoning_effort_low|>     ID 248077
<|reasoning_effort_medium|>  ID 248078
<|reasoning_effort_high|>    ID 248079

The published chat template accepts:

reasoning_effort="low"
reasoning_effort="medium"
reasoning_effort="high"

and automatically emits the corresponding control token before the thinking block.

There is currently no xhigh level.

These tokens are specific to Sherry and are not reasoning-effort controls for the original Qwen3.6-35B-A3B model.

---

Transformers usage

from transformers import AutoModelForImageTextToText, AutoTokenizer

model_id = "valendra/sherry-35b-a3b-0.1-preview"

tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    device_map="auto",
    dtype="auto",
)

messages = [
    {
        "role": "user",
        "content": "Solve: 12 * 13"
    }
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    enable_thinking=True,
    reasoning_effort="medium",
    return_tensors="pt",
).to(model.device)

outputs = model.generate(
    inputs,
    max_new_tokens=2048,
)

print(
    tokenizer.decode(
        outputs[0, inputs.shape[-1]:],
        skip_special_tokens=False,
    )
)

For a non-thinking response:

enable_thinking=False

The architecture remains multimodal-compatible through Qwen3.6, although this preview was trained and evaluated primarily through the text-generation path.

---

vLLM

When serving with vLLM, provide the effort level through chat_template_kwargs:

{
  "model": "valendra/sherry-35b-a3b-0.1-preview",
  "messages": [
    {
      "role": "user",
      "content": "Solve: 12 * 13"
    }
  ],
  "chat_template_kwargs": {
    "reasoning_effort": "high"
  }
}

This makes reasoning effort controllable on a per-request basis, allowing the same deployment to serve workloads with different latency and reasoning requirements.

---

What this preview is testing

Sherry 0.1 is primarily an experiment in inference-time reasoning control.

The questions we are exploring include:

• Can one model learn to reason differently depending on an explicit effort signal?
• Can simple requests avoid unnecessarily long reasoning traces?
• Can additional reasoning capacity be preserved for harder requests?
• Can reasoning efficiency improve without relying exclusively on hard generation limits?
• Can effort become an application-level inference parameter in the same way temperature or top_p already are?

This release is an early step toward answering those questions.

---

Limitations

• This is a preview release.
• The reported evaluation represents a single controlled internal run rather than a multi-run public benchmark.
• MATH-500 and AIME 2025 may overlap with or be indirectly represented in training data derived from NuminaMath.
• Generation length and quality vary depending on reasoning-effort level and prompt.
• The current AIME 2025 result is preliminary and under review. We plan to rerun it after validating the evaluation pipeline before drawing conclusions from the apparent regression.
• The logged GRPO reward is not an accuracy metric.
• Behavior outside the evaluated domains has not been characterized extensively.
• Validate outputs independently before using the model in production or high-stakes applications.

---

Intended use

Sherry is intended for:

• research into controllable reasoning,
• experimentation with reasoning-effort interfaces,
• inference-efficiency research,
• mathematical and scientific reasoning experiments,
• coding experiments,
• evaluation of reasoning-token allocation strategies,
• and applications that want to vary reasoning depth on a per-request basis.

It should currently be treated as an experimental model rather than a drop-in replacement for the Qwen3.6 base model.

---

Learn more

The full story behind the model — including the motivation, SFT stage, GRPO setup, reward design and reasoning-effort implementation — is available on the Valendra engineering blog:

Sherry: controllable reasoning effort

Built by Valendra.