valendra/sherry-35b-a3b-0.1-sft-preview

Sherry 35B-A3B SFT Preview

Sherry is an experimental supervised fine-tuning (SFT) LoRA adapter for Qwen/Qwen3.6-35B-A3B. It teaches the model to condition its reasoning style on one of three trained control tokens: low, medium, or high reasoning effort.

This is a preview adapter, not a standalone set of model weights. Load it on top of the base model with PEFT. The base model remains responsible for the general language, coding, mathematics, and multimodal capabilities.

What It Does

The adapter was trained on verified text-only mathematics and reasoning traces. Each example contains a thinking span and a final answer, and is labeled with one of these special tokens:

<|reasoning_effort_low|>
<|reasoning_effort_medium|>
<|reasoning_effort_high|>

The selected token is placed immediately before the Qwen3.6 thinking opener:

<|reasoning_effort_medium|><think>
... reasoning ...
</think>

... final answer ...

The tokens bias the amount and depth of reasoning. They are not hard length limits and do not guarantee a particular number of tokens, correctness, or termination behavior.

Model Details

• Model type: PEFT LoRA adapter for a causal multimodal language model
• Base model: Qwen/Qwen3.6-35B-A3B
• Base model size: 35B total parameters, approximately 3B activated per token
• Base context length: 262,144 tokens according to the base model card
• Adapter rank: 32
• LoRA alpha: 64
• Trainable effort controls: three tokenizer and embedding/output token rows
• Training modality: text-only reasoning examples
• Status: experimental SFT preview, version 0.1

The base model includes a vision encoder, but this adapter was trained and validated on text-only examples. No separate vision benchmark claim is made for this adapter.

Chat Template

This repository includes the Qwen3.6 chat_template.jinja used by the SFT run. It renders a user message and an assistant thinking opener. The trained effort token is inserted by the caller immediately before <think>\n; the template itself is intentionally kept compatible with the base Qwen3.6 format.

Usage

Install current versions of Transformers and PEFT, then load the base model and adapter:

import torch
from peft import PeftModel
from transformers import AutoModelForImageTextToText, AutoProcessor

base_id = "Qwen/Qwen3.6-35B-A3B"
adapter_id = "valendra/sherry-35b-a3b-0.1-sft-preview"

processor = AutoProcessor.from_pretrained(adapter_id)
base = AutoModelForImageTextToText.from_pretrained(
    base_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(base, adapter_id)
model.eval()
tokenizer = getattr(processor, "tokenizer", processor)

Render a text prompt with a selected effort level. The explicit insertion is important because the three effort tokens are trained controls, not arguments understood by the generic base chat template:

EFFORT_TOKENS = {
    "low": "<|reasoning_effort_low|>",
    "medium": "<|reasoning_effort_medium|>",
    "high": "<|reasoning_effort_high|>",
}
THINKING_OPEN = "<think>\n"


def format_prompt(question: str, effort: str) -> str:
    if effort not in EFFORT_TOKENS:
        raise ValueError(f"unknown effort: {effort}")
    rendered = tokenizer.apply_chat_template(
        [{"role": "user", "content": question}],
        tokenize=False,
        add_generation_prompt=True,
        enable_thinking=True,
    )
    if not rendered.endswith(THINKING_OPEN):
        raise ValueError("the Qwen3.6 chat template did not produce <think>\n")
    return rendered[:-len(THINKING_OPEN)] + EFFORT_TOKENS[effort] + THINKING_OPEN


prompt = format_prompt("What is 17 times 23?", "medium")
inputs = processor(text=prompt, return_tensors="pt")
inputs = {key: value.to(model.device) for key, value in inputs.items()}
outputs = model.generate(
    **inputs,
    max_new_tokens=4096,
    do_sample=False,
)
new_tokens = outputs[0, inputs["input_ids"].shape[-1]:]
print(tokenizer.decode(new_tokens, skip_special_tokens=False))

The example uses a 4,096-token caller-side generation budget for a practical demo. Increase or decrease that value for the available hardware and task.

Training Data And Procedure

The SFT examples come from the public valendra/sherry-reasoning-effort-0.1-dataset, which contains verified numeric-answer reasoning traces at three observed effort levels. Its problems are sampled from AI-MO/NuminaMath-CoT, with proof-style entries filtered out for this experiment.

The published adapter records the following run configuration:

• 1,854 training rows and 207 evaluation rows
• 500 optimizer steps
• Maximum training sequence length of 4,096 tokens
• BF16 mixed precision
• Per-device batch size 1 with gradient accumulation 16
• Learning rate 1e-4 with cosine scheduling and 50 warmup steps
• 8-bit AdamW optimizer
• LoRA rank 32, alpha 64, and zero dropout
• Seed 42

The private/generated source rows and training logs are not included in this model repository.

Evaluation

This repository does not claim an independent benchmark improvement. The SFT run included an evaluation split for training diagnostics, but the later GRPO experiment was not completed and its artifacts are not part of this release. Users should evaluate the adapter on their own tasks and compare it against the base model using the same prompt, decoding, and token budget.

Limitations And Risks

• This is an experimental preview and may regress on tasks unrelated to the training distribution.
• Reasoning-effort tokens influence behavior probabilistically; they are not strict compute controls.
• Long reasoning can be truncated, and the model may omit a thinking boundary or a final answer.
• The adapter inherits the base model's factual, safety, bias, and multimodal limitations.
• The 35B model requires substantial memory and is not intended for small consumer devices without quantization or offloading.
• This adapter was trained on text-only examples; vision behavior is inherited from the base model and was not independently evaluated here.

Do not use this preview as the sole basis for high-impact decisions. Inspect outputs and apply the safety and licensing requirements of the base model in your deployment context.

License And Attribution

The base model is released under the Apache 2.0 license. This repository contains a PEFT adapter that depends on the base model; download and use both repositories consistently with the base model's license and terms.

• Base model: https://huggingface.co/Qwen/Qwen3.6-35B-A3B
• Base license: https://huggingface.co/Qwen/Qwen3.6-35B-A3B/blob/main/LICENSE
• Training dataset: https://huggingface.co/datasets/valendra/sherry-reasoning-effort-0.1-dataset

Model Card Authors

Valendra Labs