Ekkoliu/QwenMathSolving

Qwen3-4B-Thinking-2507 — Math Reasoning LoRA

LoRA adapter for Qwen/Qwen3-4B-Thinking-2507, fine-tuned for a CSE 151B math/statistics reasoning competition.

Training

• Method: rejection-sampling fine-tuning (STaR). For each public training question, several reasoning traces were sampled from the base model; only traces whose extracted answer matched the gold answer were kept (1,205 traces from 624 questions). The adapter was trained on these with the prompt tokens masked, so loss is computed only on the reasoning + answer.
• LoRA: rank 32, α 64, dropout 0.05, targets all attention + MLP projections (q/k/v/o/gate/up/down).
• Optim: lr 1e-4, cosine schedule, 2 epochs, effective batch 16, bf16, gradient checkpointing.

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B-Thinking-2507", dtype="bfloat16")
model = PeftModel.from_pretrained(base, "Ekkoliu/qwen3-4b-thinking-2507-math-lora")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-4B-Thinking-2507")

Or with vLLM (as used in the competition pipeline, with self-consistency voting):

from vllm import LLM
from vllm.lora.request import LoRARequest
llm = LLM("Qwen/Qwen3-4B-Thinking-2507", enable_lora=True, max_lora_rank=32)
# pass lora_request=LoRARequest("adapter", 1, "<local_or_hub_path>") to llm.chat(...)

Framework versions

• PEFT 0.19.1