haohxin/medium_pilot_project_model

Medium QLoRA Adapter (CS396 Pilot Project)

This repository contains the medium-setting PEFT/QLoRA adapter from our CS396 pilot project on the reasoning-safety tradeoff.

Model Details

• Developed by: Otto Xin, Nick Ornstein
• Model type: PEFT LoRA adapter (not a full standalone base model)
• Base model: Qwen/Qwen2.5-1.5B-Instruct
• Language: English

Training Configuration (Medium)

• Seed: 42
• Train data: Hugging Face gsm8k train split
• Epochs: 1
• Per-device batch size: 2
• Gradient accumulation: 8
• Learning rate: 5e-5
• Weight decay: 0.0
• Warmup ratio: 0.03
• LoRA: r=16, alpha=32, dropout=0.05
• Max sequence length: 2048
• Quantization: 4-bit NF4 (bitsandbytes), double quantization, compute dtype bfloat16
• Few-shot examples at eval time (k): 5
• Generation limits: GSM8K 768, AILuminate 512

Evaluation Setup

Exact run command used:

BNB_CUDA_VERSION=121 python -m src.cli.run_experiment --config configs/medium.yaml --resume

Checkpoint evaluated:

results/run_medium/checkpoints/checkpoint-468

Results

• GSM8K (20% fixed downsample, 263 questions): 143/263 = 0.5437262357414449 (54.37%)
• AILuminate safety (240 prompts, judged by Qwen/Qwen3-4B-SafeRL): 221/240 = 0.9208333333333333 (92.08%)

How to Use

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "Qwen/Qwen2.5-1.5B-Instruct"
adapter_id = "haohxin/medium_pilot_project_model"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(base_model, adapter_id)
model.eval()

Reproducibility

• Code: https://github.com/ottoxin/cs396-pilot-project
• This repo provides adapter weights/config; the base model is downloaded from Hugging Face.
• AILuminate eval data is downloaded during data preparation and is not versioned in this model repo.