chaitalibh/finegrained-grpo-0824

Finegrained GRPO 0824

LoRA adapter trained from CELL-LAB/lora-plus-f2f-backup for Siheung RAG answering.

The fixed Toro prompt was kept unchanged during training and post-training generation. The reward setup emphasized groundedness, accuracy, and hallucination reduction.

Use

from transformers import AutoProcessor, Gemma3ForConditionalGeneration
from peft import PeftModel
import torch

base_id = "CELL-LAB/lora-plus-f2f-backup"
adapter_id = "chaitalibh/finegrained-grpo-0824"

processor = AutoProcessor.from_pretrained(base_id)
model = Gemma3ForConditionalGeneration.from_pretrained(
    base_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(model, adapter_id)
model.eval()