Kus-hal/resume-screener-lora

Resume Screener -- LoRA Fine-Tuned (Qwen2.5-0.5B)

A LoRA adapter fine-tuned on Qwen2.5-0.5B-Instruct to output structured JSON verdicts for resume screening, instead of relying on prompting alone.

Why

Prompting a base instruct model for resume screening produces unstructured, inconsistent prose -- not something you can pipe into an ATS or scoring pipeline. This adapter makes structured JSON output the model's default behavior, not something you have to coax out with prompt engineering.

Training details

• Base model: Qwen2.5-0.5B-Instruct
• Method: LoRA (r=16, alpha=32, dropout=0.05) targeting q/k/v/o projections
• Trainable params: 2,162,688 / 496,195,456 total (0.44%)
• Dataset: 800 train / 96 eval examples, resume text -> structured JSON verdict
• Training: 3 epochs on a free Colab T4 GPU

Results

Epoch	Training Loss	Validation Loss
1	0.4078	0.2784
2	0.1956	0.1883
3	0.1713	0.1714

Validation loss tracked training loss closely with no divergence -- no overfitting.

Before vs after

Prompt: Screen this resume for a Backend Engineer position and return a structured verdict. Resume: Bachelor's in CS, 4 years distributed backend experience, skilled in Python, PostgreSQL, Docker, Kubernetes.

Base model (no fine-tuning): unstructured prose with markdown headers (Summary, Key Skills, Experience) -- not machine-parseable.

Fine-tuned model:

{"role": "Backend Engineer", "ats_score": 56, "verdict": "moderate_match", "matched_skills": ["Python", "PostgreSQL", "Docker", "Kubernetes"], "missing_skills": ["Java", "REST APIs", "Redis"], "years_experience": 4}

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct")
model = PeftModel.from_pretrained(base_model, "Kus-hal/resume-screener-lora")
tokenizer = AutoTokenizer.from_pretrained("Kus-hal/resume-screener-lora")

Links

• Training code + dataset generator: GitHub -- resume-screener-lora

Stack

PEFT/LoRA, Hugging Face Transformers, TRL, PyTorch, Colab T4 GPU