ProfRutPatel/Qwen2-0.5B-QLoRA-Conversational-English

Qwen2-0.5B QLoRA — Conversational American English

A QLoRA fine-tuned version of Qwen/Qwen2-0.5B, trained on conversational American English dialogues and everyday expressions.

Training Details

Parameter	Value
Base Model	Qwen/Qwen2-0.5B
Method	QLoRA (4-bit NF4 quantization + LoRA)
LoRA Rank	16
LoRA Alpha	32
Target Modules	q_proj, k_proj, v_proj, o_proj
Training Rounds	2
Total Training Samples	284 (99 + 185)
Trainable Parameters	2,162,688 / 317M (0.68%)

Training Data

Fine-tuned on 3 Conversational American English PDFs:

1. McGraw-Hill's Conversational American English
2. Additional conversational English resource
3. Everyday Conversations English dialogues

Training Metrics

Round 1 (99 samples, 75 steps, 4.5 min):

• Loss: 2.425 → 1.794
• Token Accuracy: 52.9% → 62.0%

Round 2 (185 samples, 141 steps, 7.7 min):

• Loss: 3.339 → 2.788
• Token Accuracy: 40.9% → 46.8%

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model + LoRA adapters
base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2-0.5B")
model = PeftModel.from_pretrained(base_model, "Rut-ai/Qwen2-0.5B-QLoRA-Conversational-English")
tokenizer = AutoTokenizer.from_pretrained("Rut-ai/Qwen2-0.5B-QLoRA-Conversational-English")

# Generate
inputs = tokenizer("Teach me how to greet someone in American English", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Hardware

• Trained on NVIDIA GeForce GTX 1650 (4GB VRAM)
• Total training time: ~12 minutes