code-aitazaz/roman-urdu-qlora-qwen3-8b

Roman Urdu QLoRA Adapter for Qwen3-8B

A LoRA adapter (4-bit QLoRA fine-tune) that steers Qwen/Qwen3-8B — an already-capable multilingual instruction-tuned model — to respond fluently and consistently in Roman Urdu (Urdu written in Latin script). This is style/behavior adaptation on a small dataset (~485 examples), not knowledge injection: the base model's own multilingual pretraining already covers Urdu-adjacent text, this adapter's job is to make it default to answering in Roman Urdu.

Full training code, data pipeline, and a pedagogically-commented Kaggle training notebook: see the project repo (link in the parent GitHub repository once published).

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=torch.float16,  # not bfloat16 -- see note below if on a Turing GPU (e.g. T4)
)

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B", quantization_config=bnb_config, device_map="auto")
model = PeftModel.from_pretrained(base, "code-aitazaz/roman-urdu-qlora-qwen3-8b")

messages = [{"role": "user", "content": "Pakistan ka dar-ul-hukumat kya hai?"}]
prompt_ids = tokenizer.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True, enable_thinking=False, return_tensors="pt"
)["input_ids"].to(model.device)

output_ids = model.generate(
    prompt_ids,
    max_new_tokens=256,
    do_sample=False,
    repetition_penalty=1.2,   # see "Known limitation" below -- important
    no_repeat_ngram_size=3,
    pad_token_id=tokenizer.pad_token_id or tokenizer.eos_token_id,
)
print(tokenizer.decode(output_ids[0][prompt_ids.shape[1]:], skip_special_tokens=True))

Note on hardware: bnb_4bit_compute_dtype=torch.float16 (not bfloat16) is deliberate — a T4 (Turing, compute capability 7.5) has no bf16 tensor cores; most QLoRA examples default to bf16 assuming A100-class hardware.

Training

• Base model: Qwen/Qwen3-8B, Apache-2.0.
• Data: Redgerd/roman-urdu-alpaca-qa-mix, filtered down to ~485 genuinely-Roman-Urdu rows after finding and removing a real duplication bug in the upstream dataset (see the project repo's README for the full data-quality writeup).
• Method: 4-bit NF4 QLoRA, rank 16, alpha 32, targeting all 7 linear projections (attention + MLP), 3 epochs, effective batch size 16, on a single Kaggle T4.

Results

Real numbers from an actual training run (ROUGE-L on 40 held-out prompts, base vs. this adapter):

Model	ROUGE-L Precision	ROUGE-L Recall	ROUGE-L F-measure
Base (zero-shot)	0.194	0.128	0.095
This adapter	0.211	0.174	0.164

Known limitation, reported honestly: the adapter reliably achieves its actual goal (base Qwen3-8B often answers Roman Urdu prompts in English or garbled Urdu script; this adapter answers in Roman Urdu, on-topic, every time) — but several longer completions degrade into verbatim repetition loops under plain greedy decoding, a known small-dataset (485 rows) + do_sample=False failure mode. Use repetition_penalty≈1.2 and no_repeat_ngram_size≈3 at generation time (as in the usage snippet above) — this is a decoding-time fix, not a retraining requirement, added after the run that produced the numbers above (which predate the fix). See the project repo's README for full example transcripts showing both the successes and the repetition failure mode.

License

Adapter weights: MIT (same as the project code). Base model: Apache-2.0. Training data: Apache-2.0.