CADT-IDRI/qwen-khmer-text-sum-adapters

🇰🇭 Khmer Text Summarization Adapters (Qwen)

QLoRA adapters fine-tuned for Khmer text summarization.
Trained using Unsloth for efficient 4-bit fine-tuning.

📂 Variants

Variant	Subfolder	Description
Title-based	title_based/	Trained on raw Khmer news dataset
Synthetic	synthetic/	Trained on synthetic dataset

🚀 Usage

from unsloth import FastLanguageModel import torch

ALPACA_PROMPT = """ខាងក្រោមនេះគឺជាសេចក្តីណែនាំអំពីកិច្ចការមួយ។ សូមផ្តល់ចម្លើយឱ្យបានត្រឹមត្រូវ ពេញលេញ និងងាយយល់។

Instruction:

ចូលសង្ខេប អត្ថបទខាងក្រោមនេះ

Input:

{}

Response:

"""

model, tokenizer = FastLanguageModel.from_pretrained( model_name="unsloth/Qwen2.5-7B-Instruct-bnb-4bit", max_seq_length=8192, load_in_4bit=True, adapter_name="ChilyRan/qwen-khmer-adapters", adapter_kwargs={"subfolder": "synthetic"} # or "title_based" ) FastLanguageModel.for_inference(model)

text = "បញ្ចូលអត្ថបទខ្មែររបស់អ្នកនៅទីនេះ..." prompt = ALPACA_PROMPT.format(text) inputs = tokenizer(prompt, return_tensors="pt", truncation=True).to("cuda")

with torch.no_grad(): outputs = model.generate( **inputs, max_new_tokens=128, use_cache=True, do_sample=True, temperature=0.3, top_p=0.85 )

decoded = tokenizer.decode(outputs[0], skip_special_tokens=True) summary = decoded.split("### Response:")[-1].strip() print(summary)

⚙️ Training Details

Config	Value
Base model	unsloth/Qwen2.5-7B-Instruct-bnb-4bit
Method	QLoRA
Framework	Unsloth
Max sequence length	8192
Task	Khmer text summarization
Seed	42