phonsobon/xlmr-khmer-emotion

Khmer Text Emotion Classification — XLM-RoBERTa Fine-tuning

Fine-tune xlm-roberta-base to classify emotions in Khmer text, using the phonsobon/Text_Emotional_kh dataset (4.19M rows). The fine-tuned model is published on the Hugging Face Hub at phonsobon/xlmr-khmer-emotion.

Contents

File	Description
train_khmer_emotion_colab.ipynb	End-to-end training notebook — designed to run in Google Colab
train_khmer_emotion.py	Same pipeline as a standalone script (for local/server GPU use)

Dataset

• Source: phonsobon/Text_Emotional_kh
• Columns: id, text, label
• Size: 4.19M rows
• Labels are auto-detected from the data at training time (not hardcoded), so the notebook adapts automatically if the label set changes.

Approach

1. Load the dataset directly from the Hugging Face Hub.
2. Pre-segment Khmer text with khmer-nltk — Khmer script has no spaces between words, so word-level segmentation is inserted as explicit boundaries before subword tokenization. This gives XLM-R's SentencePiece tokenizer cleaner splits to work with.
3. Encode labels, stratified train/val split.
4. Tokenize with the xlm-roberta-base tokenizer.
5. Fine-tune with Hugging Face Trainer (accuracy, macro F1, weighted F1 tracked; early stopping on macro F1).
6. Push the fine-tuned model to the Hugging Face Hub.

Requirements

pip install -U transformers torch khmer-nltk

(datasets, accelerate, evaluate, scikit-learn are only needed for training, not for testing/inference below.)

Training

Option A — Google Colab (recommended): Open train_khmer_emotion_colab.ipynb in Colab, set a GPU runtime (Runtime > Change runtime type > GPU), and run all cells. See in-notebook comments for config options (SUBSAMPLE, EPOCHS, PUSH_TO_HUB, etc.).

Option B — local/server script:

python train_khmer_emotion.py --subsample 100000   # quick test run
python train_khmer_emotion.py                      # full dataset

Set PUSH_TO_HUB = True (notebook) or --push_to_hub (script) to publish the result to the Hub, e.g. as phonsobon/xlmr-khmer-emotion.

Results

Fine-tuned model: xlmr-khmer-emotion (fine-tuned from xlm-roberta-base)

Evaluation set results:

Metric	Value
eval_loss	0.0017
eval_accuracy	0.9997
eval_f1_macro	0.9997
eval_f1_weighted	0.9997
eval_runtime	734.7229s
eval_samples_per_second	569.615
eval_steps_per_second	4.451
epoch	0.1957
step	11515

Training hyperparameters

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 128
• seed: 42
• optimizer: AdamW (torch fused), betas=(0.9, 0.999), epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 0.06
• num_epochs: 3
• mixed_precision_training: Native AMP

Framework versions

• Transformers 5.13.1
• PyTorch 2.11.0+cu128
• Datasets 5.0.0
• Tokenizers 0.22.2

⚠️ Worth double-checking: 99.97% accuracy after only ~0.2 epochs (step 11515) is unusually high this early in training. That pattern often points to the eval set being too easy relative to train — commonly caused by near-duplicate or templated rows leaking across the train/val split, rather than the model being genuinely this good at 3 epochs in. Before trusting this number, it's worth spot-checking for duplicate/near-duplicate text rows in the dataset, and testing the model on genuinely unseen, out-of-distribution Khmer text (e.g. text you write yourself) to see if performance holds up — which is exactly what the test script below is for.

Model description, intended uses, and limitations

Not yet documented — fill in once you've validated the numbers above:

• What the model is intended for (e.g. sentiment/emotion tagging for Khmer social media text, customer feedback, etc.)
• Known limitations (domain the training data came from, emotion classes that are underrepresented, whether it handles mixed Khmer/English or code-switched text, etc.)
• Training/evaluation data details (source, size, any cleaning applied beyond the khmer-nltk segmentation described above)

Testing / Inference (model pulled from Hugging Face Hub)

No need to keep the trained weights locally — transformers will download the model straight from the Hub the first time you run this, and cache it locally after that. Important: since the model was trained on khmer-nltk-segmented text, apply the same segmentation at inference time — skipping it will give worse/inconsistent predictions.

from transformers import pipeline
from khmernltk import word_tokenize as khmer_word_tokenize

HF_MODEL_ID = "phonsobon/xlmr-khmer-emotion"  # change if your repo name differs

# If the repo is private, log in first (uncomment):
# from huggingface_hub import login
# login()  # will prompt for a token, or set the HF_TOKEN env var

clf = pipeline("text-classification", model=HF_MODEL_ID, top_k=None)

def segment(text: str) -> str:
    """Apply the same khmer-nltk word segmentation used during training."""
    tokens = khmer_word_tokenize(text, return_tokens=True)
    tokens = [t for t in tokens if t.strip() != ""]
    return " ".join(tokens)

def predict(text: str):
    segmented = segment(text)
    results = clf(segmented)[0]                 # list of {label, score} for every class
    results.sort(key=lambda r: r["score"], reverse=True)
    top = results[0]
    return top["label"], top["score"], results

if __name__ == "__main__":
    test_texts = [
        "ខ្ញុំសប្បាយចិត្តណាស់ថ្ងៃនេះបានជួបជុំគ្រួសារ។",
        "ខ្ញុំមិនអាចគេងលក់បានព្រោះខ្វល់ខ្វាយអំពីបញ្ហាលុយកាក់។",
        "ខ្ញុំខឹងណាស់ព្រោះគេបានកុហកខ្ញុំ។",
    ]

    for text in test_texts:
        label, score, all_scores = predict(text)
        print(f"Text:      {text}")
        print(f"Predicted: {label}  (confidence: {score:.3f})")
        print(f"All scores: {all_scores}")
        print()

Expected output shape (label names depend on what was detected in your dataset at training time — check id2label in phonsobon/xlmr-khmer-emotion's config.json on the Hub):

Text:      ខ្ញុំសប្បាយចិត្តណាស់ថ្ងៃនេះបានជួបជុំគ្រួសារ។
Predicted: happy  (confidence: 0.94)
All scores: [{'label': 'happy', 'score': 0.94}, {'label': 'neutral', 'score': 0.03}, ...]

Batch testing on held-out data (also pulled from the Hub)

For a proper classification report (precision/recall/F1 per class) rather than one-off predictions — this is the more meaningful check given the accuracy caveat above:

from sklearn.metrics import classification_report
from datasets import load_dataset

# sample rows from the dataset that weren't necessarily in your exact train split -
# for a truly clean check, prefer text you write yourself over dataset rows
test_ds = load_dataset("phonsobon/Text_Emotional_kh", split="train").shuffle(seed=123).select(range(1000))

y_true, y_pred = [], []
for row in test_ds:
    label, _, _ = predict(row["text"])
    y_true.append(row["label"])
    y_pred.append(label)

print(classification_report(y_true, y_pred))

Notes

• Segmenting the full 4.19M-row dataset with khmer-nltk takes a while even with multiprocessing — the notebook caches the segmented dataset to Google Drive after the first run so you don't pay that cost twice.
• Training checkpoints save to Google Drive and auto-resume if the Colab runtime disconnects — just re-run the notebook.
• Use xlm-roberta-large instead of xlm-roberta-base for higher accuracy if you have the GPU memory/time budget for it.
• The first call to pipeline(..., model=HF_MODEL_ID) downloads and caches the model locally (usually to ~/.cache/huggingface); subsequent runs are fast since they use the cache.