klohdfhk/amharic-news-classifier

Amharic News Classifier

A six-way Amharic news-topic classifier fine-tuned from AfroXLMR-mini. This repository contains the model and inference code, not source articles or dataset rows.

Source code: Public GitHub repository

Labels

• business — Business and economy (ቢዝነስ)
• entertainment — Entertainment and culture (መዝናኛ)
• national_news — National and domestic news (ሀገር አቀፍ ዜና)
• politics — Politics and government (ፖለቲካ)
• sports — Sports (ስፖርት)
• world_news — International news (ዓለም አቀፍ ዜና)

Evaluation

The source training partition contains 31,949 cleaned training examples. For bounded, reproducible CPU fitting, a fixed-seed sample of 7,899 examples (up to 1,500 per category) was used for both the baselines and transformer. The full 7,988-example validation set was used for model selection. Each candidate was evaluated once on the full 10,179-example held-out test partition after validation-based selection; test metrics did not determine the released checkpoint.

Method	Validation macro-F1	Held-out test macro-F1
majority	0.024	0.024
word_tfidf_linear_svc	0.768	0.774
char_tfidf_linear_svc	0.773	0.781
word_char_tfidf_linear_svc	0.786	0.784
AfroXLMR-mini	0.566	0.581

Transformer test accuracy: 0.555; macro-F1: 0.581 (2,000-resample 95% row-bootstrap interval 0.569–0.593); weighted-F1: 0.496; balanced accuracy: 0.732. The test set has 10,179 examples. The interval reflects row resampling of this fixed trained model, not variability across seeds.

The best validation system overall was combined word/character TF–IDF + LinearSVC. The best classical baseline scored test macro-F1 0.784, versus 0.581 for this transformer. This transformer is an experimental benchmark artifact, not the preferred classifier on this dataset. A hosted Space has not been published: free ZeroGPU creation was rejected with HTTP 402, and no converted browser weights are being distributed while source-article rights remain unresolved.

See evaluation_metrics.json, data_audit.json, and error_analysis.md for per-class scores, confusion counts, split audit, and aggregate-only error patterns. No error example includes article text.

Training

• Base model: Davlan/afro-xlmr-mini, revision bc04038b969667884bd83cdd37bed9559c2c3d9c (Hub metadata declares MIT).
• Dataset: israel/Amharic-News-Text-classification-Dataset, revision b646090ef0d09981da9c9765c4d376b407aa5955; its Hub card declares CC BY 4.0 and cites Azime & Mohammed (2021).
• Seed: 42; selected epoch: 2; validation macro-F1: 0.566.
• The classical comparison uses majority, word-TF–IDF LinearSVC, character-TF–IDF LinearSVC, and combined word/character TF–IDF LinearSVC; linear SVM regularization is fixed at C=1.0.
• Maximum input length: 192 subword tokens, chosen from the training-only length profile; only headline and article body are inputs.
• 87.5% of the sampled training texts exceeded the selected input length and were truncated.
• Class-balanced cross-entropy, AdamW, learning rate 2e-05, batch 8 with gradient accumulation 2.

Preprocessing and data quality

The pipeline NFC-normalizes text, removes zero-width controls, collapses whitespace, excludes empty labels and records with less than 10% Ethiopic letters, drops exact text records with conflicting labels, collapses same-label exact duplicates within each partition, and removes exact training records that overlap test. In this run, 41,186 training and 10,297 test rows were supplied; 258 exact train/test overlaps were removed from training. The final full train/validation/test counts are 31,949 / 7,988 / 10,179.

A 10% Ethiopic-script share is a heuristic, not a language identifier; code-switching and English items with some Ethiopic text may remain. Dataset source/article copyright has not been independently cleared beyond the CC BY 4.0 declaration on the Hub dataset card. The raw corpus is not redistributed here.

Intended use and limitations

For research, education, and exploratory topic triage only. It is not a substitute for editorial judgment and should not be used for high-impact decisions, political targeting, or decisions about people. On this test split, national-news recall was 10.4% (error rate 89.6%), and the transformer substantially trailed the TF–IDF baselines. News topics can overlap, language and editorial conventions change over time, and a random split may not reveal source or time leakage. The entertainment class is much smaller than national news; macro-F1 and class-level metrics should be considered together. Long articles are truncated, and the maximum softmax score is not calibrated confidence.

Inference

from amharic_news.inference import NewsClassifier

classifier = NewsClassifier("klohdfhk/amharic-news-classifier")
print(classifier.predict("የዜና ርዕስ", "የዜናው ይዘት", top_k=3))

The release includes src/amharic_news/inference.py, labels.py, and text.py. Install PyTorch, Transformers, and the files from this repository; the example returns label names, descriptions, and raw softmax scores.

Attribution and license

Original project code is MIT licensed. The Hugging Face model repository declares CC BY 4.0 with attribution to the dataset authors based on the dataset repository's license statement; this does not independently establish the copyright chain for underlying publisher articles. The base model retains its MIT terms. This project's public artifacts contain no news article text. Dataset citation: Israel Abebe Azime and Nebil Mohammed, “An Amharic News Text classification Dataset” (2021).