NAMAA-BAREC2026-Sent-CAMeL-AraBERTv2-d3tok-Reg-run1

Formerly NAMAA-Space/barec-msa-readability_reg.

Fine-tuned checkpoint from NAMAA Community for the BAREC Shared Task 2026 (sentence-level, Strict Track): predicting one of 19 ordered readability levels of an Arabic sentence.

This is one of the controlled experiments behind the paper NAMAA Community at the BAREC Shared Task 2026: Ordinal-Aware Training for Fine-Grained Arabic Readability Assessment Beyond Architectural Scale. It is not the paper's final submitted system (single EmbeddingGemma encoder with Gaussian soft labels + KL loss, 84.0 QWK test / 82.4 QWK blind test), which is released separately.

Model details

Base model	CAMeL-Lab/readability-arabertv2-d3tok-reg
Head / objective	Regression (1 output)
Decoding	Predict a continuous readability level; round and clip to the 19-level scale.
Labels	19 readability levels (BAREC scale)
Training data	BAREC training split only (Strict Track, no external resources)
Trained by	Fatimah Emad Eldin (Trouve Labs / NAMAA Community)

Evaluation

Official metric: Quadratic Weighted Kappa (QWK), also accuracy, ±1 accuracy, MAE and the 7/5/3-level mappings. Per-checkpoint scores for this specific model were not recorded when it was uploaded; see the paper for the evaluated systems and the final results. Do not assume this checkpoint matches any number in the paper.

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
tok = AutoTokenizer.from_pretrained('NAMAA-Space/NAMAA-BAREC2026-Sent-CAMeL-AraBERTv2-d3tok-Reg-run1')
model = AutoModelForSequenceClassification.from_pretrained('NAMAA-Space/NAMAA-BAREC2026-Sent-CAMeL-AraBERTv2-d3tok-Reg-run1').eval()
x = tok(text, return_tensors='pt', truncation=True, max_length=512)
with torch.no_grad(): out = model(**x).logits
level = out.squeeze().round().clamp(1, 19).item()  # check label offset against your training code

Input text: models whose base is a CAMeL d3tok readability model were developed on the BAREC d3tok (morphologically tokenized) column; the AraBERTv2 / AraELECTRA models expect the preprocessing of their base model. Check this against the training setup before comparing scores.

Limitations

Trained on BAREC sentences only; behaviour on other genres or dialects is untested. Adjacent-level confusions are expected, which is why QWK is the primary metric. License: inherited from the base model and the BAREC data terms; check both before reuse.

Citation

@inproceedings{sibaee2026namaa,
  title     = {{NAMAA} Community at the {BAREC} Shared Task 2026: Ordinal-Aware Training for Fine-Grained {A}rabic Readability Assessment Beyond Architectural Scale},
  author    = {Sibaee, Serry and Al Jallad, Khloud and Elhosiny, Israa and Eldin, Fatimah Emad and Nacar, Omer},
  booktitle = {Proceedings of the Fourth Arabic Natural Language Processing Conference: Shared Tasks},
  year      = {2026},
  address   = {Budapest, Hungary},
  publisher = {Association for Computational Linguistics}
}

@inproceedings{elmadani2026barecst,
  title     = {{BAREC-ST-2026}: The Second Shared Task on {A}rabic Readability Assessment},
  author    = {Elmadani, Khalid N. and Alhafni, Bashar and Taha, Hanada and Habash, Nizar},
  booktitle = {Proceedings of the Fourth Arabic Natural Language Processing Conference},
  year      = {2026},
  address   = {Budapest, Hungary},
  publisher = {Association for Computational Linguistics}
}

@inproceedings{elmadani2025barec,
  title     = {A Large and Balanced Corpus for Fine-grained {A}rabic Readability Assessment},
  author    = {Elmadani, Khalid N. and Habash, Nizar and Taha-Thomure, Hanada},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2025},
  pages     = {16376--16400},
  year      = {2025},
  address   = {Vienna, Austria},
  publisher = {Association for Computational Linguistics}
}

Acknowledgments

We thank Tuwaiq Academy for its continued support. Authors: Serry Sibaee (Prince Sultan University), Khloud Al Jallad (Arab International University), Israa Elhosiny (Alexandria University), Fatimah Emad Eldin (Trouve Labs), Omer Nacar (Tuwaiq Academy), NAMAA Community.