NAMAA-BAREC2026-Sent-AraELECTRA-OrdThr

Formerly NAMAA-Space/barec-araelectra_ord.

Fine-tuned checkpoint from NAMAA Community for the BAREC Shared Task 2026 (sentence-level, Strict Track): predicting one of 19 ordered readability levels of an Arabic sentence.

This is one of the controlled experiments behind the paper NAMAA Community at the BAREC Shared Task 2026: Ordinal-Aware Training for Fine-Grained Arabic Readability Assessment Beyond Architectural Scale. It is not the paper's final submitted system (single EmbeddingGemma encoder with Gaussian soft labels + KL loss, 84.0 QWK test / 82.4 QWK blind test), which is released separately.

Model details

Base model	aubmindlab/araelectra-base-discriminator
Head / objective	Ordinal cumulative-threshold head (18 logits = 19 levels − 1)
Decoding	Level = number of thresholds whose sigmoid exceeds 0.5.
Labels	19 readability levels (BAREC scale)
Training data	BAREC training split only (Strict Track, no external resources)
Trained by	Fatimah Emad Eldin (Trouve Labs / NAMAA Community)

Ordinal head with 18 thresholds over the ELECTRA encoder, max length 512 (from barec_meta.json).

Evaluation

Official metric: Quadratic Weighted Kappa (QWK), also accuracy, ±1 accuracy, MAE and the 7/5/3-level mappings. Per-checkpoint scores for this specific model were not recorded when it was uploaded; see the paper for the evaluated systems and the final results. Do not assume this checkpoint matches any number in the paper.

Usage

This checkpoint has a custom ordinal head that AutoModelForSequenceClassification does not load. Load the encoder from the bert.* weights in model.safetensors and apply the head tensors (classifier.weight/bias) manually, then decode as described above. The decoding and label offset are inferred from the checkpoint structure; verify them against your training code.

Input text: models whose base is a CAMeL d3tok readability model were developed on the BAREC d3tok (morphologically tokenized) column; the AraBERTv2 / AraELECTRA models expect the preprocessing of their base model. Check this against the training setup before comparing scores.

Limitations

Trained on BAREC sentences only; behaviour on other genres or dialects is untested. Adjacent-level confusions are expected, which is why QWK is the primary metric. License: inherited from the base model and the BAREC data terms; check both before reuse.

Citation

@inproceedings{sibaee2026namaa,
  title     = {{NAMAA} Community at the {BAREC} Shared Task 2026: Ordinal-Aware Training for Fine-Grained {A}rabic Readability Assessment Beyond Architectural Scale},
  author    = {Sibaee, Serry and Al Jallad, Khloud and Elhosiny, Israa and Eldin, Fatimah Emad and Nacar, Omer},
  booktitle = {Proceedings of the Fourth Arabic Natural Language Processing Conference: Shared Tasks},
  year      = {2026},
  address   = {Budapest, Hungary},
  publisher = {Association for Computational Linguistics}
}

@inproceedings{elmadani2026barecst,
  title     = {{BAREC-ST-2026}: The Second Shared Task on {A}rabic Readability Assessment},
  author    = {Elmadani, Khalid N. and Alhafni, Bashar and Taha, Hanada and Habash, Nizar},
  booktitle = {Proceedings of the Fourth Arabic Natural Language Processing Conference},
  year      = {2026},
  address   = {Budapest, Hungary},
  publisher = {Association for Computational Linguistics}
}

@inproceedings{elmadani2025barec,
  title     = {A Large and Balanced Corpus for Fine-grained {A}rabic Readability Assessment},
  author    = {Elmadani, Khalid N. and Habash, Nizar and Taha-Thomure, Hanada},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2025},
  pages     = {16376--16400},
  year      = {2025},
  address   = {Vienna, Austria},
  publisher = {Association for Computational Linguistics}
}

Acknowledgments

We thank Tuwaiq Academy for its continued support. Authors: Serry Sibaee (Prince Sultan University), Khloud Al Jallad (Arab International University), Israa Elhosiny (Alexandria University), Fatimah Emad Eldin (Trouve Labs), Omer Nacar (Tuwaiq Academy), NAMAA Community.