irwan19/albertir-quran-hadith

---

language: id license: mit base_model: indobenchmark/indobert-base-p1 pipeline_tag: fill-mask tags: - bert - indonesian - islamic - quran - hadith - masked-language-modeling - fill-mask library_name: transformers

ALBERTIR: BERT-Based Pretraining for Indonesian Religious Texts

Model Description

ALBERTIR is a domain-specific Indonesian language model pretrained on Islamic texts (Quran and Hadith translations). It is built upon the indobenchmark/indobert-base-p1 architecture and further pretrained using Masked Language Modeling (MLM) on an Islamic corpus.

This model is designed for:

• Semantic alignment of Indonesian Islamic texts
• Theological accuracy evaluation
• Indonesian Friday sermon (Khutbah) analysis
• Islamic content understanding and classification

Developed by

• Irwan Darmawan – Universitas Negeri Malang, Indonesia
• Hakkun Elmunsyah – Universitas Negeri Malang, Indonesia
• Didik Dwi Prasetya – Universitas Negeri Malang, Indonesia
• Wahyu Mulyo Utomo – Universiti Tun Hussein Onn Malaysia, Malaysia

Model Details

Parameter	Value
Model Type	BERT (Bidirectional Encoder Representations from Transformers)
Base Model	indobenchmark/indobert-base-p1
Task	Masked Language Modeling (MLM)
Language	Indonesian
Domain	Islamic / Religious Texts
Parameters	124,492,880
Tensor Type	F32

Pretraining Details

Parameter	Value
Corpus	Quran (6,236 verses) + Sahih Bukhari (7,008) + Sahih Muslim (5,362)
Total Documents	18,606
Epochs	10
Batch Size	8
Learning Rate	5e-5
Optimizer	AdamW
Total Steps	10,470
Final Training Loss	~1.2473
Final Validation Loss	~1.1415

Training Data

The model was trained on:

• Quran: Indonesian translation (Ministry of Religious Affairs)
• Sahih Bukhari: Indonesian translation
• Sahih Muslim: Indonesian translation

Total: 18,606 verses/hadith documents

Intended Uses

• Fill-mask tasks on Indonesian Islamic texts
• Semantic similarity in religious domain
• Theological content classification
• Friday sermon analysis

How to Use

from transformers import AutoModelForMaskedLM, AutoTokenizer

model_id = "irwan19/albertir-quran-hadith"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForMaskedLM.from_pretrained(model_id)

# Example: Fill-mask prediction
text = "Umat Islam wajib melaksanakan ibadah [MASK] di bulan Ramadhan."
inputs = tokenizer(text, return_tensors="pt")
outputs = model(**inputs)
# Prediksi: 'puasa'