IslamicBank-HCDi-RoBERTa

Model description

IslamicBank-HCDi-RoBERTa is a RoBERTa-based NLP transformer model fine-tuned to classify dimensions of human capital disclosure in Islamic banking. The model identifies whether a text segment is substantively related to human capital and assigns it to one of six human capital disclosure dimensions or to a separate Not related class.

The model is based on FacebookAI/roberta-base and was fine-tuned using a supervised seven-class text classification design. It is intended for research on human capital disclosure, Islamic banking, sustainability reporting, corporate disclosure, workforce disclosure and computational text analysis.

---

Labels and Training data

The model was fine-tuned on a labelled text classification dataset developed from Islamic banking disclosure text. Candidate text segments were extracted using a human capital lexicon and then classified into human capital disclosure dimensions.

The final labelled dataset contained 13,218 observations across seven classes:

Label ID	Class	Number of observations
0	Not related	2,500
1	Diversity, Equity, and Inclusion	1,106
2	Compensation and Benefits	2,000
3	Labour Relations and Organisational Culture	2,000
4	Health, Safety, and Well-being	1,731
5	Workforce Demographics	1,881
6	Training, Development, and Human Capital Investment	2,000

The dataset was split using a stratified 80:10:10 train-validation-test design:

Split	Number of observations
Training	10,574
Validation	1,322
Test	1,322

---

Training procedure

The model was fine-tuned using the Hugging Face Transformers library.

Parameter	Value
Base model	FacebookAI/roberta-base
Task	Supervised multi-class text classification
Number of labels	7
Maximum sequence length	128
Epochs	5
Learning rate	2e-5
Weight decay	0.01
Loss function	Class-weighted cross-entropy
Train-validation-test split	80:10:10
Selection metric	Validation macro-F1
Hardware	NVIDIA Tesla T4 GPU

Class-weighted cross-entropy loss was used to address class imbalance.

Evaluation results

The selected RoBERTa model achieved the following performance:

Metric	Validation	Test
Macro-F1	0.895	0.890
Weighted-F1	0.891	0.888
Accuracy	0.891	0.888

Class-wise test performance

Label ID	Class	Precision	Recall	F1-score	Support
0	Not related	0.888	0.852	0.869	250
1	Diversity, Equity, and Inclusion	0.917	0.901	0.909	111
2	Compensation and Benefits	0.914	0.955	0.934	200
3	Labour Relations and Organisational Culture	0.841	0.795	0.817	200
4	Health, Safety, and Well-being	0.925	0.925	0.925	173
5	Workforce Demographics	0.881	0.904	0.892	188
6	Training, Development, and Human Capital Investment	0.866	0.905	0.885	200

Model comparison

Four transformer models were fine-tuned and compared under the same experimental setup.

Model	Validation Macro-F1	Test Macro-F1	Test Accuracy
RoBERTa base	0.895	0.890	0.888
BERT base	0.894	0.883	0.881
DeBERTa V3 base	0.855	0.848	0.845
FinBERT pretrain	0.841	0.852	0.849

RoBERTa base was selected because it achieved the highest validation macro-F1 and strongest overall test performance.

---

GitHub repository and related model

Pipeline notebooks, data preparation files, outputs and reproducible experiments are available on GitHub:

GitHub: bilalezafar/IslamicBank-HCDi-RoBERTa

A related binary human capital classifier is also available on Hugging Face. This model classifies text into two labels: HC and Non-HC.

Related model: bilalzafar/HC-BERT-IFIs

---

Citation

If you use this model, please cite:

@misc{zafar2026_islamicbank_hcdi_roberta,
  title        = {IslamicBank-HCDi-RoBERTa: A Transformer Model for Classifying Dimensions of Human Capital Disclosure in Islamic Banking},
  author       = {Zafar, Muhammad Bilal},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/bilalzafar/IslamicBank-HCDi-RoBERTa}}
}

---

How to use

from transformers import AutoTokenizer, AutoModelForSequenceClassification, pipeline

model_id = "bilalzafar/IslamicBank-HCDi-RoBERTa"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)

classifier = pipeline(
    "text-classification",
    model=model,
    tokenizer=tokenizer,
    top_k=None,
    truncation=True,
    max_length=128
)

text = "The Bank provides regular training programmes to enhance employees' professional and Shariah-related competencies."

predictions = classifier(text)

best_prediction = max(predictions[0], key=lambda x: x["score"])

print(f"Dimension: {best_prediction['label']} | Prediction score: {best_prediction['score']:.4f}")

# Output {Dimension: Training, Development, and Human Capital Investment | Prediction score: 0.9986}