DariBERT

A Dari-focused fine-tuned version of ParsBERT.

DariBERT is an experimental fine-tuned version of HooshvareLab/bert-base-parsbert-uncased, adapted toward Dari vocabulary and usage through fine-tuning on a manually curated Dari–Persian vocabulary dataset.

The goal of the project is to explore whether an existing Persian-language BERT model can be adapted toward Dari-specific vocabulary and linguistic usage using a relatively small, manually curated dataset.

DariBERT v1.0 is an experimental research and learning project. It should not be considered a fully trained Dari language model or a replacement for a large-scale pretrained Dari NLP model.

---

Model Details

Model Description

Property	Value
Model name	DariBERT
Version	v1.0
Architecture	BERT / Masked Language Model
Base model	HooshvareLab/bert-base-parsbert-uncased
Primary task	Masked Language Modeling
Pipeline	fill-mask
Language focus	Dari
Fine-tuning dataset	211 manually curated Dari–Persian vocabulary entries
Unique Dari entries	207
Unique Persian entries	204
License	Apache-2.0

DariBERT retains the underlying ParsBERT architecture and tokenizer and is fine-tuned rather than trained from scratch.

---

Motivation

Dari and Persian share a large amount of vocabulary and linguistic structure, but Dari also contains vocabulary, expressions, pronunciations, and usage that can differ from contemporary Iranian Persian.

This project began as an exploration of whether a Persian-language BERT model could be nudged toward Dari usage by introducing a curated collection of Dari-specific vocabulary and corresponding Persian equivalents.

The project is intentionally small in scope.

Rather than attempting to train a new language model from scratch, DariBERT uses an existing Persian pretrained model as its foundation and investigates what can be achieved through targeted fine-tuning with a small amount of Dari-focused data.

---

Training Data

The v1.0 training dataset consists of 211 manually curated vocabulary entries, representing 207 unique Dari entries and 204 unique Persian equivalents.

The vocabulary was manually reviewed and organized into 24 broad categories:

• Basic Communication
• Household Items
• Family & Social
• Food & Cooking
• Fruits & Vegetables
• Animals
• Technology & Modern Terms
• Medical & Health
• Clothing & Appearance
• Administrative & Social
• Occupations & Trades
• Building & Architecture
• Agriculture & Farming
• Tools & Implements
• Nature & Geography
• Adjectives & Descriptors
• Verbs & Actions
• Time & Seasons
• Body Parts
• Emotions & States
• Money & Commerce
• Measurement Units
• Miscellaneous Important Words

The training vocabulary is provided separately as a CSV file:

dariBERT_v1_training_vocabulary.csv

Each entry contains:

dari,persian,english

The dataset is a manually curated research dataset and is not intended to
represent the full Dari language.

Training Procedure

DariBERT was fine-tuned from the ParsBERT base model using the Hugging Face
Transformers library.

The training objective was Masked Language Modeling (MLM).

The model was trained on the Dari entries from the curated vocabulary dataset,
with the corresponding text tokenized using the ParsBERT tokenizer.

Training configuration

The v1.0 training run used:

Base model: HooshvareLab/bert-base-parsbert-uncased
Training epochs: 3
Batch size: 1
Learning rate: 5e-5
Weight decay: 0.01
Maximum sequence length: 16 tokens
Training objective: Masked Language Modeling
Hardware: CPU
Mixed precision: disabled

The training was intentionally lightweight and designed as an experiment in
Dari-specific adaptation rather than large-scale pretraining.

Intended Use

DariBERT is intended for experimentation and research involving Dari NLP,
including:

Masked-token prediction
Vocabulary exploration
Dari/Persian lexical comparison
Experimentation with low-resource language adaptation
Educational NLP experimentation
Further fine-tuning for downstream Dari NLP tasks

For example, a user can provide a sentence containing a [MASK] token and ask
the model to predict likely replacements.

Usage

DariBERT can be loaded using the Hugging Face Transformers library.

Installation
pip install transformers torch
Masked-token prediction
from transformers import AutoTokenizer, AutoModelForMaskedLM, pipeline

model_id = "hahang/DariBERT"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForMaskedLM.from_pretrained(model_id)

fill_mask = pipeline(
    "fill-mask",
    model=model,
    tokenizer=tokenizer
)

text = "من دیروز به بازار رفتم و یک [MASK] خریدم."

results = fill_mask(text)

for result in results:
    print(
        result["sequence"],
        result["score"]
    )

Example output may look like:

من دیروز به بازار رفتم و یک لپتاپ خریدم.
من دیروز به بازار رفتم و یک تبلت خریدم.
من دیروز به بازار رفتم و یک ماشین خریدم.

The exact predictions and scores may vary depending on the Transformers
version and runtime environment.

Example
Input
این کتاب بسیار [MASK] است.
Example predictions

The fine-tuned model produced predictions including:

این کتاب بسیار خوب است.
این کتاب بسیار خوبی است.
این کتاب بسیار ارزشمند است.
این کتاب بسیار جالب است.
این کتاب بسیار مهم است.

Another test used:

هوا امروز خیلی [MASK] است.

The model produced predictions including:

هوا امروز خیلی گرم است.
هوا امروز خیلی سرد است.
هوا امروز خیلی خوب است.
هوا امروز خیلی خنک است.

These examples demonstrate that the model can perform masked-token prediction,
but they should not be interpreted as a formal benchmark.

Evaluation

DariBERT v1.0 was evaluated through qualitative inference tests rather than a
formal held-out benchmark.

Testing included manually constructed Dari sentences covering areas such as:

Everyday activities
Shopping
Descriptive language
Weather
Vocabulary usage

The model demonstrated meaningful masked-token predictions in several examples.

However, the small size of the fine-tuning dataset means that these results
should be considered preliminary.

No formal accuracy, F1, perplexity, or downstream-task benchmark is provided
for v1.0.

Future versions may include a dedicated held-out Dari evaluation dataset and
quantitative benchmarks.

Limitations

DariBERT v1.0 has significant limitations.

Small training dataset

The model was fine-tuned using only 211 manually curated vocabulary entries.
This is extremely small compared with the datasets normally used to train
modern language models.

Not trained from scratch

DariBERT is not an independently pretrained Dari language model. It is a
fine-tuned version of ParsBERT.

The underlying model therefore retains substantial characteristics of its
Persian-language pretraining.

Limited Dari coverage

The vocabulary included in v1.0 represents only a small portion of the Dari
language.

Many Dari words, expressions, grammatical constructions, dialectal variants,
and contextual usages are not represented.

No translation capability

DariBERT is not a Dari-to-English or English-to-Dari translation model.

It should not be used as a translation system.

Not an autoregressive text-generation model

DariBERT is a BERT-style masked language model.

Its primary intended operation is predicting masked tokens within text rather
than generating long passages autoregressively like GPT-style language models.

Limited evaluation

The current version has not been evaluated against a large, independent Dari
test set.

Relationship to ParsBERT

DariBERT is derived from:

HooshvareLab/bert-base-parsbert-uncased

ParsBERT is a Persian-language BERT model developed for Persian natural
language processing.

DariBERT does not replace or compete with ParsBERT. Instead, this project
explores a small-scale adaptation of ParsBERT toward Dari vocabulary and usage.

Please refer to the original ParsBERT model and its documentation for
information about the underlying architecture, pretraining data, and original
model development.

Dataset

The manually curated v1.0 vocabulary is included with the project as:

dariBERT_v1_training_vocabulary.csv

The dataset contains three columns:

dari
persian
english

The CSV is provided primarily for transparency and reproducibility.

Future Work

Possible future versions of DariBERT may include:

A substantially larger Dari vocabulary
More naturally occurring Dari sentences
Curated Dari text corpora
Improved coverage of Dari-specific grammar and syntax
Additional dialectal and regional vocabulary
A dedicated Dari evaluation dataset
Quantitative benchmarking
Comparison against the original ParsBERT model
Further fine-tuning for downstream NLP tasks
Investigation of Dari-specific tokenization
Exploration of larger pretrained multilingual or Persian-language models

The long-term goal is to investigate practical approaches for improving
open-source NLP resources for Dari.

Citation

If you use DariBERT in your research, experiments, or projects, please cite
this repository:

Ahang, Hamid. DariBERT: A Dari-focused fine-tuned version of ParsBERT.
2026.

The underlying ParsBERT model should also be appropriately acknowledged when
DariBERT is used.

License

DariBERT v1.0 is released under the Apache License 2.0.

See the Apache License 2.0 text for the complete terms and conditions.

This model is derived from the ParsBERT model and users should also review the
license and attribution requirements associated with the original model.

Acknowledgments

This project builds upon the open-source Persian NLP ecosystem and, in
particular, the ParsBERT model developed by HooshvareLab.

Special thanks to the developers and researchers who have contributed to
open-source Persian and Dari natural language processing resources.

Project Status

DariBERT v1.0 — Experimental / Initial Release

This release represents an initial exploration of adapting ParsBERT toward
Dari using a small manually curated vocabulary.

The project is intended to remain open for experimentation, evaluation, and
future improvement.