MahdiAlikhah/pira-persian-summary

Pira (پیرا) - Abstractive Persian Text Summarization

Pira is a lightweight, highly efficient abstractive text summarization model specifically fine-tuned for the Persian language.

The Meaning Behind the Name

Pira (پیرا) is derived from the Persian infinitive Pirastan (پیراستن), which means to prune, trim, or purify. Just as a gardener prunes a tree to reveal its core structure and encourage healthy growth, this model is designed to strip away verbose journalistic fluff, rhetorical padding, and excess details to extract the pure, core message of a Persian text.

Model Details

• Model Type: Sequence-to-Sequence (Seq2Seq)
• Base Architecture: google/mt5-small (~300M parameters)
• Language: Persian / Farsi (fa)
• Task: Abstractive Text Summarization
• Developer: Mahdi Alikhah

Training Data

To prevent the model from learning lazy "extractive" habits (simply copying the first paragraph), Pira was trained on a meticulously blended and balanced corpus of ~96,000 curated rows:

1. csebuetnlp/xlsum (Persian Split): ~47k rows of BBC Persian articles. This provided the foundation for high-quality, professional, and purely abstractive journalistic summaries.
2. RohanAiLab/persian_daily_news (Sampled): 50k rows downsampled from a massive 2M-row dataset of domestic Iranian news (ISNA, Tasnim, etc.) to ensure deep coverage of local political, economic, and social vocabulary without overpowering the abstractive traits of XL-Sum.
3. msdos34/persian-summary-dataset: ~1k rows formatted as Instruction-Tuning data, helping the model learn prompt responsiveness.

Training Environment & Hardware

The model was fine-tuned to completion with optimal efficiency using modern hardware acceleration.

• Hardware: Single NVIDIA L4 GPU (Google Colab environment)
• Compute Precision: bfloat16 (bf16) native mixed precision, preventing NaN gradient explosions and significantly accelerating training time.
• Epochs: 3
• Total Training Steps: 18,066
• Batch Size: 16 (per device)
• Learning Rate: 5e-4
• Optimizer: AdamW

Evaluation Results

The training run demonstrated textbook convergence with zero signs of overfitting:

• Final Training Loss: 1.895
• Final Validation Loss: 1.758

Note on Loss: A validation loss in the 1.7 range for generative tasks indicates strong abstractive capabilities. Extractive models often achieve lower loss by memorizing exact phrases, whereas Pira learned to rephrase and synthesize, resulting in a slightly higher but healthier abstractive generation loss.

Intended Use & Limitations

• Best For: Formal news articles, corporate press releases, political dispatches, and standard non-fiction prose.
• Limitations: Because the training data heavily indexes on formal journalism (Farsi-e-Ketabi), the model may struggle with highly informal social media text, slang, or deep technical/medical documents. It is also limited by the mT5 context window (max 512 tokens), meaning very long documents should be chunked before summarization.

How to Use

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

model_name = "MahdiAlikhah/pira-persian-summary"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)

text = """
تلسکوپ فضایی جیمز وب موفق به ثبت تصاویر جدید و خیره‌کننده‌ای از یک کهکشان دوردست در اعماق کیهان شده است.
به گفته دانشمندان ناسا، این کهکشان که در فاصله بیش از ۱۳ میلیارد سال نوری از زمین قرار دارد، اطلاعات جدیدی درباره چگونگی شکل‌گیری نخستین ستارگان پس از انفجار بزرگ ارائه می‌دهد.
پژوهشگران امیدوارند با تحلیل داده‌های طیف‌سنجی این تلسکوپ، به ساختار شیمیایی و گازهای موجود در این کهکشان پی ببرند.
"""

inputs = tokenizer(
    text,
    return_tensors="pt",
    max_length=512,
    truncation=True
)

outputs = model.generate(
    inputs["input_ids"],
    max_new_tokens=128,
    min_new_tokens=20,
    num_beams=8,
    repetition_penalty=2.5,
    length_penalty=1.2,
    early_stopping=True
)

summary = tokenizer.decode(
    outputs[0],
    skip_special_tokens=True
)

print(summary)