Persian Paraphrase Identification with ParsBERT
A Persian Natural Language Processing model for identifying whether two Persian sentences have the same meaning (paraphrase) or not.
Task
This model performs binary sequence classification on pairs of Persian texts.
- Class 0: Non-Paraphrase
- Class 1: Paraphrase
The model predicts whether the two sentences convey the same meaning.
Base Model
The base pretrained model used in this project is:
HooshvareLab/bert-base-parsbert-uncased
The model was fine-tuned for Persian paraphrase identification using a binary classification head.
Dataset
The model was trained and evaluated on the PARSINLU QQP dataset.
Dataset source:
https://github.com/persiannlp/parsinlu
The dataset contains pairs of Persian questions/sentences with paraphrase labels.
Model Architecture
Base architecture: BERT / ParsBERT
Task: Sequence Classification
Number of classes: 2
Labels
| ID | Label |
|---|---|
| 0 | Non-Paraphrase |
| 1 | Paraphrase |
Fine-Tuning Strategy
Part of the pretrained BERT model was frozen during fine-tuning.
Frozen
- Embedding layer
- Encoder layers 0–7
Trainable
- Encoder layers 8–11
- Classification head
This strategy allows the upper transformer layers and classification head to adapt to the paraphrase identification task while keeping the lower-level pretrained representations fixed.
Training Configuration
| Parameter | Value |
|---|---|
| Epochs | 5 |
| Learning Rate | 2e-5 |
| Train Batch Size | 16 |
| Evaluation Batch Size | 32 |
| Weight Decay | 0.01 |
| Maximum Sequence Length | 128 |
| Random Seed | 42 |
Evaluation Metrics
The following metrics were used:
- Accuracy
- Precision
- Recall
- F1 Score
Test Results
| Metric | Score |
|---|---|
| Accuracy | 0.7636 |
| Precision | 0.6934 |
| Recall | 0.8189 |
| F1 | 0.7510 |
Parameter Statistics
| Parameter Type | Number |
|---|---|
| Total Parameters | 162,842,882 |
| Trainable Parameters | 28,943,618 |
| Frozen Parameters | 133,899,264 |
Usage
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
model_id = "amir1381sa/persian-paraphrase-bert"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)
text1 = "امروز هوا خیلی خوب است."
text2 = "امروز وضعیت آب و هوا بسیار مناسب است."
inputs = tokenizer(
text1,
text2,
return_tensors="pt",
truncation=True,
max_length=128
)
with torch.no_grad():
outputs = model(**inputs)
prediction = torch.argmax(outputs.logits, dim=-1).item()
print(model.config.id2label[prediction])
Output
The model returns one of the following labels:
Non-Paraphrase
Paraphrase
Repository Contents
This repository contains:
- Fine-tuned ParsBERT model
- Tokenizer
- Model configuration
- Test evaluation results
- Training logs
- Experiment configuration
Project Summary
This project demonstrates fine-tuning of a pretrained Persian BERT model for the task of Persian Text Paraphrase Identification.
The pretrained model was adapted to a binary classification task using the PARSINLU QQP dataset.
Reproducibility
Experiments were performed using:
- Python
- PyTorch
- Hugging Face Transformers
- Hugging Face Datasets
- Scikit-learn
Random seed:
42
- Downloads last month
- -