Tishko/sorani-gec

Sorani GEC — Agreement-Aware Grammatical Error Correction for Central Kurdish

This repository contains the trained model checkpoints for the MSc thesis:

Agreement-Aware Grammatical Error Correction for Central Kurdish (Sorani):
A Morphology-Driven Neural Approach
Tishko Salah Hawez · University of Kurdistan Hewlêr · 2026
Supervisor: Dr. Hossein Hassani

Code and evaluation results: github.com/6ebeng/sorani-gec

---

Models

Two variants, both fine-tuned from google/byt5-small on the 5,253-pair splits_v2 synthetic Central Kurdish (Sorani) GEC training set.

Variant	Description
baseline	Byte-level seq2seq; no linguistic features
morphaware	Same backbone + 9 morphological features per word, a 33-edge agreement graph, and an auxiliary agreement-prediction loss

Each variant ships with three seeds (42, 123, 777) from the multiseed training campaign on the 5,253-pair splits_v2 training set (span F₀.₅ ≈ 0.165–0.177).

The thesis headline numbers (span F₀.₅ 0.5057 baseline / 0.5105 morphology-aware) come from a later clean campaign trained on 26,841 pairs; those checkpoints are not yet published here.

Results (multiseed campaign checkpoints in this repo, span F₀.₅)

Model	F₀.₅	Precision	Recall
Baseline (3-seed mean)	0.165	—	—
Morphology-aware (3-seed mean)	0.177	—	—

Paired bootstrap p = 0.08 (not significant at α = 0.05).

Full results, ablations, human evaluation (37 native raters), and discussion are in the thesis and in campaign_2_multiseed/eval_summary.json (word-level metrics).

File layout

campaign_2_multiseed/
  baseline_seed42/best_model.pt
  baseline_seed123/best_model.pt
  baseline_seed777/best_model.pt
  morphaware_seed42/best_model.pt
  morphaware_seed123/best_model.pt
  morphaware_seed777/best_model.pt
  eval_summary.json

Quick usage

from transformers import AutoTokenizer, T5ForConditionalGeneration
import torch

model_path = "Tishko/sorani-gec"  # or a local path to best_model.pt

# The checkpoints are raw PyTorch state-dicts saved with torch.save().
# Load with the ByT5-small tokenizer:
tokenizer = AutoTokenizer.from_pretrained("google/byt5-small")

model = T5ForConditionalGeneration.from_pretrained("google/byt5-small")
state = torch.load("campaign_2_multiseed/baseline_seed42/best_model.pt", map_location="cpu")
# State dict may be nested under a key — unwrap if needed:
sd = state.get("model_state_dict", state)
model.load_state_dict(sd, strict=False)
model.eval()

sentence = "کوڕەکە دەڕۆن"   # corrupted: singular subject, plural verb
inputs = tokenizer(sentence, return_tensors="pt")
with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=64)
print(tokenizer.decode(out[0], skip_special_tokens=True))
# → کوڕەکە دەڕوا  (corrected: singular verb)

Citation

@mastersthesis{hawez2026soranigec,
  author  = {Tishko Salah Hawez},
  title   = {Agreement-Aware Grammatical Error Correction for Central Kurdish
             (Sorani): A Morphology-Driven Neural Approach},
  school  = {University of Kurdistan Hewl\^{e}r},
  year    = {2026},
  type    = {MSc Thesis},
}