Hodfa71/gemma4-e4b-da-saga-delta-dpo

Gemma4-E4B — Danish Grammar-Aligned (SAGA Δ-DPO, no SFT)

Fine-tuned with SAGA (Syntax-Aware Grammar Alignment). Danish base PS (85.5%) is above τ=0.80 so SFT is skipped; Δ-DPO is applied directly from base.

This is a LoRA adapter. Load it on top of google/gemma-4-E4B-it.

Results (Stanza DA — held-out evaluator)

Metric	Base	+ Δ-DPO
Stanza PS ↑	85.5%	100.0%
Stanza score ↑	0.427	0.658
PPL-Wiki ↓	12.2	13.1
Summarisation PS ↑	27.5%	100.0%

+14.5pp parse success; 100% summarisation PS confirms strong Danish grammar transfer.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("google/gemma-4-E4B-it", torch_dtype="auto")
model = PeftModel.from_pretrained(base, "Hodfa71/gemma4-e4b-da-saga-delta-dpo")
tokenizer = AutoTokenizer.from_pretrained("google/gemma-4-E4B-it")

prompt = "Dansk er"
inputs = tokenizer(prompt, return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=60, temperature=0.8, do_sample=True)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Training details

• Base model: Gemma 4 E4B (base DA PS 85.5% ≥ τ=0.80 → SFT skipped)
• Stage: Δ-DPO directly from base — N=8 candidates, δ≥0.25, β=0.1
• Oracle: SpaCy da_core_news_lg (Danish dependency parser)
• LoRA: rank 16, α=32, all linear layers, bfloat16

Citation

@article{fakhar2025saga,
  title={SAGA: Syntax-Aware Grammar Alignment for Low-Resource Nordic Languages},
  author={Fakhar, Hoda and others},
  year={2025},
  note={Under review}
}

License

Apache-2.0 (Gemma 4 license).