i4ds/2026_07_30_1038__t5_base_finetune__full_data_de2ch

t5-base — German → Swiss German (de2ch)

A google-t5/t5-base model fine-tuned to translate Standard (High) German → Swiss German dialect text. Trained on the full i4ds de→ch corpus (~1.1 M sentence pairs across 8 sources) with dialect-aware prompting.

This is the best-aggregate T5 model in our held-out benchmark (see Results): it edges the sweep on the aggregate metric and has the best T5 nzz (sentence) score.

Usage

The model uses a plain T5 instruction prefix. Build the prompt exactly as at training time and generate with beam search.

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

repo = "i4ds/2026_07_30_1038__t5_base_finetune__full_data_de2ch"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSeq2SeqLM.from_pretrained(repo)

def translate(text, dialect=None):
    # Generic Swiss German:
    prompt = f"translate German to Swiss German: {text}"
    # Or target a canton (dialect names below):
    if dialect:
        prompt = f"translate German to {dialect} Swiss German: {text}"
    enc = tok(prompt, return_tensors="pt", truncation=True, max_length=256)
    out = model.generate(**enc, num_beams=4, max_new_tokens=128, early_stopping=True)
    return tok.decode(out[0], skip_special_tokens=True)

print(translate("Ich gehe morgen in die Stadt."))
print(translate("Ich gehe morgen in die Stadt.", dialect="Zurich"))

Dialect names (map canton code → prompt word): zh→Zurich, be→Bern, gr→Graubunden, vs→Valais, sg→St. Gallen, lu→Lucerne, ag→Aargau, bs→Basel. A generic/unknown dialect falls back to translate German to Swiss German: ….

The tokenizer was extended with the umlaut characters Ä Ö Ü ä ö ü ß as dedicated tokens (Swiss-German spelling keeps ä/ö/ü, e.g. Ä ≠ Ae), so no external normalization is needed.

Training

• Base: google-t5/t5-base (tokenizer extended with Ä Ö Ü ä ö ü ß).
• Data: full de→ch corpus (~1.1 M pairs) from stt4sg-350, swissdial_corr, wikipedia, technorama, soundolino, liza_ch, swisscrawl, idiotikon (all under the i4ds org). Duplicates and empty rows dropped.
• Prompt: translate German to [<dialect>] Swiss German: <text> (dialect in prompt).
• Source normalization: probabilistic — 50 % of examples get a light source-side normalization (abbreviation + unit lookup, num→words); target left untouched. Validation is kept un-normalized for comparability.
• Objective / regime: seq2seq cross-entropy, label smoothing 0.1, bf16.
• Schedule: 40 epochs, effective batch 256 (per-device 32 × grad-accum 8), LR 1e-3 cosine decay, 500 warmup steps, weight decay 0.01, max_seq_length 256.
• Inference defaults: 4 beams, max_new_tokens 128.
• Hardware: single NVIDIA A100 (40 GB).

Evaluation

Held-out normalized WER (lower is better; lowercase + alphanumeric, jiwer), plus BLEU / chrF (sacrebleu). Three held-out sets: wikipedia_test (per-dialect, capped 500/dialect), chantal_sg (230 St. Gallen sentences), nzz (one long Zurich article, scored document-level over source-split strategies).

Headline (WER)

set	WER	BLEU	chrF
wikipedia_test (overall)	0.347	45.9	76.6
chantal_sg (SG)	0.298	53.1	81.1
nzz (sentence split, best)	0.280	48.6	83.2

Aggregate = mean(wiki, chantal, nzz-best) = 0.308 — top of our T5 sweep.

wikipedia_test — per-dialect WER

ag	be	bs	ch	lu	sg	vs	zh
0.422	0.396	0.292	0.392	0.556	0.323	0.322	0.254

zh is easiest (0.254), lu hardest (0.556) — consistent across the whole sweep.

nzz — document-level WER by source-split strategy

sentence	nsent:2	paragraph	char:500
0.280	0.320	0.382	0.458

T5 is sentence-trained, so keep evaluation/inference chunks short — WER climbs on longer chunks. Split long documents into sentences before translating.

Limitations

• Sentence-level model: quality degrades on long multi-sentence chunks (split first).
• Dialect coverage is uneven (strong zh/bs, weaker lu); rare cantons rely on the generic Swiss-German fallback.
• Swiss German has no fixed orthography; outputs reflect the training corpora's spelling conventions, not a single standard.