t5-title-model

Small T5 (google/t5-efficient-small) trained from scratch for chat sidebar title generation: one English user message in, a short 2-7 word title out.

Usage

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tok = AutoTokenizer.from_pretrained("viaang/t5-title-model")
model = AutoModelForSeq2SeqLM.from_pretrained("viaang/t5-title-model")

msg = "my nginx keeps returning 502 bad gateway"
enc = tok("User: " + msg + chr(10) + "Title: ", return_tensors="pt")
out = model.generate(**enc, max_length=32, num_beams=4)
print(tok.decode(out[0], skip_special_tokens=True))
# -> Nginx 502 Bad Gateway

Prompt format is exactly User: {message}\nTitle: (note the trailing space). Beam search (num_beams=4) is what the eval used.

dtype note (important)

The residual stream of this checkpoint reaches ~82k at the last encoder block, which overflows fp16 max (65504) and produces nan. Train/eval it in bf16, never fp16. fp32 also works. The stored weights are fp16/bfloat16-compatible safetensors.

Training

• base: google/t5-efficient-small (60.5M params, tied embeddings)
• data: 584,298-row blend (v8), 19.2% short-text rows (10k short seeds)
• 2 epochs, 2xT4, bf16, batch 16 x grad-accum 4
• best eval_loss 2.7184 (Trainer-reported; inflated ~1.79x by grad-accum reporting, true eval loss ~1.52)

Known limits

Measured on a 276-row failure-cluster probe set (beam-4, deterministic): 179/276 distinct titles (64.9%), 29.0% of outputs containing "acknowledg".

Solved (0% acknowledgment, all distinct): tech and technical proper nouns (Nginx 502 Bad Gateway, Kubernetes CrashLoopBackOff), layoff/firing (Layoff Notice, Contract Termination Notice - no birth collapse), temporal, safety, emotional, long-message abstraction.

Still weak:

• Emoji collapse (worst issue). All 12 emoji inputs map to Sadness Acknowledgment, including 🔥💯, 👍, 🎉, ❤️. Wrong label for positive emoji.
• Short / boundary / low-info inputs are 63-80% acknowledgment labels (cool -> Casual Acknowledgment).
• Abbreviation coverage is uneven: g2g -> Got To Go and afk -> Away From Game are right, but wtf/tldr both -> Got To Go and ikr -> Hello Acknowledgment.

Files

• model.safetensors - fp16 weights
• config.json, generation_config.json, tokenizer.json, tokenizer_config.json

An int4 QaT export is planned (groupwise symmetric int4 g64, fp16 embeddings).