BY YATHARTH.
Thursday 1.8 Lite
More Models: Thursday 1.1 Gate, Thursday 2.0 Lite
Thursday Training Pipeline Overview: Thursday is a lightweight, deterministic training workshop designed for small-scale language models in JAX, prioritizing resumable states and zero-drama infrastructure over complex distributed overhead. Core Design Principles: Deterministic Resumption — Every checkpoint captures exact optimizer states, step counts, epoch markers, and tokenizer vocabularies, ensuring zero loss of progress when interrupted. Pure Semantic Ingestion — Strips away noisy JSONL metadata keys to focus strictly on text payload boundaries. Stable Optimization — Utilizes optax schedules alongside custom cross-entropy stabilizers to prevent gradient divergence during lower-precision training runs.