openeurollm/Prelude-SFT-Excellent-v0.1

Prelude SFT — Excellent Instructions (Old)

Prelude fine-tuned on Excellent-SFT for two epochs. This checkpoint predates explicit thinking-instruction conditioning and can produce empty thinking blocks even when thinking is enabled.

Reproduction details

Item	Value
Training repository	OpenEuroLLM/post-training
Training code reference	Current main
Experiment details	Taskboard #416
Base model	birgermoell/oellm-9b-128k-theta32m-prelude-anneal300b
Base revision	c5b5a9fffdb8d09fe5071c5fa99028bdd76eb235
Training data	openeurollm/OpenEuroLLM-Excellent-SFT
Data revision	c4ee55804f256c5f8cc4a6ef3fdecbe4a4ea95e0
Tokenizer	openeurollm/tokenizer-256k
Tokenizer revision	94bae05cf351c4feb435d6a73c2aa8edfd88ae04
Chat template	Training-time Qwen-style template bundled with this checkpoint
Thinking instructions	No explicit /think or /no_think assignment
Training duration	2 epochs; 55,836 steps
Maximum sequence length	32,768 tokens; packing enabled
Supervision	Assistant-only loss
Partially truncated supervised responses	Retained with warnings (truncated_span_action: warn)
Optimizer	AdamW; β₁=0.9, β₂=0.95, ε=1e-8; weight decay=0
Learning rate	8e-5; linear decay; 3% warmup
Effective batch size	32 packed sequences; per-device batch size 1
Gradient clipping	1.0
Precision and kernels	BF16; FlashAttention 3; Liger enabled
Distributed training	DeepSpeed ZeRO-2; communication overlap disabled
Hardware	32 NVIDIA H100 GPUs across 8 nodes on MareNostrum 5 (MN5)
Gradient checkpointing	Enabled, non-reentrant
Seeds	Training and data: 42
Processed input tokens	58,309,862,656, across both epochs
Processed supervised targets	Not explicitly tracked, but tracked in a very similar run: Prelude-SFT-Excellent-Instructions

This repository contains inference weights and tokenizer files, not optimizer or resumable training state. Consult the base-model and dataset cards for applicable licenses; no separate checkpoint release license has been assigned.