Kthulhu-0.1-0.6B (580M) — Native Rust Bilingual (EN/RU) + Rust/GPU Code LLM

Kthulhu-0.1-0.6B (580M parameters) is a hybrid Transformer + System-1 Jev calibrated decision model trained 100% from scratch in pure native Rust using lgpt-rs (Burn 0.22 + CubeCL 0.11 + custom fused WMMA FlashAttention-2 with native GQA, chunked online Cross-Entropy, and FusedAdamW).

Model Architecture & Artifacts

Parameter	Value
Parameters	~580M (d_model = 1280, n_layer = 28, d_ff = 3584 SwiGLU)
Attention	Gated Attention (head_gate) + QK-Norm, 20 Q heads / 4 KV heads (5:1 GQA), head_dim = 64
Context Length (seq_len)	2,048 (RoPE theta = 1,000,000)
Vocabulary (V)	65,536 custom BPE (EN + RU + Rust/CUDA/WGPU + <think>...</think> + `<
System-1 Head (Jev)	Calibrated yes / no / null (noul) ternary classification + 1..5 rubric expectation + Shannon entropy
Included Weights	model.safetensors (BF16), Kthulhu-0.1-0.6B-q8_0.gguf (590 MB), Kthulhu-0.1-0.6B-bf16.gguf (1.1 GB)

Training Telemetry & 2K Convergence Dashboard

• Pretraining (12,500 steps / 1.64B tokens): Final EMA Loss 2.5926, Best Step Loss 2.2690 (PPL = 9.67), Sustained Throughput 41,285 tok/s on 1x NVIDIA RTX PRO 6000 Blackwell Server Edition (96 GB)
• Stage 2 Clean SFT (220 steps): Final SFT EMA Loss 1.8243 (PPL = 6.33), Sustained Throughput 41,515 tok/s

Full-Width 2K (2560 × 1080) Telemetry Panels

• Chronological 1-minute sample & Jev calibration evolution table: training_samples_evolution.md
• Raw telemetry logs: pretrain_telemetry.jsonl, sft_telemetry.jsonl, gpu_telemetry.jsonl