jiazhe868/nanogpt_seis_posttrain

🎯 nanoGPT-Seis-Posttrain (113M SFT & GRPO RL Checkpoints)

nanoGPT-Seis-Posttrain provides the instruction-tuned (SFT) and programmatically verified reinforcement learning (GRPO RL) model checkpoints for nanoGPT-Seis, a 113M-parameter domain-specific LLM trained on earthquake and seismology literature.

• GitHub Repository: jiazhe868/nanogpt-seis-posttrain
• Base Pretrained Model: jiazhe868/nanogpt_seis (822.7M tokens)

---

📦 Uploaded Checkpoints Breakdown

This repository contains the weights for both post-training phases:

Checkpoint Path	Phase	Description & Details
checkpoints/sft/sft_ckpt_best.pt	Phase II (SFT)	Full-parameter SFT model trained on 14,068 instruction-response pairs across 8 epochs. Teaches ChatML interface <|im_start|>/<|im_end|>, response termination, and step-by-step <think> reasoning formatting.
checkpoints/grpo/grpo_ckpt_step_100.pt	Phase III (GRPO RL)	Selected RL Model (Step 100). Fine-tuned using Group-Relative Policy Optimization with programmatic verifiers (FormatVerifier, NumericalVerifier, EchoVerifier). Achieves 97.8% <think> compliance.
checkpoints/grpo/grpo_ckpt_step_200.pt	Phase III (GRPO RL)	Final RL step (Step 200) checkpoint. Used for training dynamics analysis and over-training audit checks.

---

📐 Model Architecture Specs

Spec	Value
Total Parameters	113.3M total (100.7M non-embedding, weight-tied)
Attention Mechanism	Grouped-Query Attention (12 Query heads : 4 KV heads)
Position Embeddings	Rotary Position Embeddings (RoPE)
Activation & Norm	SwiGLU (2048 hidden width), pre-RMSNorm
Context Length	4,096 tokens
Vocabulary Size	16,388 (16,384 base BPE + 4 special tokens)

---

📊 Benchmark & Evaluation Summary

Evaluated on the frozen audit test set (345 held-out prompts, 4 rollouts per prompt):

Metric	Phase II (SFT)	Phase III (GRPO @100)	Phase III + RAG Context Grounding
<think> Compliance	73.1%	97.8%	97.8%
Closed-Book Numerical Accuracy (@5%)	10.6%	11.4%	—
Open-Book RAG Grounded Accuracy (@5%)	—	—	44.4%
Echo / Contamination Rate	0.0%	0.0%	0.0%

Key Finding: Verifier RL (GRPO) decisively enforces format structure and response termination (<think> compliance: 73.1% $\rightarrow$ 97.8%). When paired with open-book RAG context retrieval over seismology literature, numerical calculation accuracy increases to 44.4%.

---

🚀 Quick Start / How to Load Checkpoints

Clone the GitHub code repository and run inference:

git clone https://github.com/jiazhe868/nanogpt-seis-posttrain.git
cd nanogpt-seis-posttrain
pip install -r requirements.txt

# Run interactive CLI chat with the GRPO RL model
python -m src.sft.chat --ckpt checkpoints/grpo/grpo_ckpt_step_100.pt

# Run interactive chat with RAG context grounding enabled
python -m src.sft.chat --ckpt checkpoints/grpo/grpo_ckpt_step_100.pt --rag

To load checkpoints directly in Python:

import torch
from src.model.model import GPT, GPTConfig

# Initialize 113M model configuration
config = GPTConfig(
    block_size=4096,
    vocab_size=16388,
    n_layer=16,
    n_head=12,
    n_kv_head=4,
    n_embd=768,
)
model = GPT(config)

# Load state dict from downloaded checkpoint
checkpoint = torch.load("checkpoints/grpo/grpo_ckpt_step_100.pt", map_location="cpu")
model.load_state_dict(checkpoint["model"])
model.eval()

---

📄 Citation

@software{nanogpt_seis_posttrain_2026,
  author = {jiazhe868},
  title = {nanoGPT-Seis-Posttrain: SFT and verifier RL on a 113M seismology model},
  url = {https://github.com/jiazhe868/nanogpt-seis-posttrain},
  year = {2026}
}