quantum-gpt-109M

A ~109M parameter GPT-style base model trained with nanochat, using a quantum-inspired MLP block (mlp_type: quantum) in place of the standard feed-forward layer.

Checkpoint

• Training step: 1634
• Validation bpb: 0.9571
• Epoch: 1

Architecture

Param	Value
n_layer	16
n_head / n_kv_head	16 / 16
n_embd	1024
sequence_len	1024
vocab_size	4096
mlp_type	quantum
quantum_num_qubits	2
quantum_depth	2
window_pattern	L

Files

• model_001634.pt — model weights
• meta_001634.json — full training/model config and metrics at this step

Optimizer state is not included in this upload (only needed to resume training, not to use the model).

Usage

Load with the nanochat checkpoint manager from MarkCodering/Quantum-GPT:

from nanochat.checkpoint_manager import load_model
# see repo for exact loading API