TommyPanLab/breakout-dueling-double-dqn

Breakout Dueling Double DQN (2.5M transitions)

This is the only model from breakout-rl-engineering that is published for download. It is the strongest trained neural network in the project and the same policy that runs in the browser demo at breakout.tommypan.com. No other checkpoints (training intermediates, ablations, reward-shaping, PER or Contract v3 runs) are released.

The derived fp16 ONNX and TensorRT engines from later experiments are not published either; they were all exported from this same checkpoint.

Files

File	What it is	SHA-256
model.pt	Canonical PyTorch checkpoint (online network + config + provenance)	6002029dcdbcbb7c93fca0c589880611aed2e2e7924db0f6b0c1f5160824389a
metadata.json	Freeze record: training config, runtime, selection and holdout evaluation	276c8429ffa2f03df0b762a0bd01219149b699490be4fa648e495484f0027d44
model.onnx	fp32 ONNX export used by the browser demo (web/public/models/final_model/ in the repo)	cf90c74b1d09d5d7e2e0c71014f2daef2b5e1833425d300da06e34d1e4ef7f12
model.onnx.metadata.json	ONNX export record and parity checks against model.pt	3b66b8135c501324b08fffa5a94a682c7c22db64d2fec2684d820591312d3e4f

Model

• Algorithm: Double DQN, Dueling architecture (Nature-DQN CNN trunk, hidden size 512), 3,292,837 parameters
• Training: 2,500,000 agent transitions, seed 2022, 2 vectorized environments, run day21-final-long-training-seed2022
• Input: float32 tensor (N, 4, 84, 84), NCHW, four stacked 84×84 grayscale frames, uint8 values divided by 255.0 once before the network
• Output: 4 raw Q-values (not probabilities); greedy action = argmax
• Actions: 0 NOOP, 1 FIRE, 2 RIGHT, 3 LEFT

Results

Measured under Breakout Contract v2 (configs/eval/breakout_contract_v2.json: frame skip 4, frame stack 4, sticky actions 0.25, FIRE on reset) with ALE 0.12.0:

Evaluation	Episodes	Mean	Median	Min	Max
Selection	15	51.4	51	26	74
Final holdout	15	30.93	32	6	55

The holdout is the reported number. It is a Contract v2 result and is not directly comparable with the project's Contract v3 numbers or with the deterministic pixel controller (which is not a neural network and is not published here). The policy does not clear the game.

Usage

PyTorch, with the project's code on PYTHONPATH (run from the repository root):

import torch
from huggingface_hub import hf_hub_download
from breakout_rl.models.factory import build_q_network

path = hf_hub_download("TommyPanLab/breakout-dueling-double-dqn", "model.pt")
payload = torch.load(path, map_location="cpu", weights_only=False)  # pickle: only load files you trust
cfg = payload["model_config"]
net = build_q_network(
    payload["architecture"],
    num_actions=cfg["num_actions"],
    input_shape=tuple(cfg["input_shape"]),
    hidden_dim=cfg["hidden_dim"],
)
net.load_state_dict(payload["online_network"])
net.eval()

obs = torch.zeros(1, 4, 84, 84)  # stacked frames, already divided by 255
action = net(obs).argmax(dim=1).item()

ONNX Runtime (no project code needed):

import numpy as np
import onnxruntime as ort
from huggingface_hub import hf_hub_download

session = ort.InferenceSession(hf_hub_download("TommyPanLab/breakout-dueling-double-dqn", "model.onnx"))
q_values = session.run(["q_values"], {"observation": np.zeros((1, 4, 84, 84), dtype=np.float32)})[0]
action = int(q_values.argmax(axis=1)[0])

Use the same environment wrapper as the repository (breakout_env.py with Contract v2) to reproduce the scores; a different frame skip, sticky-action setting or life-loss handling changes the task.

License

GPL-2.0, the same as the source repository. The Atari Breakout ROM is not included and is not covered by this license.