huyquoctrinh/madryat

runs/madry_at — Checkpoint Index

Which checkpoint belongs to which experiment. All runs are CIFAR-10, trained/evaluated with python -m madry_at.main.

Paths below are relative to the repo root (runs/madry_at/<exp-name>/...). Logs for every run are in runs/logs/<exp-name>*.log.

Each run directory contains:

File	Meaning
checkpoint.pt.best	Best epoch by validation accuracy (adversarial accuracy for AT runs, clean accuracy otherwise).
checkpoint.pt.latest	Last saved epoch.
149_checkpoint.pt	Snapshot at the final epoch (epoch 150). Same weights as .latest for completed runs.
store.h5	Cox store: logs table with nat_prec1, adv_prec1, losses per logged epoch.
tensorboard/	TensorBoard events.

1. Canonical checkpoints (use these)

These are the checkpoints used for all reported results in results/effnet_b2_robustness/README.md.

EfficientNet-B2

Experiment	Threat model (train)	Checkpoint to use	Epoch	Clean val	Robust val (7-step PGD)
Standard training	none (ε = 0)	cifar10_efficientnet_b2_standard_20260824/checkpoint.pt.best	71	81.92%	–
L2 AT, ε = 0.25	L2, ε=0.25, lr_atk=0.05	cifar10_efficientnet_b2_l2_advtrain_epstrain_0p25_20260825/checkpoint.pt.best	141	81.79%	70.23%
L2 AT, ε = 0.5	L2, ε=0.5, lr_atk=0.1	cifar10_efficientnet_b2_l2_advtrain_epstrain_0p5_20260825/checkpoint.pt.best	141	76.08%	59.49%
L2 AT, ε = 1.0	L2, ε=1.0, lr_atk=0.2	cifar10_efficientnet_b2_l2_advtrain_epstrain_1p0_resume_1gpu_gpu2_20260826/checkpoint.pt.best	150	57.80%	41.24%
L∞ AT, ε = 8/255	L∞, ε=8/255, lr_atk=2/255	cifar10_efficientnet_b2_linf_advtrain_epstrain_8over255_1gpu_bs256_20260826/checkpoint.pt.best	136	58.45%	37.24%

Common training settings: SGD, lr 0.1, step_lr 50 (×0.1), weight decay 5e-4, 150 epochs, batch size 128 (L∞ run: 256), data augmentation on. AT runs use 7 PGD steps with random start.

Note on L2 ε = 1.0: this model was trained in three segments because the run was interrupted:

1. cifar10_efficientnet_b2_l2_advtrain_epstrain_1p0_20260825 — epochs 1–21 (stopped).
2. cifar10_efficientnet_b2_l2_advtrain_epstrain_1p0_resume_20260826 — resumed from (1) .latest (epoch 21), ran to epoch 36 (stopped).
3. cifar10_efficientnet_b2_l2_advtrain_epstrain_1p0_resume_1gpu_gpu2_20260826 — resumed from (2) .latest (epoch 36) with optimizer state, completed to epoch 150. ← use this one.

ViT-B/16

Experiment	Optimizer	Checkpoint to use	Epoch	Clean val
Standard training (SGD)	SGD, lr 0.1, wd 5e-4, bs 128	cifar10_vit_b_16_standard_20260826/checkpoint.pt.best	141	53.13%
Standard training (AdamW)	AdamW, lr 1e-3, wd 0.05, bs 64, lr ×[(0,1.0),(150,0.01)]	cifar10_vit_b_16_adamw_bs64_20260827/checkpoint.pt.best	150	62.88%

No adversarially trained ViT checkpoints exist yet.

2. Evaluation-only runs → source checkpoint

These directories contain no model weights, only store.h5 with the attack result. All use --eval-only 1 --adv-eval 1 --use-best 1 --random-restarts 0 --random-start 0. "Robust acc" is adv_prec1 from store.h5.

L2 attack evaluations (100-step PGD, attack lr = ε_test / 5)

Eval run	Source checkpoint (train ε)	ε_test	Clean	Robust acc
cifar10_efficientnet_b2_l2_attack_eval_epstest_0p25_20260825	standard (0.0), epoch 71	0.25	81.91%	21.84%
cifar10_efficientnet_b2_l2_attack_eval_20260824	standard (0.0), epoch 71	0.5	81.92%	1.24%
cifar10_efficientnet_b2_l2_attack_eval_epstest_1p0_20260825	standard (0.0), epoch 71	1.0	81.91%	0.00%
cifar10_efficientnet_b2_l2_attack_eval_epstest_2p0_20260825	standard (0.0), epoch 71	2.0	81.91%	0.00%
cifar10_efficientnet_b2_l2_attack_eval_epstrain_0p25_epstest_0p5_20260826	L2 AT 0.25, epoch 141	0.5	81.82%	49.70%
cifar10_efficientnet_b2_l2_attack_eval_epstrain_0p25_epstest_1p0_20260826	L2 AT 0.25, epoch 141	1.0	81.82%	14.94%
cifar10_efficientnet_b2_l2_attack_eval_epstrain_0p25_epstest_2p0_20260826	L2 AT 0.25, epoch 141	2.0	81.82%	0.18%
cifar10_efficientnet_b2_l2_attack_eval_epstrain_0p5_epstest_0p25_20260826	L2 AT 0.5, epoch 141	0.25	76.09%	67.33%
cifar10_efficientnet_b2_l2_attack_eval_epstrain_0p5_epstest_1p0_20260826	L2 AT 0.5, epoch 141	1.0	76.09%	30.96%
cifar10_efficientnet_b2_l2_attack_eval_epstrain_0p5_epstest_2p0_20260826	L2 AT 0.5, epoch 141	2.0	76.09%	2.80%
cifar10_efficientnet_b2_l2_attack_eval_epstrain_1p0_epstest_0p25_20260826	L2 AT 1.0, epoch 150	0.25	57.80%	53.43%
cifar10_efficientnet_b2_l2_attack_eval_epstrain_1p0_epstest_0p5_20260826	L2 AT 1.0, epoch 150	0.5	57.80%	48.54%
cifar10_efficientnet_b2_l2_attack_eval_epstrain_1p0_epstest_2p0_20260826	L2 AT 1.0, epoch 150	2.0	57.80%	14.17%

There is no separate attack-eval run for the matched-ε cells (train ε = test ε, ε ∈ {0.25, 0.5, 1.0}). For those, the robust number is the 7-step training validation value in Section 1.

L∞ attack evaluations — MadryLab benchmark format (attack lr = 2.5 · ε_test / steps)

Eval run	Source checkpoint (train ε)	ε_test	Steps	Clean	Robust acc
cifar10_efficientnet_b2_linf_madry_epstrain_0over255_epstest_8over255_steps20_20260827	standard, epoch 71	8/255	20	81.93%	0.00%
cifar10_efficientnet_b2_linf_madry_epstrain_0over255_epstest_8over255_steps100_20260827	standard, epoch 71	8/255	100	81.93%	0.00%
cifar10_efficientnet_b2_linf_madry_epstrain_0over255_epstest_16over255_steps20_20260827	standard, epoch 71	16/255	20	81.93%	0.00%
cifar10_efficientnet_b2_linf_madry_epstrain_0over255_epstest_16over255_steps100_20260827	standard, epoch 71	16/255	100	81.93%	0.00%
cifar10_efficientnet_b2_linf_madry_epstrain_8over255_epstest_8over255_steps20_20260827	L∞ AT 8/255, epoch 136	8/255	20	58.44%	36.54%
cifar10_efficientnet_b2_linf_madry_epstrain_8over255_epstest_8over255_steps100_20260827	L∞ AT 8/255, epoch 136	8/255	100	58.44%	36.56%
cifar10_efficientnet_b2_linf_madry_epstrain_8over255_epstest_16over255_steps20_20260827	L∞ AT 8/255, epoch 136	16/255	20	58.44%	16.79%
cifar10_efficientnet_b2_linf_madry_epstrain_8over255_epstest_16over255_steps100_20260827	L∞ AT 8/255, epoch 136	16/255	100	58.44%	16.69%

L∞ attack evaluations — older format (100-step PGD, attack lr = ε_test / 4)

Superseded by the MadryLab-format runs above; kept for reference.

Eval run	Source checkpoint (train ε)	ε_test	Clean	Robust acc
cifar10_efficientnet_b2_linf_attack_eval_epstrain_0over255_epstest_8over255_20260826	standard, epoch 71	8/255	81.93%	0.00%
cifar10_efficientnet_b2_linf_attack_eval_epstrain_0over255_epstest_16over255_20260826	standard, epoch 71	16/255	81.93%	0.00%
cifar10_efficientnet_b2_linf_attack_eval_epstrain_8over255_epstest_16over255_20260826	L∞ AT 8/255, epoch 136	16/255	58.44%	16.20%

3. Incomplete, superseded, or debug runs (do not use)

Run	Status	Notes
cifar10_efficientnet_b2_l2_advtrain_epstrain_1p0_20260825	stopped at epoch 21	Segment 1 of L2 ε=1.0; continued by the _resume_ runs.
cifar10_efficientnet_b2_l2_advtrain_epstrain_1p0_resume_20260826	stopped at epoch 36 (.best = epoch 31)	Segment 2 of L2 ε=1.0; continued by _resume_1gpu_gpu2_.
cifar10_efficientnet_b2_linf_advtrain_epstrain_8over255_20260826	no checkpoint (died in epoch 0)	First L∞ attempt at bs 128; replaced by the _1gpu_bs256_ run.
cifar10_vit_b_16_adamw_20260827	stopped at epoch 6 (24.98% clean)	AdamW at bs 128; replaced by cifar10_vit_b_16_adamw_bs64_20260827.
cifar10_vit_b_16_adamw_debug_20260827	1 epoch	AdamW smoke test.
_probe_linf_1gpu_bs24576_1787744411	1 epoch	Batch-size / memory probe for L∞ AT.
_probe_linf_1gpu_bs32768_1787744385	no checkpoint	Batch-size / memory probe (failed).

4. Canary runs

None yet. Canary-trained runs (--canary-json ...) will also write canary_metrics.jsonl in their run directory. Add them to Section 1 with the canary JSON path they used.