Tonylmz/ParentDopingFlowFinal-checkpoints

ParentDopingFlowFinal Checkpoints

This repository stages the checkpoints and benchmark tables selected for external release.

Start here

Read PROJECT_STATUS.md first. It is the single current overview: scope, all three boards, contributions, limits and remaining work.

Use checkpoints/joint_v2_parentgroup_frozen_ep300.pt. It is trained with a parent-grouped split, so no parent compound appears on both sides of the holdout.

What this delivers

Given a parent structure and a set of candidate doping edits: pick the optimal concentration (mean optimal-x error 0.0122; 0.0135 on an unseen family), pick the best edit (regret 3.20 K, about half of random), and construct the chosen edit into a full structure with site occupancies (100% composition validity on oxides against a 99.6% real-data control).

What it does not deliver

Proposing edits absent from the dataset. Three independent routes fail: the CSP trunk copies atom types verbatim so the flow cannot generate composition; a learned proposer has 155 training examples over 35 parents against ~930 classes; and 88.6% of enumerated proposals have no known Tc, so they cannot be scored at all. See PROJECT_STATUS.md section 5.

Three things that are easy to misread

1. The dome metric fluctuates by up to 0.029 rho between epochs of the same run, so smaller differences are not real.
2. rho and top-1 already sit at the label-noise limit; only regret has headroom, and 89% of it is on parents whose best edit is multi-channel.
3. Under a protocol that permits memorisation a random forest on composition reaches rho 0.776 against this model's 0.745. It is nearest-neighbour retrieval (r=0.987), and on a held-out family it collapses to -0.142 while this model holds 0.839. Always quote the cross-family number alongside.

A finetune must set atom_type_manifold=null_manifold to match the pretrain. Mismatching it silently collapses the doping-axis metric from 0.730 to 0.229 while the training curves still look healthy.

Files marked SUPERSEDED are the July release, measured on a split where 91.7% of the benchmark cases were training data; they are kept only for reproducibility.

Included artifacts

File	Role	Notes
PROJECT_STATUS.md	project_status_entry_point	START HERE. Scope, all three boards, contributions, limits, artifacts.
checkpoints/joint_v2_propose_v1_ep300.pt	propose_mode_experiment_checkpoint	Trained with --mark_identity_condition and a learnable propose token to let generation choose the edit itself. The token did learn (norm 0.71) but behaviour is unchanged, because the CSP trunk copies atom types verbatim and cannot generate composition. Kept as the negative result.
benchmark/composition_baselines_leaky.json	composition_baselines_leaky	RF/SVR/Ridge on composition only. Under the memorisation-permitting protocol RF reaches rho 0.776 vs the flow model's 0.745; it must be reported alongside the cross-family result below.
benchmark/composition_baselines_cross_family.json	composition_baselines_cross_family	With cuprate_Y123 removed from baseline training, RF/SVR/Ridge score rho -0.142 / 0.054 / -0.005 on the 89 held-out Y123 series against the flow model's 0.839. RF is nearest-neighbour retrieval (r=0.987 vs its nearest training formula's Tc), so it has nothing to retrieve here.
benchmark/composition_validity.json	composition_validity	Charge-neutrality validity, oxides only. Conditioned construction 100%, real-data control 99.6%, unconditioned generation 66.7%. The criterion is a false negative for the intermetallic families.
benchmark/rf_vs_flow_scorer.json	rf_vs_flow_scorer	Same candidate pool, two scorers. RF picks the right channel slightly more often, which shows the discovery board cannot separate discovery from retrieval rather than that RF is better at discovery.
benchmark/de_novo_proposal.json	de_novo_proposal	Unconditioned generation returns the parent unchanged for 5 of 6 parents; the one exception is chemically impossible (O5Ba1Pb1).
benchmark/enumerate_construct_score.json	enumerate_construct_score	Enumerate, construct, score. 88.6% of top picks are edits absent from the data and therefore unscoreable: only 4 of 35 parents yield a regret.
benchmark/alignment_tolerance_ladder.json	alignment_coverage_limit_ladder	Why board B covers only 30 parents. Loose tolerances rescue 98% at the fit stage but those correspondences carry median 0.354 fractional drift.
benchmark/regret_attribution.json	regret_attribution	20 of 30 parents are solved exactly; multi-channel optima carry 89% of the remaining regret.
benchmark/channel_synergy.json	channel_synergy	With the primary channel fixed, true delta Tc still spans 13.59 K and the per-group
benchmark/ranking_ep300_mc2.json	ranking_board_ep300	Board B for all three splits plus four trivial baselines, all at epoch 300.
checkpoints/joint_v2_parentgroup_frozen_ep300.pt	delivery_checkpoint_leakage_free	Primary delivery checkpoint. Frozen backbone, parent-grouped split (0% parent overlap between train and val), epoch 300. Dome benchmark rho 0.745 / peak_x_err 0.0122 / regret 1.32 K.
checkpoints/joint_v2_familyholdout_y123_ep300.pt	cross_family_checkpoint	Cross-family run: every cuprate_Y123 doping pair held out of training. Scores rho 0.839 on the 89 held-out Y123 series vs 0.840 for the model trained on them.
checkpoints/joint_v2_pairrow_frozen_ep300.pt	leaky_split_reference_checkpoint	Same recipe on the historical pair-row split, kept as the leakage reference: rho 0.742 vs 0.745, i.e. removing leakage costs nothing.
checkpoints/structure_csp_all_tcfix_v2_best_tc.pt	structure_pretrain_warm_start	CSP-conditioned pretrain all three runs warm-start from. Trained with atom_type_manifold=null_manifold; a finetune must match that or the doping-axis metric collapses from 0.730 to 0.229.
metadata/joint_v2_parentgroup_split_summary.json	parentgroup_split_summary	Parent-grouped split, 0% parent overlap.
metadata/joint_v2_familyholdout_split_summary.json	familyholdout_split_summary	Family holdout split; val contains only cuprate_Y123.
benchmark/DELIVERABLE_BENCHMARK_AND_CONTRIBUTIONS_20260812.md	current_deliverable	Current benchmark definitions, results and contributions. Start here.
benchmark/MEASUREMENT_FIXES_AND_NEXT_ACTIONS_20260807.md	measurement_investigation_log	Full investigation log, including every retracted conclusion.
benchmark/dome_parentgroup_ep300_summary.json	dome_benchmark_summary	Benchmark A result for the delivery checkpoint (213 series).
benchmark/ranking_ep300_mc2.json	ranking_benchmark_summary	Benchmark B for all three splits plus four trivial baselines.
benchmark/label_noise_floor.json	label_noise_floor	Label-noise floor. rho 0.745 and top-1 0.700 match a perfect estimator limited by ~2 K label noise, so those metrics are already at the label resolution limit.
benchmark/channel_synergy.json	channel_synergy_ablation	Multi-channel synergy: with the primary channel fixed, true delta Tc still spans 13.59 K and the per-group
benchmark/regret_attribution.json	regret_attribution	20 of 30 parents are solved exactly; multi-channel optima carry 89% of the remaining regret.
benchmark/alignment_tolerance_ladder.json	alignment_coverage_limit	Why benchmark B covers only 30 parents: loose tolerances rescue 98% at the fit stage, but those correspondences carry median 0.354 fractional drift and are correctly rejected.
checkpoints/joint_v2_hardfam_preserve_best_tc.pt	main_guided_replay_checkpoint	SUPERSEDED (2026-08-13): measured on a leaky split. Kept for reproducibility.
checkpoints/joint_v2_with_others_best_tc.pt	baseline_checkpoint	Main baseline for guided replay comparison.
checkpoints/joint_v2_hardfam_v1_best.pt	hardfam_ablation_checkpoint	Hard-family weighted ablation checkpoint.
checkpoints/structure_csp_no_others_last.pt	structure_pretrain_checkpoint	Optional structure pretrain initialization checkpoint.
metadata/joint_v2_hardfam_preserve_pair_row_split_summary.json	preserve_split_summary	Train/val split summary for the preserve run.
metadata/joint_v2_hardfam_v1_pair_row_split_summary.json	hardfam_v1_split_summary	Train/val split summary for the hardfam v1 run.
metadata/joint_v2_with_others_pair_row_split_summary.json	baseline_split_summary	Train/val split summary for the baseline run.
benchmark/FINAL_BENCHMARK_TABLE_20260805.md	final_benchmark_markdown	SUPERSEDED (2026-08-13): all numbers are training-set scores. See DELIVERABLE_*.md.
benchmark/overall.csv	overall_benchmark_csv	Overall benchmark table in CSV.
benchmark/by_family.csv	family_benchmark_csv	Per-family benchmark table in CSV.

Recommended checkpoint

• Primary guided replay checkpoint: checkpoints/joint_v2_hardfam_preserve_best_tc.pt
• Baseline reference: checkpoints/joint_v2_with_others_best_tc.pt
• Hard-family ablation: checkpoints/joint_v2_hardfam_v1_best.pt

Benchmark

See benchmark/FINAL_BENCHMARK_TABLE_20260805.md and the CSV tables under benchmark/.