Ced-Collab/RNASBDD

RNA-SBDD baseline checkpoints

Twenty-five trained checkpoints behind a frozen 713-pocket RNA structure-based drug design benchmark: six architectures under one common training budget, in four transfer settings. These are the exact files pinned by path and SHA-256 in the per_pocket.csv of the 30-arm aggregate full30-v1, so every number below is reproducible from the file next to it.

Companion release — the frozen dataset, splits and evaluation references: CedLJH/rna-sbdd-v2.

Layout

<architecture>/<setting>.pt

setting	meaning
crossdocked-source	protein–ligand pretraining on CrossDocked. Applied to RNA it is the zero-shot arm; it is also the initialisation of the matching rna-finetune.
crossdocked-source-336000	the same DiffSBDD pretraining run carried to 336,000 iterations instead of 194,000
rna-scratch	RNA only, random initialisation, frozen budget
rna-finetune	RNA, initialised from that architecture's crossdocked-source
joint-1to1	one model trained on CrossDocked and RNA mixed 1:1
*-t250	trained with 250 diffusion timesteps instead of 1000

Frozen training budget: 75,000 iterations at batch 16, seed 2022. Missing combinations are settings that were never run, not settings that failed.

What each checkpoint scores

Frozen 713 test pockets, 20 requests per pocket, one evaluator identity. clash_atom and site F1 are defined on 712 of 713 — one pocket's native ligand contacts no receptor residue at 4.0 Å, so no interaction metric exists there for any arm.

file	validity	mol_stab	clash_atom	site F1	QED	SA
biopolymer_context_targetdiff/crossdocked-source.pt	0.889	0.600	0.080	0.691	0.361	0.641
biopolymer_context_targetdiff/rna-finetune.pt	0.916	0.775	0.096	0.728	0.341	0.670
biopolymer_context_targetdiff/rna-scratch.pt	0.769	0.796	0.100	0.714	0.343	0.713
diffbp/joint-1to1-t250.pt	0.971	0.147	0.608	0.657	0.417	0.521
diffbp/rna-scratch-t250.pt	0.917	0.192	0.573	0.673	0.264	0.553
diffbp/rna-scratch.pt	0.904	0.169	0.522	0.692	0.370	0.610
diffsbdd/crossdocked-source-336000.pt	0.199	0.318	0.626	0.646	0.326	0.667
diffsbdd/crossdocked-source.pt	0.214	0.275	0.607	0.646	0.317	0.652
diffsbdd/joint-1to1-t250.pt	0.121	0.582	0.650	0.635	0.406	0.740
diffsbdd/rna-finetune.pt	0.445	0.681	0.580	0.662	0.372	0.746
diffsbdd/rna-scratch-t250.pt	0.342	0.505	0.602	0.644	0.362	0.685
diffsbdd/rna-scratch.pt	0.321	0.453	0.571	0.650	0.368	0.708
graphbp/crossdocked-source.pt	0.750	0.002	0.927	0.167	0.490	0.610
graphbp/joint-1to1.pt	0.770	0.009	0.894	0.241	0.480	0.640
graphbp/rna-finetune.pt	0.809	0.057	0.800	0.245	0.482	0.669
graphbp/rna-scratch.pt	0.829	0.039	0.834	0.243	0.480	0.644
pocket2mol/crossdocked-source.pt	0.275	0.372	0.264	0.428	0.387	0.692
pocket2mol/joint-1to1.pt	0.499	0.201	0.315	0.721	0.381	0.645
pocket2mol/rna-finetune.pt	0.571	0.181	0.167	0.720	0.407	0.644
pocket2mol/rna-scratch.pt	0.503	0.181	0.183	0.695	0.381	0.653
targetdiff/crossdocked-source.pt	0.886	0.652	0.057	0.696	0.350	0.640
targetdiff/joint-1to1-t250.pt	0.990	0.724	0.383	0.686	0.369	0.551
targetdiff/rna-finetune.pt	0.887	0.764	0.099	0.733	0.340	0.680
targetdiff/rna-scratch-t250.pt	0.951	0.588	0.295	0.701	0.315	0.572
targetdiff/rna-scratch.pt	0.893	0.768	0.095	0.715	0.338	0.675

Read these before comparing anything

Six things about this benchmark are not discoverable from the numbers, and each has misled us at least once.

interaction_site_f1 is not a valid optimisation target. A zero-parameter rule — the residues nearest the cavity centroid — scores 0.6784, and a structureless always-emitting null 0.6750, against 0.7330 for the best arm here. Under a named test on the pockets where every arm is defined, zero of eight published arms are significantly above that null when all requests are scored. Use the column as a guard, never as an objective.

diversity, uniqueness and novelty are inverted quality proxies here. Across arms, their correlation with site F1 is −0.601 / −0.523 / −0.505, and diversity correlates with atom-level clash at +0.596. A generator making implausible molecules maximises all three for free. Native crystallographic ligands score 0.185 on novelty; every arm scores 0.80–0.99, so high novelty is further from the reference behaviour, not closer.

The receptor-clash floor is 0.440, not zero, at molecule level (0.042 at atom level). Native ligands clash with their own receptor in 44% of cases under this kernel, so inter_clash_molecule_ratio must never be read as if zero were the target. inter_clash_atom_ratio is the discriminating one.

A *-t250 checkpoint may only be compared with another *-t250 checkpoint. Training at 250 timesteps is not a cheap substitute for 1000: TargetDiff RNA scratch goes inter_clash_atom_ratio 0.0948 → 0.2945 and molecule_stability 0.7682 → 0.5884, while sampling-time respacing of a T=1000 checkpoint moves clash by 0.002. So joint-1to1-t250 reads against rna-scratch-t250, never against rna-scratch or rna-finetune.

Four of the six RNA arms had not converged at the frozen budget — their monitor loss was still descending at 75,000. Every absolute number here is a floor, and cross-architecture rankings carry a convergence-speed confound. Within-family paired contrasts are much less exposed.

Some of these baselines are weak, and that is inherited rather than introduced. GraphBP, DiffBP and Pocket2Mol post low molecule_stability and high clash. The CBGBench benchmark these implementations come from reports the same failure modes on its own protein data — GraphBP's atom-level clash there is 0.8634 against 0.834 here — and names the causes: GraphBP's "difficulty in locating the correct atoms for auto-regressive growth", and DiffBP/DiffSBDD generating "a large number of unreasonable triangular and tetrahedral rings". DiffBP is the one arm whose degradation from protein to RNA (11.6×) is out of line with TargetDiff's 2.0×; the suspect is its hard-coded Ångström-scale interior regulariser (k=48, rho=2, gamma=5), tuned for protein pocket geometry. We report that rather than retune it, because retuning would make the arm not-DiffBP.

One more, on accounting. validity, coverage and funnel_validity are the same number in this evaluator: after the largest-fragment policy all three reduce to evaluated / requested. Report one.

Integrity

SHA256SUMS and MANIFEST.json carry every file's digest, the arms it backs in full30-v1, and its trained timestep count.

sha256sum -c SHA256SUMS

Not included

Checkpoints produced by combining others rather than by training — the weight-space merge used in an interpolation-locus study — are excluded. They are derived artifacts rather than baselines, and are reproducible from the two endpoint checkpoints published here.