momo00798/cyp-challenge-method-report

FP-MLP + D-MPNN Continuation

Completed 2026-10-25 at 07:28:47 EST (UTC-05:00). Canonical JSON audit times retain UTC.

Candidate

The recommended local submission is submissions/neural-followup-001/:

• direct_inhibition.csv: 750 molecules, four direct-inhibition pIC50 predictions.
• tdi.csv: 750 molecules, two boolean TDI labels.
• Both files pass pinned official validators and stronger local portal preflight.
• Original identifiers, SMILES and test-row ordering are preserved.
• Nothing is published or submitted. No server acceptance receipt is claimed.

The original submissions/final/, CPU audit, CPU development lock and CPU release remain unchanged. reports/methodology.md describes that historical CPU release; this document is its separate neural-continuation supplement.

Data And Leakage Controls

Only the pinned official competition data are used. The five source hashes, feature manifest, molecule table and frozen folds are verified. No external data, pretrained models, auxiliary-label training or residual correction are used.

Selection uses only the 4,328 development rows and their frozen chemical-identity exclusions. Every non-refit GPU bundle has exactly its approved D-only fitting and query fold. Query labels and confidence bounds, and all other out-of-fit measurements, are NaN in uploaded inputs. Auxiliary tables participate in identity auditing, not model supervision.

Inputs are fixed molecular structures, Morgan radius-2 fingerprints and 16 physicochemical descriptors. D-MPNN uses directed molecular graphs plus those descriptors. Outcomes, measurement uncertainty and missingness are not molecular input features. The interval-aware FP-MLP preset can use official training-label bounds in its regression loss; those bounds are not query features.

P/S were already seen during the CPU campaign. They are not a fresh independent test for this continuation. They enter the full-data refits only after locking the D-only model, epoch, blend and threshold decisions.

Training And Selection

All four existing presets were screened on development fold 0, seed 17. One finalist per family completed five folds at seeds 17 and 43. The two matching screen results were reused, so full OOF coverage required 18 additional jobs.

Finalist	Architecture	Dropout	Batch	Final Epochs, Regression / TDI
fp_mlp-1	Morgan + descriptors, hidden 256/128, interval-aware regression	0.25	128	4 / 2
dmpnn-0	Directed message passing, hidden 300, depth 3 + descriptors	0.15	64	19 / 10

Training uses separate regression and classification networks, masked Huber / interval-aware regression losses and masked BCE-with-logits classification loss. AdamW uses learning rate 0.001 and weight decay 0.00001. Inner, identity-safe stopping uses a 100-epoch ceiling and patience 12; selected epoch counts are then fixed for refits. Each finalist has two all-official-training-data refits, at seeds 17/43. No blinded-test labels are available or used.

The immutable decision.json records practical-gain, endpoint-degradation, seed/fold-repeatability and paired chemical-component bootstrap checks. Thresholds and blending are selected solely from D OOF evidence.

Output	CPU Anchor	FP-MLP	D-MPNN
Direct inhibition	0.375	0.375	0.250
TDI probabilities	0.500	0.500	0.000

The CPU anchor retains its frozen ExtraTrees/LightGBM recipe. The exported TDI decision is probability >= 0.20 for each endpoint. D-MPNN is retained for direct inhibition, not TDI.

Development Results And Limitations

These are selection scores, not new independent audit or blind-test scores. The CPU comparison is the released fallback recipe's D OOF reference, rather than the earlier pre-fallback classifier champion.

Metric	CPU Reference	Neural Complement
Macro ST-RAE, lower is better	0.750064	0.701159
CYP1A2 ST-RAE	0.816841	0.786798
CYP2C9 ST-RAE	0.704780	0.626983
CYP2D6 ST-RAE	0.925868	0.890272
CYP3A4 ST-RAE	0.552769	0.500584
Macro MCC, higher is better	0.219495	0.233807
CYP2D6 TDI MCC	0.048741	0.098402
CYP3A4 TDI MCC	0.390249	0.369213

The relative reduction in development macro ST-RAE is about 6.52%. Each successive regression addition improves all five folds and both seeds. Its 2,000-replicate paired chemical-component bootstrap supports an improvement in every replicate.

TDI gains are less certain: the macro MCC gain is 0.014313, with bootstrap support 0.8715 and a 95% paired gain interval of [-0.010277, 0.039573]. That interval includes zero. CYP2D6 TDI remains weak, and CYP3A4 TDI declines slightly. The predeclared promotion checks pass, but no independent improvement is established.

The existing analogue-holdout/test neighbour-distribution mismatch remains. OOF-based model/weight/threshold selection can be optimistic; it does not establish competition rank, clinical validity or biochemical mechanism.

Verification And Recovery

Uncached local replay loaded the CPU and neural checkpoints after checksum verification, using weights_only=True for Torch exports. It regenerated the candidate from fixed features without retraining. Maximum absolute regression difference is 2.3615785416097879e-7, below the 1e-5 neural tolerance. Every TDI boolean is identical.

The first local replay stalled in a Torch/OpenMP join barrier on macOS. Only that verified local host was terminated; the interrupted CPU attempt remains failed in the ledger and its successful retry has an explicit parent link. Single-threaded local runtime limits resolved the issue. GPU jobs were neither stopped nor repeated, and the approved training implementation hash is unchanged.

A second artifact audit verifies all 26 continuation input bundles, exact feature matrices, chemical exclusions, hidden measurements, permitted archive members, output/archive/checkpoint hashes, provider settlements and complete OOF coverage. A read-only provider reconciliation matches all 27 project GPU jobs by exact provider receipt and cyp-<attempt_id> model name; all are completed.

Resources And Environment

• Cumulative GPU cost: $2.86 of the $100 project budget, including the $0.09 pilot.
• Continuation GPU cost: $2.77 for 26 successful jobs: four screens, 18 additional OOF jobs and four full-data refits. No GPU retries were needed.
• Shared ledger: 85 attempts; this continuation adds 28, including two local replay attempts. One local CPU retry; zero pending jobs or reservations.
• Follow-up billable time totals 6,781 GPU-seconds; training-loop wall time totals 1,137.71 seconds. These sums are not elapsed campaign duration.
• Peak recorded Torch GPU allocation is 155,030,016 bytes, about 147.85 MiB. This is not whole-instance memory or a CPU memory measurement.

The user subsequently authorised recoverable retries without a separate retry count cap, subject to $100 cumulative GPU spending. No GPU retry was necessary; the original approval, configuration and counters are preserved, not retroactively rewritten. Budgets are ceilings, not spending targets.

Hardware is Scigantic ml.g5.xlarge, NVIDIA A10G, quoted at $1.41/hour. GPU runs record Python 3.11.11, Torch 2.6.0+cu124, CUDA 12.4, NumPy 2.2.6 and RDKit 2025.3.6. Provider-supplied ancillary packages differ from the local environment; every run records their actual versions. Local Python 3.11.13 / Torch 2.6.0 successfully replays predictions within tolerance. Seeded CUDA reductions are not claimed to be bitwise deterministic.

Reproduction

From the repository root:

uv sync --locked --extra dev --extra neural
uv run --locked --extra dev --extra neural python scripts/verify_neural_followup.py
uv run --locked --extra dev --extra neural cyp validate --directory submissions/neural-followup-001

The verifier is local-only and does not launch experiments or contact a provider. It regenerates reports/neural_followup_validation.json and reports/neural_followup_resources.json. The immutable decision/release and per-run checkpoints provide the full prediction recipe. The host resume adapter's --finish-local-only mode rechecks the release stage and reuses a successful cached replay; it never dispatches GPU work.

No publication, competition upload, account top-up or budget increase occurred. Public report hosting and any required submission URL need separate authorisation.