ReMEDi / docs /VALIDATION.md
pranamanam's picture
Upload 62 files
3f98d52 verified
|
Raw
History Blame Contribute Delete
3.94 kB

Executed validation

The release was exercised on 12 September 2026 with Python 3.12 and CPU execution. Installed dependency versions are recorded in requirements-tested.txt.

Tests

All 15 tests passed. These cover convex optimization, common-shift invariance, covariance calculations, molecular split integrity, independent control pools, concentration conversion, sparse counts, gene reordering, missing genes, and serialization. The synthetic end-to-end demo also completed.

Measured-data runs

The following runs completed through the repository's validate_release.py command.

Run Cells used Compounds Treated condition summaries Queries Candidate pairs per query
Tahoe A549, plates 6 and 14 387 48 96 4 6
sci-Plex3 A549, both replicates 120 9 36 4 2
Frozen Tahoe → sci-Plex3 Source 387, external 120 Source 48, external 9 Source 96, external 36 4 16

The Tahoe response ensemble was fitted on 26 compounds. The small sci-Plex ensemble was fitted on three compounds, with three others reserved for calibration and two for testing. Both study-specific runs completed all eight generation rules, CSV exports, catalog sampling, and Ubuntu plots. The catalog sampler was also exercised with a five-pair shortlist.

For external transfer, the studies were aligned on 35,460 unique gene symbols. The PCA transform, response ensemble, and calibration were fitted using Tahoe. All nine external structures were absent from response-head training. Every ReMEDi query solve returned optimal. Maximum reported comparator-constraint violations were below 1e-6.

Five uncertainty configurations completed on sci-Plex: the main configuration, permuted scenarios, diagonal covariance, omitted residual floor, and fixed target/control means. The forward-evaluation command completed the zero, training-mean, chemical-neighbor, and fitted-head comparisons on Tahoe.

Measured result CSVs are included under examples/results/. These small cohorts establish execution on the released schemas and counts. Full-study accuracy, encoder comparisons, dose interpolation, and REAL candidate activity remain experimental work. The tests used fingerprints and eight-dimensional training-only PCA features. UCE and MolFormer checkpoint extraction were not executed in this environment.

Reproduction

pip install -e '.[data,dev]'
python -m pytest
python scripts/validate_release.py --output runs/validation
python scripts/run_ablations.py \
  --data runs/validation/sciplex/data --model runs/validation/sciplex/model \
  --output runs/ablations --tolerance 1 --max-queries 2 --scenarios 16
python scripts/evaluate_forward.py \
  --data runs/validation/tahoe/data --model runs/validation/tahoe/model \
  --output runs/forward

The fixtures include measured cells and source identifiers. The Tahoe fixture contains 568 cells before the two-block eligibility filter. The sci-Plex fixture uses two treated cells per condition and four controls per plate. Increase these counts for scientific evaluation.

Source acquisition

Tahoe shards train-00600-of-03388.parquet, train-00601-of-03388.parquet, and train-02600-of-03388.parquet were downloaded from the official release and decoded with its sample and gene metadata. Reservoir sampling was restricted to A549 (CVCL_0023) with a cap of four cells per drug, concentration, and plate.

sci-Plex was read from the corrected scPerturb v1.4 H5AD through cached HTTP byte ranges. The published full-file MD5 is recorded with the subset. Only subset bytes were retrieved for the successful range-based run, so a full-file checksum was not recomputed. The full-download command verifies the published MD5 after download.

The standalone wheel built successfully and contains both Ubuntu fonts. The source archive excludes temporary downloads, caches, environments, and build products.