Hibbaan/Oncoct_v1

OncoCT-1B comprehensive Google Cloud research package

This repository is a reproducible research pipeline for an approximately 1B-parameter CT oncology foundation model. It is not a trained clinical checkpoint and is not approved for diagnosis, triage, or treatment selection. Training and clinical validation require approved data, qualified investigators, and execution on GPUs.

Recommended free-data stack

The pipeline uses a deliberate dataset portfolio rather than pretending that one public dataset contains all required labels.

Dataset	Primary role	Access and license gate
NCI Imaging Data Commons	Broad CT representation learning, annotations, and external cohorts	Public cloud access; retain per-file license; most data are CC-BY, but a subset is CC-NC.
ULS23	Universal 3D lesion segmentation in chest/abdomen/pelvis CT	Use challenge terms and source licenses; hidden multicentre test is reserved for external evaluation.
Medical Segmentation Decathlon	Organ/tumour segmentation pretraining for liver, lung, pancreas, colon, and related tasks	Public AWS bucket; CC-BY-SA 4.0.
Lung-PET-CT-Dx	Lung lesion localization and pathology-conditioned representation learning	TCIA; CC-BY 4.0; CT/PET and radiologist-reviewed XML boxes.
Longitudinal-CT	Paired whole-body baseline/follow-up lesion tracking and response representation	Public repository; verify the current repository license before commercial reuse.
HCC-TACE-Seg	Treatment response, time-to-progression, survival, multiphasic CT	TCIA; CC-BY 4.0; small but clinically valuable longitudinal treatment cohort.
NSCLC-Radiomics	Research-only lung segmentation/prognostic evaluation	TCIA; CC-BY-NC 3.0; block from commercial training until cleared.

The dataset rationale and primary source URLs are recorded in dataset_catalog.yaml and oncology_ct_research_notes.md.

Acquisition

The repository never silently downloads an entire archive or bypasses access controls. Acquire only approved data and retain citations and license metadata.

python3 scripts/acquire_data.py --dataset msd
python3 scripts/acquire_data.py --dataset lung_pet_ct_dx --download-manifest --out data_sources
python3 scripts/acquire_data.py --dataset hcc_tace_manifest --download-manifest --out data_sources

For IDC, generate a cohort using idc-index or IDC BigQuery, export the query result, and keep the per-file license in each manifest row. For TCIA imaging, use the official manifest with TCIA Data Retriever after accepting the applicable terms. Do not mix patient identifiers across source systems without an approved linkage protocol.

Preprocessing and manifests

The preprocessing contract is NIfTI-first: CT volumes are clipped to [-1024, 3071] HU, normalized, patchified into 8³ voxel tokens, and saved as .pt tensors with metadata. DICOM series and DICOM-SEG/RTSTRUCT should be converted using an audited SimpleITK/dcmqi pipeline before invoking the NIfTI preprocessor.

from oncoct.preprocess import preprocess_nifti
preprocess_nifti("input.nii.gz", "processed/patient.pt", "processed/patient.json")

Build and validate a unified manifest as follows:

python3 scripts/build_manifest.py \
  --metadata-dir processed \
  --out manifests/all.jsonl \
  --source-dataset longitudinal_ct \
  --license CC-BY-4.0

Every row must include patient_id, institution, tensor_path, source_dataset, license, split, and label provenance. Splits are patient- and institution-scoped; any patient appearing in multiple splits is a hard error. Treatment and follow-up labels must be timestamped so future information never leaks into baseline prediction.

Model and curriculum

The default model uses 1,536-dimensional embeddings, 32 Transformer blocks, 24 attention heads, 8³ voxel patches, and up to 4,096 volume tokens. It exposes lesion, response, quality-control, volume, long-axis, embedding, and abstention outputs. Optional report embeddings can be supplied, but the product boundary remains evidence extraction and clinician review rather than autonomous treatment recommendation.

Training proceeds from broad CT representation learning on IDC/MSD/ULS23, to lesion characterization, to longitudinal correspondence on Longitudinal-CT, to treatment-response multitask learning on HCC-TACE-Seg, followed by institution-held-out evaluation. NSCLC-Radiomics is retained as a noncommercial research evaluation source unless its license is separately cleared.

python3 train.py --manifest manifests/train.jsonl --out runs/exp-001 --epochs 1
python3 evaluate.py --predictions runs/exp-001/predictions.jsonl

Cloud execution

Read deploy/README.md and replace placeholders only after confirming the current Google Cloud regional accelerator catalog, quota, machine family, networking, and storage policies. Use FSDP or the supported distributed launcher for the 50-GPU job. Start with one GPU, then 2–8 GPUs, then a controlled multi-node run with checkpoint recovery before using the full quota.

Non-negotiable clinical controls

The delivered source is research-only. Any clinical pilot requires institutional review, data-use agreements, de-identification verification, prospective protocol, external site validation, calibration and subgroup analysis, human-factor testing, audit logs, model-version pinning, drift monitoring, rollback, and a defined regulatory intended use. The service must abstain when input quality, domain shift, or model confidence is inadequate.