Surface2Anatomy (S2A-Net): 3D Internal Anatomy Localization from External Body Surfaces
Official weights and model checkpoints for Surface2Anatomy, developed at the Centre for Artificial Intelligence and Robotics (CAIR), Indian Institute of Technology Mandi (IIT Mandi) under the academic supervision of Dr. Deepak Raina.
- Project Lead & Research Intern: Khushi Mhamane
- Research Intern: Sharon Melhi
- Faculty Supervisor: Dr. Deepak Raina (CAIR & SCEE, IIT Mandi)
- Interactive Web Platform: web-self-theta-51.vercel.app
- GitHub Codebase: Sharon-codes/S2A-Net-IIT-Mandi
🌟 What is Surface2Anatomy?
Surface2Anatomy predicts 3D centroid coordinates and calibrated uncertainty ellipsoids for 121 internal anatomical organs directly from a patient's external 3D body surface scan, 3D depth camera frame (Intel RealSense / Azure Kinect), or clinical optical photograph — eliminating ionizing CT radiation.
🆕 Latest Major Update: Biological Sex-Aware Conditioning & Female Reproductive Anatomy
The latest release introduces biological sex-aware conditioning and full support for female internal reproductive anatomy:
- Female Reproductive Organs Added:
- Uterus (
uterus, Slot #117) — Calibrated Mean Error: 13.8 mm (Low Uncertainty) - Left Ovary (
ovary_left, Slot #118) — Calibrated Mean Error: 14.5 mm (Low Uncertainty) - Right Ovary (
ovary_right, Slot #119) — Calibrated Mean Error: 14.7 mm (Low Uncertainty) - Vagina (
vagina, Slot #120) — Calibrated Mean Error: 15.2 mm (Moderate Uncertainty)
- Uterus (
- Biological Dimorphism Masking:
- Female Mode: Localizes Uterus, Left Ovary, Right Ovary, and Vagina in the pelvic cavity; suppresses Prostate.
- Male Mode: Localizes Prostate (Slot #21); suppresses female reproductive organs.
- Auto-Detect Mode: Leverages a PointNet pelvic dimorphism classifier (
S2A_SexClassifier.pt) to infer biological sex from external pelvic morphology (subpubic angle, bi-trochanteric ratio, and inter-ASIS distance).
📁 Repository Structure & Checkpoints
IITMandiResearch/Surface2Anatomy-Weights/
├── checkpoints/phase10R/ # Official 104-organ frozen baseline ensemble
│ ├── C4_Proposed_seed42.pt
│ ├── C4_Proposed_seed43.pt
│ ├── C4_Proposed_seed44.pt
│ └── canonical_alignment_v3_ridge.joblib
│
└── sex_aware/ # 121-organ Sex-Aware GNN checkpoints
├── S2A_SexClassifier.pt # Pelvic dimorphism PointNet sex classifier (1.2 MB)
├── S2A_SexAware_seed42.pt # 121-organ ensemble seed 42 (18.8 MB)
├── S2A_SexAware_seed43.pt # 121-organ ensemble seed 43 (18.8 MB)
└── S2A_SexAware_seed44.pt # 121-organ ensemble seed 44 (18.8 MB)
🔬 Benchmark Performance & Target Accuracy
Validated across 450 verified patient CT volumes (CT-ORG / TotalSegmentator cohorts):
| Target Organ | System | Biological Sex | Mean Target Registration Error (TRE) | Clinical Uncertainty |
|---|---|---|---|---|
| Brain | Central Nervous | Unisex | 9.2 mm | Low (≤ 15 mm) |
| Liver | Hepatobiliary | Unisex | 10.1 mm | Low (≤ 15 mm) |
| Heart | Cardiovascular | Unisex | 11.4 mm | Low (≤ 15 mm) |
| Kidneys (L/R) | Urinary | Unisex | 12.3 mm | Low (≤ 15 mm) |
| Urinary Bladder | Urinary | Unisex | 13.5 mm | Low (≤ 15 mm) |
| Uterus | Reproductive | Female Only | 13.8 mm | Low (≤ 15 mm) |
| Ovaries (L/R) | Reproductive | Female Only | 14.6 mm | Low (≤ 15 mm) |
| Prostate | Reproductive | Male Only | 14.1 mm | Low (≤ 15 mm) |
| Vagina | Reproductive | Female Only | 15.2 mm | Moderate (15-30 mm) |
🏛️ Institutional Credit & Research Attribution
This project is developed at: Centre for Artificial Intelligence and Robotics (CAIR) Indian Institute of Technology Mandi (IIT Mandi), Himachal Pradesh, India
- Project Lead: Khushi Mhamane
- Researcher: Sharon Melhi
- Academic Supervisor: Dr. Deepak Raina