PhacoSight deployment checkpoints

Trained model weights for PhacoSight, a cataract-surgery video analysis system for resident-physician education. This repo holds the exact checkpoints the deployed analysis stack loads (E7-b, adopted 2026-08-18):

Files	Model	Role
runs/segformer_b2_anatomy_instrument/fold0/best.pt	SegFormer-B2 (5 classes)	anatomy + instrument segmentation, tool-fusion features, UI overlays
runs/segformer_b2_instrument_multiclass/fold0/last.pt	SegFormer-B2 (7 classes)	multi-class instrument segmentation, tool-fusion features
runs/phase_mstcnpp_tools_1fps_aug_seed{0,1,2}/fold{0-3}/val_best.pt	MS-TCN++ ×12	surgical-phase recognition ensemble on DINOv2 + tool features, 1 fps

Usage

Don't load these directly — the PhacoSight repo fetches and checksum-verifies them into place:

git clone https://github.com/jdlaurence/phacosight.git && cd phacosight
python -m venv .venv && .venv/bin/pip install -e ".[app]" torch torchvision transformers
.venv/bin/python scripts/download_weights.py   # downloads this repo's files into runs/
.venv/bin/python -m app                        # physician web app → http://localhost:7860

Checksums live in the repo's configs/weights_manifest.json.

Intended use & limitations

Research and surgical-education use (post-hoc review of recorded cataract surgeries). Not a medical device; not for intraoperative or clinical decision-making. Trained on 1024×768 microscope video from a single center — expect domain shift on other rigs.

Provenance & licenses

• Segmentation models are fine-tuned from nvidia/mit-b2 (ImageNet-pretrained SegFormer encoder; NVIDIA releases those pretrained weights for non-commercial research use).
• Phase heads consume features from facebook/dinov2-with-registers-large (Apache-2.0), downloaded separately at runtime.
• Training data: Cataract-1K (CC BY 4.0). Any use must cite:

@inproceedings{Cataract-1K,
    author    = {Negin Ghamsarian and
                Yosuf El-Shabrawi and
                Sahar Nasirihaghighi and
                Doris Putzgruber-Adamitsch and
                Martin Zinkernagel and
                Sebastian Wolf and
                Klaus Schoeffmann and
                Raphael Sznitman},
    title     = {Cataract-1K: Cataract Surgery Dataset for Scene Segmentation, Phase Recognition, and Irregularity Detection (to appear)},
}