CAPTOR pretrained tables

Term-by-word tables for CAPTOR (Caption-Anchored Patch-Term Occurrence Representation), text-encoder-free grounding by counting caption co-occurrence over a frozen image encoder's visual words. Each table holds a frozen encoder's visual words (k-means centroids of its patch tokens) and one signature per term, counted from a caption corpus. With the encoder, a table scores every term at every patch of an image; no text encoder is involved.

Authors: Elad Yoshai and Natan T. Shaked, Tel Aviv University. Code: https://github.com/EladYoshai/captor

file	encoder	K × vocabularies	caption corpus	terms	reference result
coco_dinov2reg_k640.npz	DINOv2 ViT-B/14 with registers, 196 px	640 × 4, joint words	COCO train2017, 15,000 images	78 COCO objects	COCO val2017 reporting half: mIoU 0.5654
pathcap_conch_k80.npz	CONCH image trunk	80 × 4	PathCap, 32,463 figures	25 tissue terms	BCSS, 62 held-out patients: mIoU 0.4966

MANIFEST.json gives each file's settings, size and SHA-256.

Use

pip install git+https://github.com/EladYoshai/captor.git

from captor import CAPTOR

model = CAPTOR.from_pretrained("coco_dinov2reg_k640")    # fetched from this repository
seg = model.segment("photo.jpg", ["cat", "dog", "bed"])

Each file is a plain .npz (no pickle): centroids (V, K, D), signatures (V, K, T), terms, corpus moments, optional joint-word arrays, and a JSON meta entry with the read-out settings. The format is documented in captor/io.py.

Licences

The tables contain no images. They are statistics counted from caption corpora over frozen encoders' features, and are to be used under the terms of those sources:

• coco_dinov2reg_k640: COCO captions (CC BY 4.0) and DINOv2 (Apache 2.0).
• pathcap_conch_k80: PathCap captions (CC BY-NC 2.0; Sun et al., PathAsst, 2023) over CONCH features (CONCH: CC BY-NC-ND 4.0, gated). Non-commercial research use only.

The encoders themselves are not included and must be obtained from their own model pages under their own terms.

Citation

@article{yoshai2026captor,
  title   = {{CAPTOR}: Text-Encoder-Free Grounding by Counting Caption Co-occurrence
             over the Visual Words of a Frozen Encoder},
  author  = {Yoshai, Elad and Shaked, Natan T.},
  journal = {arXiv preprint arXiv:TODO},
  year    = {2026}
}