LibreEoMTs-panoptic

EoMT-S (DINOv2-Small backbone) for COCO panoptic segmentation at 640 px, repackaged for LibreYOLO. 133 classes: 80 "things" (countable instances) + 53 "stuff" (amorphous regions). Every pixel receives exactly one non-overlapping segment.

Source

Derived from tue-mps/eomt, checkpoint tue-mps/coco_panoptic_eomt_small_640_2x (upstream 2x training schedule). Copyright (c) 2025 Mobile Perception Systems Lab at TU/e. Licensed under the MIT License.

The architecture uses a DINOv2 Vision Transformer backbone (Meta Platforms, Inc., Apache-2.0), fine-tuned end-to-end by the upstream authors and released under MIT.

Modifications

State-dict key remapping only. Learned parameters are unchanged, and all 133 panoptic classes are retained. The checkpoint additionally carries a thing_class_ids metadata field (COCO panoptic is contiguous: things 0-79, stuff 80-132) so the panoptic merge knows which categories to fuse. See weights/convert_eomt_weights.py in the LibreYOLO source repository.

Usage

from libreyolo import LibreYOLO

model = LibreYOLO("LibreEoMTs-panoptic.pt")
result = model.predict("image.jpg")[0]

pan = result.panoptic              # PanopticSegmentation
seg_map = pan.data                 # (H, W) int segment-id map; 0 = void
for segment in pan.segments_info:  # id, category_id, isthing, score
    name = result.names[segment["category_id"]]
    print(segment["id"], name, "thing" if segment["isthing"] else "stuff")

Install the EoMT runtime dependency with:

pip install "libreyolo[eomt]"

Validation

Panoptic inference matches the upstream post-process: identical segment counts and category multisets, with >=99.9999% per-pixel category agreement on test images. Panoptic Quality (PQ) validation via model.val() is not implemented yet; COCO-panoptic is not redistributed here.

License

MIT License. See the LICENSE and NOTICE files in this repository.