garment-detector-seg
Instance segmentation of garments, accessories, and garment components in photographs. RF-DETR-Seg (Apache 2.0) fine-tuned on Fashionpedia (CC BY 4.0), predicting all 46 Fashionpedia classes: 13 main garments (shirt/blouse … cape), 14 accessories, and 19 garment parts (collar, lapel, sleeve, pocket, neckline, zipper, buckle, and trim classes such as rivet, ruffle, sequin).
Built for wholesale apparel manufacturing QC, not e-commerce tagging: the
segmentation masks feed deterministic color measurement, and the part classes
map to BOM trim lines and construction details in a tech pack. It is one of
two models in a pipeline (see garment-attributes for fine-grained
construction attributes).
Intended use
- Locate garments and their visible components in sample/production photos.
- Provide masks for downstream color QC (ΔE2000 against a spec target).
- Verify component presence/count against a tech pack (e.g. "2 flap pockets, 1 center-front zipper, no hood").
Out of scope: fiber content, GSM, measurements, stitch class, interior construction — none of these are visually determinable and this model does not pretend to output them.
Training
- Base: RF-DETR-Seg-Small (Apache 2.0), DINOv2 backbone, 33.6M params.
- Data: Fashionpedia train2020 (45,623 images, 333,401 instances), converted
by
scripts/prepare_detector_data.py(attributes stripped; masks kept). - Recipe:
scripts/train_detector.pywithconfigs/detector_mac.yaml(8 epochs, effective batch 16, lr 1e-4, fp16 AMP, trained on Apple-silicon MPS in ~37 hours).
Evaluation (Fashionpedia val2020, 1,158 images, epoch 8)
| Metric | Value |
|---|---|
| mask mAP@[.5:.95] | 0.352 |
| mask mAP@.5 | 0.502 |
| box mAP@[.5:.95] | 0.419 |
| box mAP@.5 | 0.549 |
| box mAP, main garments (ids 0–12) | 0.589 |
| box mAP, accessories (ids 13–26) | 0.534 |
| box mAP, garment parts (ids 27–45) | 0.204 |
Strongest classes: dress (0.87), pants (0.84), jacket and coat (approx. 0.75). Weakest: small trims — zipper (0.10), beads/appliqué (0.05) — and rare categories (cape, jumpsuit). Full per-class table ships in this repo.
Small trim classes (rivet, bead, sequin) score materially lower than garment
classes in all published Fashionpedia baselines; expect the same here and
check the per-class table in runs/detector before relying on a trim class
for QC decisions.
Known limitations & biases
- Fashionpedia images are street/celebrity photos of worn garments; performance on flat-lay or on-hanger factory photos is lower. Fine-tune on in-domain photos for production deployment (the training script supports any COCO-format dataset).
- Occluded components (e.g. back pockets in a front photo) are not detected — the downstream validator reports missing components as REVIEW, not FAIL, for this reason.
- Small trims (<32 px) are frequently missed at 1024-px inference resolution.
- Dataset skews toward Western womenswear street fashion; expect weaker performance on technical/workwear categories.
License & attribution
Weights: Apache 2.0. Training data: Fashionpedia, CC BY 4.0 — cite Jia et al., Fashionpedia: Ontology, Segmentation, and an Attribute Localization Dataset, ECCV 2020. Architecture: RF-DETR (Roboflow, Apache 2.0), ICLR 2026.
Usage
from rfdetr import RFDETRSegSmall
model = RFDETRSegSmall(pretrain_weights="checkpoint_best_ema.pth")
detections = model.predict("sample.jpg", threshold=0.5)
Full pipeline (attributes + color + tech pack validation): see the
garment-classifier repository this model ships with.