adopd/RF-DETR-Seg-2XL-segmentation-ADOPD

RF-DETR-Seg 2XL Segmentation ADOPD

Thinking with Anchors Project | ADOPD 2026 Paper: Thinking with Anchors: Grounded and Efficient Document Reasoning | ADOPD 2024 Paper | Dataset | Code

Model Overview

• Model developer: Thinking with Anchors project contributors
• Base architecture: RF-DETR
• Architecture variant: RF-DETR-Seg 2XL
• Task: class-agnostic document entity segmentation
• Fine-tuning dataset: adopd/adopd2026
• Input: one RGB document image
• Output: entity boxes, confidence scores, and instance masks

Description

RF-DETR-Seg 2XL Segmentation ADOPD is an RF-DETR instance-segmentation model fine-tuned to identify visual entities in document pages. It predicts a single entity class and retains every polygon component associated with each source mask during conversion to COCO supervision.

Training Data

The checkpoint was fine-tuned on the ADOPD Doc2Mask task. Public supervision is stored in:

human_annotated_masks[].polygons

The released data adapter converts these polygons to RF-DETR's COCO training layout without using the independent legacy annotation layers.

Checkpoint Format

model.ckpt is a PyTorch Lightning checkpoint for the seg-2xlarge architecture. It is not an RFDETR.from_checkpoint() export. The architecture must therefore be supplied when loading the model through the companion code.

Quick Start

git clone https://github.com/SichenZhu/ADOPD2026.git
cd ADOPD2026/release_code

git clone https://github.com/roboflow/rf-detr.git upstream/rf-detr
git -C upstream/rf-detr checkout 7f2490d4ece5a894b6bfe69e876a1d5d9936e2e1

python -m pip install -e model_zoo/common
python -m pip install -e 'upstream/rf-detr[train,loggers]'
python -m pip install -e model_zoo/rf_detr

hf download adopd/RF-DETR-Seg-2XL-segmentation-ADOPD \
  --local-dir checkpoints/rfdetr-seg-2xl

adopd-rfdetr-infer \
  --checkpoint checkpoints/rfdetr-seg-2xl/model.ckpt \
  --architecture seg-2xlarge \
  --image document.jpg \
  --threshold 0.5 \
  --output prediction.json

prediction.json contains pixel-space boxes, confidence scores, and masks.

Fine-Tuning And Evaluation

Prepare ADOPD2026 with adopd-rfdetr-prepare --task segment, then follow the training and evaluation commands in rf_detr.

Limitations

The model predicts a single document-entity class rather than semantic entity types. Confidence and mask thresholds may require calibration when document styles, languages, resolutions, or scan quality differ from the fine-tuning data.

License

The companion RF-DETR code is released under Apache-2.0. Review the licenses of the RF-DETR base model and all installed dependencies before use.

Citation

Please cite the ADOPD 2026 and ADOPD 2024 papers.

@misc{zhu2026thinkingwithanchors,
  title={Thinking with Anchors: Grounded and Efficient Document Reasoning},
  author={Sichen Zhu and Yuchen Zhu and Wenzhuo Xu and Jason Kuen and Wanrong Zhu and Jing Shi and Xuan Shen and Quanyi Wang and Yiwei Wang and Yujun Cai and Bing Shuai and Qin Zhang and Yongxin Chen and Shilong Liu and Molei Tao and Jiuxiang Gu},
  year={2026}
}

@inproceedings{gu2024adopd,
  title={{ADOPD}: A Large-Scale Document Page Decomposition Dataset},
  author={Jiuxiang Gu and Xiangxi Shi and Jason Kuen and Lu Qi and Ruiyi Zhang and Anqi Liu and Ani Nenkova and Tong Sun},
  booktitle={The Twelfth International Conference on Learning Representations},
  year={2024},
  url={https://openreview.net/forum?id=x1ptaXpOYa}
}