pi-r2 — DETR for document layout detection (loc_beyond_words)

Fine-tuned DETR (ResNet-50) for detecting layout regions in historical newspaper page scans, trained on biglam/loc_beyond_words (2,846 train / 712 validation images, 7 classes).

Model / license

• Base model: facebook/detr-resnet-50 — Apache-2.0 (open license, free to share).
• The fine-tuned model inherits the Apache-2.0 license.
• Dataset biglam/loc_beyond_words is CC0-1.0 (public domain).

Classes (7)

Photograph, Illustration, Map, Comics/Cartoon, Editorial Cartoon, Headline, Advertisement

Validation results (COCO-style metrics on the held-out validation split, 712 images)

Metric	Value
AP @[IoU .50:.95]	0.423
AP @IoU .50	0.566
AP @IoU .75	0.486
AP small	0.050
AP medium	0.250
AP large	0.453
AR max=1	0.286
AR max=10	0.574
AR max=100	0.625

Per-class AP

Class	AP
Photograph	n/a
Illustration	n/a
Map	n/a
Comics/Cartoon	n/a
Editorial Cartoon	n/a
Headline	n/a
Advertisement	n/a

Note: AP_small/AP_medium/AP_large are computed on the resized evaluation inputs, not original pixel areas; treat small-object numbers cautiously.

Training details

• Recipe: standard DETR fine-tuning (AdamW, lr=1e-4, backbone lr=1e-5, wd=1e-4, grad clip 0.1, fp16, linear warmup/decay, batch size 2 with grad accumulation 2).
• Input: images resized to shortest edge 480px / longest edge 800px (aspect kept), RGB.
• Epochs: up to 15 with early stopping (patience 3 on AP@.5) and validation every 4 epochs.
• Hardware: 1x NVIDIA T4 (16GB) via Hugging Face Jobs (t4-small).
• Cost: < $5 total GPU compute.

Quickstart

from transformers import DetrImageProcessor, DetrForObjectDetection
from PIL import Image
import torch

processor = DetrImageProcessor.from_pretrained("harness-race/pi-r2")
model = DetrForObjectDetection.from_pretrained("harness-race/pi-r2")

image = Image.open("page.png").convert("RGB")
inputs = processor(images=image, return_tensors="pt")
with torch.no_grad():
    outputs = model(**inputs)

results = processor.post_process_object_detection(
    outputs, threshold=0.5, target_sizes=[(image.height, image.width)])[0]
for score, label, box in zip(results["scores"], results["labels"], results["boxes"]):
    print(model.config.id2label[label.item()], round(score.item(), 3), [round(v, 1) for v in box.tolist()])

Files

• model.safetensors, config.json, preprocessor_config.json — fine-tuned model + processor
• val_metrics.json — full COCO validation metrics (incl. per-class AP)
• figures/ — sample predictions on the validation set
• train_detr.py, requirements.txt — training code for reproducibility

Limitations

• Trained on grayscale newspaper scans; color documents may perform worse.
• Classes are heavily imbalanced (Headline/Advertisement dominate; Map/Editorial Cartoon are rare).
• Small text-heavy regions may be missed at 480–800px input resolution.