wellesley-easel/randall-segmentation-model

Randall segmentation model

This checkpoint segments medieval manuscript pages into eight semantic classes. It is a multi-class adaptation of MapSAM (Xia et al. 2024).

Important: base model required

model.pth is not a standalone SAM checkpoint. It contains the trained rank-8 DoRA parameters, four depthwise self-prompt-generator branches, prompt encoder, and eight-class mask decoder. Inference also requires Meta's original SAM ViT-L checkpoint, sam_vit_l_0b3195.pth, obtained separately under its own terms.

Classes

ID	Label
0	unlabeled
1	line filler
2	marginal images
3	painted initials
4	miniatures
5	borders
6	pen flourishing
7	pen flourished texts

Evaluation

The checkpoint was evaluated on 154 held-out manuscript pages after retiling 764 model-input tiles to page resolution.

Metric	Value
Overall pixel accuracy	0.953899
Foreground pixel accuracy	0.702386
Foreground mean IoU (classes 1–7)	0.552400
Marginal-images IoU	0.460426
Marginal-images Dice	0.630537

Class	IoU	Dice
unlabeled	0.967716	0.983593
line filler	0.665873	0.799428
marginal images	0.460426	0.630537
painted initials	0.525954	0.689345
miniatures	0.618410	0.764220
borders	0.553674	0.712729
pen flourishing	0.507451	0.673257
pen flourished texts	0.535014	0.697080

Full counts, precision, recall, and the confusion matrix are in evaluation/metrics.json.

Loading and inference

Clone the source revision above, install its dependencies, download the base SAM ViT-L checkpoint, and download this repository's files. From the source checkout, inference follows this pattern:

python run_inference.py \
  --input_dir INPUT_IMAGES \
  --output_dir OUTPUT_SEGMENTATIONS \
  --dora_ckpt model.pth \
  --ckpt sam_vit_l_0b3195.pth \
  --config config.txt \
  --id2label id2label.json

config.txt contains only inference-relevant settings and intentionally omits the machine-local paths recorded by the training job. config.json provides the same architecture metadata in a structured form.

Training summary

• Backbone: SAM ViT-L
• Input size: 1024 × 1024
• Output resolution: 256 × 256 before page-space resizing
• DoRA rank: 8, applied to image-encoder query and value projections
• Self-prompt generator: depthwise variant
• Optimizer: AdamW
• Base learning rate: 0.0001 with 750-iteration warmup
• Batch size: 6
• Training data: 340 manuscript pages converted to 1,925 tiles
• Evaluation data: 154 pages converted to 764 tiles

The test split was evaluated repeatedly during training, so the reported test metrics should be treated as descriptive rather than as an unbiased estimate from a never-observed final test set.

Limitations and intended use

This model was trained on a specialized collection of medieval manuscript images. Performance may degrade for other repositories, imaging conditions, layouts, periods, or visual traditions.

This checkpoint is released under the MIT License. The separately obtained SAM base checkpoint remains governed by its own license.