anonimkaq4/manga-page-element-segmentation

Manga Page Element Segmentation

YOLO11 instance-segmentation checkpoints for three core elements of a manga page: panel frames, people, and speech bubbles.

What the model recognizes

The model predicts an instance mask, bounding box, class label, and confidence score for every detected object. It has three classes:

Output label	Meaning	Typical use
frame	A manga panel/frame. The mask follows the visible panel area or border.	Split a page into panels before reading-order or per-panel analysis.
character (person)	A visible person or character. The exported checkpoint label is character; this is the model's person class.	Find people for cropping, composition analysis, or character-aware editing.
speech_bubble	A dialogue/speech balloon.	Send dialogue regions to OCR or exclude them from artwork processing.

character and person refer to the same class in this card. Use the literal label character when reading result.names from Ultralytics.

The repository contains two YOLO11 checkpoints. manga-yolo11m-seg-validation-best.pt is the recommended checkpoint: it has the strongest validation result among the files currently published.

Checkpoints

Checkpoint	Base model	Training snapshot	Mask mAP50	Mask mAP50-95
snapshots/yolo11m_1024_b2/manga-yolo11m-seg-validation-best.pt	yolo11m-seg.pt	epoch 15	0.8742	0.6078
snapshots/yolo11s_1024_b8/manga-yolo11s-seg-validation-best.pt	yolo11s-seg.pt	epoch 18	0.8649	0.5895

Metrics were recorded by Ultralytics on the held-out validation split used during each run, at 1024 px. They are useful for comparing these snapshots, not as a claim of performance on every manga style, scan quality, language, or page layout.

Quick start

pip install ultralytics huggingface_hub

from huggingface_hub import hf_hub_download
from ultralytics import YOLO

checkpoint = hf_hub_download(
    repo_id="anonimkaq4/manga-page-element-segmentation",
    filename="snapshots/yolo11m_1024_b2/manga-yolo11m-seg-validation-best.pt",
)

model = YOLO(checkpoint)
results = model("manga_page.png", imgsz=1024, conf=0.25)

for result in results:
    print(result.names)  # {0: 'frame', 1: 'speech_bubble', 2: 'character'}
    result.save(filename="prediction.png")

For a smaller model, change the filename to snapshots/yolo11s_1024_b8/manga-yolo11s-seg-validation-best.pt.

Training details

• Task: instance segmentation
• Architecture: Ultralytics YOLO11 segmentation
• Input resolution: 1024 × 1024
• Classes: frame, speech_bubble, character (person)
• Data: Manga109s images combined with MS92/MangaSegmentation annotations converted to YOLO segmentation format
• Augmentation: Ultralytics default training augmentation; horizontal flip enabled

Exact run settings and learning curves are kept alongside every checkpoint in args.yaml, dataset.yaml, and results.csv.

Intended use

These weights are intended as a research/prototyping component for manga-page analysis: proposing panel frames, people/characters, and dialogue balloons before downstream OCR, layout analysis, or editing workflows. Run a visual check on your own pages before using predictions in production.

Limitations

• This is an experimental snapshot, not a finished general-purpose manga detector.
• Dense action scenes, stylized or partial characters, overlapping balloons, double-page spreads, borders without frames, and low-quality scans can reduce quality.
• The model recognizes only the three listed classes. Text, sound effects, backgrounds, and reading order are out of scope.
• Training data and validation split details should be considered when comparing these numbers with other work.

Usage and license

This repository does not grant rights to the underlying manga artwork or annotations. Make sure your intended use complies with the licenses and attribution requirements of the source datasets, and with the license of the Ultralytics base model/software. This experimental checkpoint is shared without warranty; contact the repository owner before redistributing it or using it in a commercial product.

Citation

If you use the MangaSegmentation annotations, please cite:

@inproceedings{xie2025advancing,
  title={Advancing Manga Analysis: Comprehensive Segmentation Annotations for the Manga109 Dataset},
  author={Xie, Minshan and Lin, Jian and Liu, Hanyuan and Li, Chengze and Wong, Tien-Tsin},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2025}
}

Files

Each checkpoint directory contains:

• manga-yolo11*-seg-validation-best.pt — checkpoint selected by validation metric;
• results.csv — per-epoch training and validation metrics;
• args.yaml and dataset.yaml — run configuration;
• snapshot_metadata.json — compact summary of the training run.