cedarrapidsboy/inkwell-panel-models

Inkwell Panel Models

Two models for automated comic panel processing, trained on the same hand-annotated comic page corpus:

1. Panel Detector — locates panel bounding boxes on a comic page (object detection, YOLOv8-n).
2. Panel Reading Order — given a page's detected panels, predicts the order a reader should view them in (pairwise gradient-boosted classifier).

They're meant to be used together: run the detector on a page to get panel boxes, then run the ordering model on those boxes to get a reading sequence. Each model has its own license — see License below, they are not the same.

---

1. Panel Detector (YOLOv8-n)

A YOLOv8-nano object detection model that locates comic panels on scanned or digital comic book pages.

Model description

• Task: single-class object detection
• Class: panel (class_id 0) — the rectangular (or near-rectangular) bounded region of a single comic panel on a page. The model does not distinguish panel shape, gutter, speech balloons, or reading order — it only predicts the bounding box of each panel.
• Architecture: YOLOv8-n (nano), fine-tuned from the pretrained yolov8n.pt COCO checkpoint.
• Input size: 640×640
• What it's for: automatically detecting panel boundaries on a comic page so downstream tools can crop, reorder, or reflow panels — e.g. guided panel-by-panel reading, panel extraction for translation/OCR pipelines, or building reading-order datasets. This is the "edge" tier model: small and fast enough for on-device / mobile inference.

Training data

• ~3,800 annotated comic book pages (3,380 train / 423 validation, plus a held-out test split) drawn from a range of genres and art styles (e.g. superhero, manga-influenced, indie/alternative, all-ages), so the model isn't overfit to one visual style or layout convention.
• Every panel bounding box was hand-tuned — each page's annotation was manually created or corrected before being ingested into the training set.
• Pages with no panels (e.g. full-page splash art, covers) were included with empty label files as negative examples, so the model also learns to correctly predict zero panels on non-paneled pages.
• Labels use standard YOLO format: <class_id> <cx> <cy> <w> <h>, normalized to 0–1, single class (panel).

Training procedure

Trained from the yolov8n.pt pretrained checkpoint using Ultralytics YOLOv8.

Parameter	Value
Base weights	yolov8n.pt
Image size	640
Epochs	300 (early-stopping patience 30)
Batch size	16
Mosaic augmentation	1.0
Mixup augmentation	0.0
Train / Val pages	3,380 / 423

Results

Model	Type	Epochs	Batch	imgsz	Train pages	Val pages	Precision	Recall	mAP@50	mAP@50-95
comic-panels-v1	nano	300	16	640	3,380	423	0.9681	0.9331	0.9769	0.9249

Limitations

• Trained on comic pages — performance on non-comic images, storyboard art, or extremely non-standard/artistic panel layouts (irregular borders, panel-less "flow" layouts) is not guaranteed.
• Detects panel bounding boxes only; it does not infer reading order, panel type, or contents (speech bubbles, characters, etc.). That's handled by the second model in this repo.
• Single-class model — all detections are labeled panel regardless of panel shape or size.

---

2. Panel Reading Order (Gradient Boosting Classifier)

A pairwise reading-order model. Given the panel boxes on a comic page (e.g. from the detector above, or any other source), it predicts the order a reader should view them in — handling non-obvious layouts like Z-paths, tall panels that span both a left and right column, splash panels, and nested/overlapping panels, without hardcoded row/column rules.

Model description

• Task: pairwise ranking / reading-order prediction over a variable number of panels on a page.
• Architecture: sklearn.ensemble.GradientBoostingClassifier (100 estimators, max depth 4, learning rate 0.1). For every ordered pair of panels (a, b) on a page, 14 geometric features are computed (relative position, vertical/horizontal overlap, aspect-ratio, tallness, page-column side, panel-count normalization, etc. — see FEATURE_NAMES in the training script) and the classifier predicts P(a is read before b). Each panel's final rank comes from summing its "beats" probability against every other panel on the page and sorting descending.
• Not a neural network — pure gradient-boosted trees over geometry features, so inference is microseconds per page.
• Shipped format: panel-order-model.onnx, exported from the trained scikit-learn classifier via skl2onnx. GradientBoostingClassifier (rather than the faster HistGradientBoostingClassifier used by an internal v2 candidate) is used specifically because it converts to ONNX reliably, letting this model run in-browser via onnxruntime-web.

Training data

Same underlying corpus as the panel detector: comic pages with hand-tuned panel boxes and a hand-verified reading-order index per panel, pulled from panels.json annotations across the same range of genres and art styles. Only pages with ≥2 panels and an approved annotation flag are used (single-panel and unapproved pages are skipped, since there's no ordering signal to learn from a single panel).

Training procedure

• Trained with leave-one-file-out cross-validation: for each comic file, the model is trained on every other file and evaluated on the held-out one, so reported accuracy reflects generalization to unseen books, not just unseen pages of a book already seen in training.
• Every panel pair on a page contributes two training examples (a before b and b before a), so the effective training set is much larger than the page count alone suggests.

Results

Leave-one-file-out cross-validation, page-level exact-match accuracy (the predicted order must match the human-annotated order for every panel on the page to count as correct):

Metric	Value
Pages evaluated	3,610
Pages perfectly ordered	3,493
Page-level accuracy	96.76%

Accuracy by layout pattern (a page can match more than one pattern):

Pattern	Accuracy	Pages
Simple grid	98.58%	494
2–3 panels	99.52%	419
4–6 panels	97.99%	2,585
Unequal columns	96.76%	1,451
Has overlapping panels	95.97%	2,233
Has splash panel	95.48%	1,770
7–10 panels	91.06%	559
Has nested panels	91.91%	544
11+ panels	72.34%	47

Accuracy degrades on the densest, least common layouts (11+ panels on a single page), which is also the smallest evaluation bucket.

Limitations

• Accuracy drops on pages with very high panel counts (11+) — these are rare in the training corpus, so the model has seen fewer examples of that layout complexity.
• Predicts order from geometry only — it doesn't read panel content, so narratively-driven reading orders that contradict geometric layout (rare, but they exist in some comics) won't be captured.
• Expects panel boxes as input (from the detector above or another source); it does not detect panels itself.

---

License

The two models in this repo are licensed differently — check which one you're using.

Panel Detector — AGPL-3.0

The panel detector is a derivative of Ultralytics YOLOv8, which Ultralytics distributes under the GNU Affero General Public License v3.0 (AGPL-3.0). Because these weights were produced by fine-tuning Ultralytics' pretrained yolov8n.pt checkpoint, the detector weights are likewise distributed under AGPL-3.0.

In practice, this means:

• You are free to use, modify, and redistribute this model, provided any distribution (including source code) also uses/discloses source under AGPL-3.0.
• If you run this model (or a modified version of it) as part of a network service (e.g. a hosted API or web app), AGPL-3.0's network-use clause requires you to make the complete corresponding source code of that service available to its users.
• If you need to use this model in a closed-source or proprietary product without these obligations, Ultralytics offers a commercial Enterprise License — see their site for details. That commercial license applies to the underlying YOLOv8 architecture/code; you would need to confirm licensing terms for these specific fine-tuned weights with their author before using them under such a license.

Full AGPL-3.0 license text: https://www.gnu.org/licenses/agpl-3.0.en.html

Attribution

This model builds on:

Ultralytics YOLOv8
https://github.com/ultralytics/ultralytics
Copyright (C) Ultralytics
Licensed under AGPL-3.0

If you use the detector, please also cite Ultralytics YOLOv8:

@software{yolov8_ultralytics,
  author = {Glenn Jocher and Ayush Chaurasia and Jing Qiu},
  title = {YOLO by Ultralytics},
  version = {8.0.0},
  year = {2023},
  url = {https://github.com/ultralytics/ultralytics},
  license = {AGPL-3.0}
}

Panel Reading Order — MIT

The reading-order model is trained from scratch on top of scikit-learn (BSD-3-Clause) and does not derive from or embed any AGPL code or weights, so it does not carry the detector's AGPL-3.0 obligations. It's released under the MIT License.