Rectina

The problem is to read ink from the X-ray CT of the carbonised Herculaneum papyri of the Vesuvius Challenge: a detector must say, for each point of a segmented surface, whether it carries ink, and the readers in use are Reader v2 and Reader v3. We built Rectina, an ink reader that gives a score for every surface pixel, released as four checkpoints: Rectina m1 after the first training step and Rectina m2 after two more, and the later rebuilt pair Rectina m3 (after stage 2) and Rectina m4 (after stages 3 and 4 of the same chain). On the 14 evaluation units Rectina m1 is above Reader v2 on 12 of 14 units (mean AUC difference +0.0192) and Rectina m2 is above Reader v3 on 11 of 14 units (mean AUC difference +0.0104); Rectina m1 is above Reader v3 on 6 of 8 held-out and 3 of 6 control units. Rectina m3 and m4 were trained afresh on villa main with bicubic renders; on the same 14 units Rectina m4 is above Reader v3 on 10 of 14 units (mean AUC difference +0.0056) and above Reader v2 on 11 of 14 (mean difference +0.0175), while Rectina m3 is above Reader v2 on 9 of 14 (mean +0.0072) and below Reader v3 (mean -0.0047, above it on 6 of 14), differences that are small beside the evaluation to evaluation variation. Not shown here: a spread over seeds (each model is a single run), a correction for multiple comparisons, and calibration of the scores. An overlap check of the segment keys found none of the 14 units among the segments of the training configurations (it compares segments, not neighbouring areas of one volume).

Usage

Both checkpoints are PyTorch files whose embedded configuration rebuilds the architecture (vesuvius_unet_3d_stem_2d, a 17 by P by P crop) with villa's ink_detection code. They read a surface volume rendered along the normal of a tifxyz surface and give, for each surface pixel, a float32 score between 0 and 1; the scores rank pixels and are not calibrated probabilities (the README of Reader v3 says the same of its values: "Reading the values: they are a relative ink score, not a calibrated probability."). Rectina m1 reads with a 128 pixel patch and Rectina m2 with a 384 pixel patch, as read from the checkpoint configurations. The Usage section of each card lists the exact input, the read command, the output files, one worked example (Rectina m1 on unit ag1: AUC 0.8395; Rectina m2 on unit ag1: AUC 0.8466) and what has not been tested. The reader program and the render, read and convert components are to be published with the article results folder.

Contents

This repository holds four ink detection checkpoints trained on the open data of the Vesuvius Challenge (scrollprize.org): Rectina m1, m2, m3 and m4. Each model has its own folder with its checkpoint, its figure panels and its own card; this page is a short summary of the family.

model	folder	compared with	card
Rectina m1	m1/	Reader v2	m1/README.md
Rectina m2	m2/	Reader v3	m2/README.md
Rectina m3	m3/	Reader v2 and Reader v3	m3/README.md
Rectina m4	m4/	Reader v2 and Reader v3	m4/README.md

Training scrolls: the first training step (s1) of the recipe draws its batch of 64 patches from eight scrolls with per scroll quotas 0139 12, 1667 8, Paris4 6, 0814 7, 0009B 8, 0343P 5, 0500P2 7, 0172 11, on the label set 43965c9d120c of 103 units (74 with labels moved between scan resolutions and placed with a registered transformation and offset, 5 native units whose shift was measured, 24 composite units). Rectina m1 is the checkpoint of that step. Rectina m2 continues from the Rectina m1 checkpoint through two further steps whose per scroll quotas are not in the published recipe table.

Rectina m3 and Rectina m4 are a second family trained with the main branch of villa (bicubic renders of the training surfaces, the sampler revisit cap and the stages s1, ft116k and s2 for m3, then s3 and s4 for m4); their recipe, the training segments and the files are in the cards m3/README.md and m4/README.md. m3 is the checkpoint at the end of stage 2 and m4 the checkpoint at the end of stage 4. Both were trained once, with one seed.

Results

The numbers below are copied by the release tool from the two cards, which read them from pinned score and statistics files.

AUC on the 8 held-out units: Rectina m1 above Reader v2 on 8 of 8 units, the median difference is +0.0270, the mean difference is +0.0275.

AUC on the 6 control units: Rectina m1 above Reader v2 on 4 of 6 control units, the median difference is +0.0104, the mean difference is +0.0081, and the unit that lowers the mean is w058 (-0.0083).

Over the 14 units, 8 held-out and 6 control, Rectina m1 is above Reader v2 on 12 and the mean difference is +0.0192.

AUC on the 8 held-out units: Rectina m2 above Reader v3 on 6 of 8 units, the median difference is +0.0137, the mean difference is +0.0157, and the unit that lowers the mean is 20250919140325 (-0.0082).

AUC on the 6 control units: Rectina m2 above Reader v3 on 5 of 6 control units, the median difference is +0.0040, the mean difference is +0.0035, and the unit that lowers the mean is ag1 (-0.0047).

Over the 14 units, 8 held-out and 6 control, Rectina m2 is above Reader v3 on 11 and the mean difference is +0.0104.

The counts over the 14 units are descriptive. The rule for adopting a model stated in the article (at least 6 of the 8 held-out units above the reference and a median AUC gain above 0.003 on them) is descriptive too: of the six variants of the same procedure scored on the same 14 units, four meet it against Reader v3 and six against Reader v2, and the choice among them was made on these units.

Against Reader v3, Rectina m1 is above it on 6 of 8 held-out units, with a median AUC difference of +0.0162, and on 3 of 6 control units, with a median AUC difference of -0.0046. Beside the six control units stand the four control units with the organisers' labels (0343, ag1, ag2, w00): there Rectina m1 is above Reader v3 on 2 of 4 (median -0.0046) and above Reader v2 on 3 of 4 (median +0.0104); Rectina m2 is above Reader v3 on 3 of 4 (median +0.0063) against 5 of 6 on the six (median +0.0040).

Rectina m3 and Rectina m4, AUC on the same 14 units (8 held-out and 6 control), computed from the per unit AUC of the evaluation:

comparison	units above	median difference	mean difference
m3 against Reader v2, 14 units	9 of 14	+0.0089	+0.0072
m3 against Reader v3, 14 units	6 of 14	-0.0077	-0.0047
m4 against Reader v2, 14 units	11 of 14	+0.0159	+0.0175
m4 against Reader v3, 14 units	10 of 14	+0.0049	+0.0056
m4 against Reader v3, 8 held-out units	6 of 8	+0.0065	+0.0090
m4 against Reader v3, 6 control units	4 of 6	+0.0008	+0.0010

The unit that lowers the mean of m4 against Reader v3 is 20250919140325 on the held-out units (-0.0142) and ag2 on the control units (-0.0082). The six control units of m3 and m4 were rendered again with the current binary before being read, while the Reader v2 and Reader v3 maps of the control units were read on the earlier renders. The difference of m4 above Reader v3 is a few thousandths of AUC on one run, and the size of the evaluation to evaluation variation of these means was not measured, so no ranking is claimed.

Article

The method, the figures and the code pieces of Rectina m1 and m2 are in the article, db27c806-rectina. The article of Rectina m3 and m4 is in the same folder, db27c806-rectina, where it replaced the article on Rectina m1 and m2 (the earlier version stays in the git history of that repository).

Licence

Licence: the weights and this card are released under CC BY-NC 4.0, for non-commercial research use with attribution, and the code of the recipe under MIT, subject to the data terms at scrollprize.org.

Data licence: the Vesuvius Challenge data licence of the organisers applies to the data these weights were trained on (dl.ash2txt.org LICENSE.txt).

Data terms: the data of the Vesuvius Challenge are subject to its data licence (dl.ash2txt.org LICENSE.txt). The figure panels show the CT surface, the human ink labels of the evaluation units and the model maps, with attribution to their authors; they are published on the author's own responsibility as figures of this work. This repository holds no data volumes and no label files.