mlboydaisuke/ViT-Base-ImageNet-ExecuTorch

ViT-Base/16 ImageNet-1k — ExecuTorch

The original Vision Transformer, fine-tuned on ImageNet-1k: an image in, 1000 logits out. The reference classifier most tutorials reach for, as four .pte files.

• Source: google/vit-base-patch16-224 — 86M parameters, ViT-Base/16, 12 layers, hidden 768, 224x224
• License: apache-2.0
• Input: pixel_values [1, 3, 224, 224] fp32 — RGB resized to 224x224, scaled to [0,1], then normalised with mean=(0.5, 0.5, 0.5) std=(0.5, 0.5, 0.5). Not the ImageNet statistics; read off this model's own preprocessor_config.json.
• Output: logits [1, 1000], in the order of the repo's config.json id2label. Softmax is left to the caller.

Variants

build	file	size (MB)	Mac median (ms)*	labels kept	margin shift at the boundary (logits)
fp32	imgcls_vit_base_in1k_xnnpack_fp32.pte	346.4	34.9	24 of 24	0.0000
fp16	imgcls_vit_base_in1k_xnnpack_fp16.pte	174.6	71.0	24 of 24	0.0117
Core ML (fp16, iOS)	imgcls_vit_base_in1k_coreml_all.pte	173.6	4.0	24 of 24	0.0547

*Mac arm64, single process, median of 10. PyTorch eager fp32 on the same machine is 27.5 ms, so the Core ML build is 6.9x eager, 100% delegated in one subgraph. fp16 is slower than fp32 — XNNPACK emulates it — and is listed only because it halves the file.

The int8 build is not published, and correlation would have shipped it

Dynamic int8 converts, comes out at 89.7 MB, runs at 31.7 ms, and reads correlation 0.999814 against fp32 eager. It keeps the fp32 label on all 24 photographs. Every number a conversion normally reports says ship it.

The number that decides says otherwise:

worst margin shift at the decision boundary   0.1856 logits
closest photograph's distance to a decision   0.0900 logits

The build's error is twice the distance that separates the closest of these photographs from being called something else. It cannot be trusted to agree with fp32 on an image the model is not already sure about.

What changed is the number of classes, not the quantisation. The same recipe on this shelf's two-class ViT head shifts the margin by 1.4795 logits and passes comfortably — because with two classes the closest photograph sits 7.88 logits from the decision. Put a thousand classes in competition and the runner-up is usually a near tie: median distance 1.09 logits, minimum 0.09. The error budget collapses by roughly two orders of magnitude while the error itself does not.

So the honest form of the rule is not int8 is fine on ViTs. It is: an error is small or large only relative to the decision it has to survive, and a 1000-way decision has almost no room.

How these were checked

Label agreement alone would not have separated any of these builds — all four keep 24 of 24. So each photograph is also walked along the gradient of its class margin (the winner's lead over its closest rival) until the margin crosses zero, bisected onto it, and the builds compared there. All 6 boundary inputs were reached for every build; a walk that never leaves the class is reported as not measured rather than as a zero.

The boundary inputs are perturbed photographs, not natural ones. They exercise the arithmetic where it decides something; they are not a claim about accuracy on real data.

What is not tested here

The 24 photographs are this shelf's calibration set — street scenes, animals, people, general views — not an ImageNet validation split. What is verified is fidelity to the fp32 model, not the model's own top-1 accuracy. These files reproduce whatever the upstream checkpoint does.

Worth knowing before you build on it: on these images the fp32 model's own confidence runs from 0.022 to 0.501 (median). A 1000-class classifier is rarely sure, and a caller that thresholds on probability should read that row before picking a threshold.

Conversion

python convert/export_imgcls.py vit_base_in1k
python convert/check_imgcls.py vit_base_in1k int8

(conversion scripts: executorch-models)