yapwithai/phone-screen-classifier

Yap Phone Screenshot Classifier

Yap Phone Screenshot Classifier predicts two labels for one image:

1. screen: the phone-screen category.
2. safety: the content-safety category.

The model returns exactly two outputs: screen and safety.

The model is fine-tuned from timm/mobilenetv4_conv_medium.e250_r384_in12k.

What It Is For

This model is built for routing and filtering mobile screenshot workflows. It separates UI classification from safety classification so an application can answer both of these questions before running more expensive analysis:

• What kind of screen is this?
• Which safety category does this image belong to?

generic and other are real screen labels when included in the exported label map. generic means phone UI that is not one of the more specific trained screen classes. other means non-phone UI or images outside the phone-screenshot distribution.

Files

File	Purpose
onnx/model.onnx	ONNX model for CPU/server inference.
onnx/model.fp16.onnx	Optional FP16 ONNX candidate.
onnx/model.onnx.data	External ONNX weight data loaded beside model.onnx.
model.safetensors	PyTorch state dict for reproducibility and continued training.
config.json	Model identity, base model, output names, and label arrays.
preprocess.json	Resize and normalization contract used during training/export.
train.json	Sanitized training recipe for post-training or reproduction.
inference/python.py	Python helper for ONNX inference from image paths.
inference/typescript.ts	TypeScript helper for ONNX inference from image paths.
inference/labels.json	Screen and safety labels used to decode model logits.
README.md	This model card and runnable inference reference.

Outputs

The ONNX graph has two outputs in this exact order:

Output	Shape	Meaning
screen	[batch, screen_class_count]	Screen-category logits.
safety	[batch, safety_class_count]	Content-safety logits.

Inference uses argmax for both heads in this version.

Test Results

Model	Test	Output	Accuracy	Balanced Acc	Macro F1	Top-2	Images
fp32	full test	screen	0.9154	0.8385	0.7265	0.9719	23615
fp32	full test	safety	0.9575	0.8941	0.8600	0.9921	23615
fp32	screen-balanced test	screen	0.8377	0.8421	0.8133	0.9400	6246
fp32	screen-balanced test	safety	0.9776	0.9285	0.9110	0.9978	6246
fp32	safety-balanced test	screen	0.9580	0.7752	0.6547	0.9870	3000
fp32	safety-balanced test	safety	0.8957	0.8957	0.8947	0.9847	3000
fp16	full test	screen	0.9153	0.8384	0.7264	0.9719	23615
fp16	full test	safety	0.9575	0.8941	0.8600	0.9921	23615
fp16	screen-balanced test	screen	0.8373	0.8419	0.8130	0.9400	6246
fp16	screen-balanced test	safety	0.9776	0.9285	0.9110	0.9978	6246
fp16	safety-balanced test	screen	0.9580	0.7752	0.6547	0.9870	3000
fp16	safety-balanced test	safety	0.8957	0.8957	0.8947	0.9847	3000

CPU Timing

Model	Images/s	Load Mean	Model Mean	Total Mean	Total Median	Total P95	Provider
fp32	20.0538	0.12 ms	49.52 ms	49.87 ms	57.84 ms	85.82 ms	onnxruntime:CPUExecutionProvider
fp16	20.8351	0.11 ms	47.69 ms	48.00 ms	48.60 ms	63.65 ms	onnxruntime:CPUExecutionProvider

Timing is measured with ONNX Runtime CPU execution on Apple M4 Max (16 logical cores). Total latency includes image load/preprocess, model inference, and label decoding.

Classification Labels

For the screen and safety label lists, see config.json.

Preprocessing

Use the preprocessing contract in preprocess.json.

1. Read the image.
2. Apply EXIF orientation.
3. Convert to RGB.
4. Resize so the longest side is 1024 pixels and keep the original aspect ratio.
5. Do not crop, stretch, square-pad before preprocessing, or horizontally flip.
6. Convert to float32 in [0, 1].
7. Normalize with the mean and standard deviation from preprocess.json.
8. Pad each normalized tensor to the batch maximum height and width, rounded up to a multiple of 32.

The exported model accepts dynamic batch, height, and width.

ONNX Inference

The helpers intentionally return only screen and safety.

Keep onnx/model.onnx.data beside onnx/model.onnx; ONNX Runtime loads the external tensor data when it opens the graph. By default the helpers load the FP32 model at onnx/model.onnx. Pass fp16 to load onnx/model.fp16.onnx.

Download

Download the exported model folder from Hugging Face:

from huggingface_hub import snapshot_download

model_dir = snapshot_download("yapwithai/phone-screen-classifier")

Python

Install the runtime dependencies:

python -m pip install numpy pillow onnxruntime

Import the helper from the exported model folder:

from inference.python import Classifier, classify

print(classify("example.png"))
print(classify("example.png", model_format="fp16"))

classifier = Classifier(model_dir)
print(classifier.classify_batch(["one.png", "two.png"]))

fp16_classifier = Classifier(model_dir, model_format="fp16")
print(fp16_classifier.classify_batch(["one.png", "two.png"]))

Or run it directly:

python inference/python.py example.png another.png --model-format fp16

TypeScript

Install the runtime dependencies:

bun add sharp onnxruntime-node

Import the helper from the exported model folder:

import { Classifier, classify } from './inference/typescript.ts';

console.log(await classify('example.png'));
console.log(await classify('example.png', modelDir, 'fp16'));

const classifier = await Classifier.create(modelDir);
console.log(await classifier.classifyBatch(['one.png', 'two.png']));

const fp16Classifier = await Classifier.create(modelDir, 'fp16');
console.log(await fp16Classifier.classifyBatch(['one.png', 'two.png']));

Or run it directly:

bun inference/typescript.ts example.png another.png --model-format fp16

Citation

If you use this model, please cite:

@misc{phone-screen-classifier,
  title={Yap Phone Screenshot Classifier},
  year={2026},
  publisher={Yap With AI},
  url={https://huggingface.co/yapwithai/phone-screen-classifier}
}