webbrain-one/safesocial-trigger-classifier-efficientnet-lite0

SafeSocial Trigger Classifier (EfficientNet-Lite0, ONNX)

Compact on-device multi-label image classifier that detects Instagram-style social-comparison triggers. Intended as a privacy-preserving, user-controlled filtering aid — not a diagnostic tool, moderation authority, or measure of human validity.

All metrics below are student-teacher agreement against the Qwen teacher (qwen3.6-35b-a3b), not human-ground-truth accuracy. A human-verified evaluation set is still the main missing piece before making strong real-world claims.

Files

• model.onnx — deployment model, opset 17, input pixel_values (NCHW float32), output probabilities (sigmoid multi-label)
• safesocial-model.json — machine-readable manifest: labels, per-label thresholds, preprocessing, checkpoint provenance, train/val metrics

SHA-256 of model.onnx at release: fd37c1cd4aafa2bc318b7d29725fa6be3931d09cc1723bb30be3ee819ee22933

Labels (10, core taxonomy)

romance_jealousy, social_fomo, luxury_status, travel_lifestyle, body_beauty_comparison, achievement_status, social_proof_popularity, exclusivity_access, none, uncertain

Two labels from the earlier broad taxonomy were removed before this run and are not predicted by this checkpoint: envy (too subjective / visually underdetermined) and financial_fomo (too rare in the image sample).

Performance (tuned per-label thresholds)

Primary merged test (core + Kaggle partial, unchanged benchmark split):

metric	value
micro F1	0.6817
macro F1	0.5126
exact match	0.2952
label accuracy	0.8684

Separate Karpathy COCO 500-image OOD holdout:

metric	value
micro F1	0.6526
macro F1	0.3466
exact match	0.5480
label accuracy	0.9176

Report the two domains separately. Do not mix COCO numbers into the primary Instagram-domain benchmark.

Thresholds (tuned on validation split, shipped in safesocial-model.json)

label	threshold
romance_jealousy	0.75
social_fomo	0.80
luxury_status	0.45
travel_lifestyle	0.70
body_beauty_comparison	0.35
achievement_status	0.90
social_proof_popularity	0.60
exclusivity_access	0.60
none	0.75
uncertain	1.00

These thresholds need human review before product claims, especially for rare labels.

Preprocessing

• image_size: 224, resize: 256, center crop
• color: RGB, layout NCHW
• normalize with ImageNet mean [0.485, 0.456, 0.406], std [0.229, 0.224, 0.225]
• input name: pixel_values, output name: probabilities

Training

• backbone: timm/tf_efficientnet_lite0 (feature dim 1280, trained end to end)
• teacher: qwen3.6-35b-a3b via local vLLM, core reduced taxonomy
• data: core kkcosmos/instagram-images-with-captions (19,991 valid rows) + partial prithvijaunjale/instagram-images-with-captions (2,641 rows) = 22,632 merged rows, plus capped ~20% Karpathy COCO auxiliary (4,500 train rows), 20 epochs
• this checkpoint is the aux20 20-epoch EfficientNet-Lite0 run from paper/core_20k_findings.md

Quick use (Python + onnxruntime)

import json
import numpy as np
import onnxruntime as ort
from PIL import Image

manifest = json.load(open("safesocial-model.json"))
labels = manifest["labels"]
thresholds = manifest["label_thresholds"]

img = Image.open("photo.jpg").convert("RGB")
img = img.resize((256, 256))
w, h = img.size
img = img.crop(((w-224)//2, (h-224)//2, (w+224)//2, (h+224)//2))
x = np.asarray(img, dtype=np.float32) / 255.0
mean = np.array([0.485, 0.456, 0.406], np.float32)
std = np.array([0.229, 0.224, 0.225], np.float32)
x = (x - mean) / std
x = np.transpose(x, (2, 0, 1))[None]

sess = ort.InferenceSession("model.onnx", providers=["CPUExecutionProvider"])
probs = sess.run(None, {"pixel_values": x})[0][0]
print({l: float(p) for l, p in zip(labels, probs)})
print([l for l, p in zip(labels, probs) if p >= thresholds[l]])

Browser: ONNX Runtime Web (WebGPU/WASM). Mobile: ONNX Runtime React Native via an Expo development build (Expo Go cannot run the native path); mock scores are only a fallback. See the source repo browser_extension/ and mobile_app/ prototypes.

Limitations / data caveat

• Teacher-labeled test data measures agreement with the teacher, not human validity.
• uncertain had ~zero support in test splits.
• Public Instagram-style datasets often have unclear scraping/licensing histories: fine for exploratory research, but product training should prefer opt-in, licensed, or synthetic imagery.
• COCO/Flickr data is auxiliary out-of-domain robustness data, not primary benchmark evidence.
• Do not redistribute source Instagram/Kaggle/HF image files unless the source license explicitly allows it.
• No LICENSE file ships in the source repo at release time; confirm rights before commercial use.

Source

Experiment details: paper/core_20k_findings.md in the SafeSocial repo. Model export: image_classifier/export_deployment_bundle.py.