laya-vision-iolaus

screen_type judge for Iolaus (HackerRank Orchestrate, October 2026). Hackathon use only.

Fine-tuned from thaitea/laya-vision @ f2fe3c12cb6d04c59d8a190250bf3fb40fc828dc with the laya-vision fork @ 9e1e2419d855ad3e1a2af4d4bd1ef6be5418842c (vision tower frozen, about 115M parameters trained).

License: CC BY-NC-SA 4.0, inherited from the base weights (whose training data includes ScienceQA and CrisisMMD). Non-commercial, share-alike. The fork's code is Apache 2.0.

Task

One 12-way choice question: which internal admin system an application screenshot belongs to (account_admin, analytics_view, backup_console, connector_console, export_console, deleted_account_reference, marketing_console, object_storage, search_inspector, security_explorer, support_ticket, other). Input: the whole screen (one 512 px tile) plus its Tesseract 5.3.4 text (psm 3, first 900 characters) as context. Confidence is the chosen option's probability after the fitted choice temperature (1.185).

Data

3,600 synthetic full screens (300 per kind, seed 20261025) plus data v3 views: shell-only and cropped screens (soft target 0.6 on the true kind), near-blank canvases (uniform target) and non-console documents (other). No real personal data. Split by layout template (7 train / 1 calib / 1 test per kind), app names held out by split.

Results (synthetic test split, full screens, unseen layouts and app names)

	accuracy	ECE	share below 0.80
zero-shot base	0.429	0.188	1.000
fine-tuned, T=1	0.970	0.027	0.023
fine-tuned, fitted T	0.970	0.023	0.028

Partial views (taught to stay below the 0.80 act bar): crop: accuracy 0.780, 98.6% below 0.80, shell: accuracy 0.846, 100.0% below 0.80. Training: 1314 steps, batch 8, 88.5 min on Tesla T4 (bf16 autocast).

Limitations

Trained only on synthetic look-alikes of one console design system; real screens from other products will need re-evaluation. Not a PII detector: identifiers are read by OCR and matched in code, never by this model.