Pine532/chameleon-layout-critic

Chameleon layout critic

A 113-weight linear model that scores a finished banner layout and is used to choose among near-equal candidates produced by the Chameleon resize engine. It was distilled from the pairwise verdicts of a strong vision-language model ("the judge") and runs on numpy alone, offline, in about 2 ms per task.

Status: enabled in the engine since 2026-10-27 (engine/models/critic.npz; CHAMELEON_CRITIC=0 turns it off). On held-out designs it improves agreement with the judge over the engine's own pick by about +4.5 points over all labelled comparisons and +3 points on the current engine's comparisons. It changed 103 of 1,162 picks on the evaluation grids; in a fifth judge round on every one of those (old pick against the critic's pick, both letter orders) the judge took the critic's 67 to 13 (critic_report.md, docs/crello-gains-2026-10-26.md §2.5). The one ship criterion it missed before (it preferred one uniform-stretch layout over the engine's pick) was outweighed by that round.

Licence. Apache-2.0 (confirmed by the team on 2026-10-28). The weights derive from team-made designs, the organizers' three sample designs and a third-party model's verdicts; no third-party weights are included. Published: https://huggingface.co/Pine532/chameleon-layout-critic (critic.npz, critic_features.py, this card, critic_report.md; the same bytes as engine/models/critic.npz).

Model details

• Type. Antisymmetric L2 logistic regression on layout features; score(layout) = sum_k w_k * x_k / sd_k, no intercept. The critic prefers layout B over layout A when score(B) - score(A) >= margin.
• Developed by. Dev 3, Chameleon hackathon team (2026-10-26). Framework. scikit-learn for training, numpy for inference.
• Size. 16.8 KB (critic.npz); 113 weights; margin 0.5; window 0.06.
• Files. critic.npz (arrays and strings only; loads with allow_pickle=False), critic_features.py (the feature code; pure python plus numpy), MODEL_CARD.md.

critic.npz keys: feature_names (113 strings, in order), sd (per-column scale), w (weights), bias (0), margin (0.5), window (0.06), card (JSON: training counts, engine commit, held-out numbers).

Intended use

Inside the resize engine (engine/resize.py, section 5b, backend judge), after the rule-based selector has produced its pick and its finalists (valid candidates within 0.06 of the best rule score, at most four more). The engine keeps its pick unless the model scores another finalist higher by at least the margin; the finalist must itself be within 0.06 of the pick's rule score. Nothing else: the model never generates, moves or edits an element, and abstains (rule score decides) without the file, with a corrupt file or an unknown feature name.

Out of scope. Ranking layouts that are not engine finalists (in particular layouts far worse on the rule score: it was never trained to reject those and prefers a uniform-stretch layout to the engine's pick in 1 to 2 of the 38 judged stretch pairs where the judge preferred the pick), scoring a single layout in absolute terms, any design language other than the Solstice-style banners it saw, and standing in for the organizers' scorer.

Inputs

Per finalist, 113 numbers, all computed from the engine's own report and the output elements (no rendering, no fonts, no files):

• 12 report numbers: nine soft metrics (soft_readability, soft_collisions, soft_hierarchy, soft_fidelity, soft_utilization, soft_spacing, soft_alignment, soft_wrapping, soft_adjacency), rule_score, min_text_px (smallest font), texts_below_floor.
• 24 complaint features (c_*, critic_features.COMPLAINT_NAMES): margins and edge contact, largest empty band, utilisation, badge-row regularity, CTA against the copy, headline dominance, logo and text contrast on panels, dots on text, text over pills, tiny-text share, stretched ellipses, panels flush with the canvas. Definitions: critic_report.md section 4.
• 77 geometry features (critic_features.GEO_NAMES): occupancy and thirds, margins, overlaps, alignment, gaps, balance, element and font sizes.

Usage with the engine: copy critic.npz to engine/models/critic.npz; CHAMELEON_CRITIC=0 or deleting the file disables it. Standalone:

import numpy as np, critic_features as cf
z = np.load("critic.npz", allow_pickle=False)
names, coef, margin = [str(n) for n in z["feature_names"]], z["w"] / z["sd"], float(z["margin"])
def score(design, src_elements, report):        # report: the 12 engine numbers by name
    vals = {**report, **cf.design_features(design, src_elements)}
    return float(np.array([vals[n] for n in names]) @ coef)
# prefer the alternative only if score(alt) - score(pick) >= margin and |rule(alt) - rule(pick)| <= window

Training data

Pairwise comparisons of the engine's own layouts, judged by a strong vision model twice with the letters swapped; a comparison is decided when both orders agree on a non-tie winner.

round	engine	comparisons	kept for training
1	0fdf32b	300	159 decided finalist pairs
2	main v branch picks	140	56 decided old-v-new pairs
3	4d70239	620	403 decided pairs (370 with the engine's real pick, 36 clearly-worse calibration pairs)

618 decided comparisons from 61 designs (3 organizer samples, 18 synthetic, 40 holdout designs in 26 template families), after removing pairs judged twice. In addition: 284 comparisons the judge could not separate (soft label 0.5, weight 2) and 361 unjudged "engine pick v uniform stretch" pairs built from the training designs (label: the pick wins, weight 0.3). 44 judged stretch and reference pairs are held back as sanity tests. Layouts and labels: tools/judge/data/critic_inputs.tar.gz, tools/judge/labels*/. The labels reflect one judge's taste (larger type, fuller canvases, tidy badge rows, CTA close to its copy) and say nothing about the organizers' scorer.

Training procedure

tools/judge/train_critic.py, export_critic.py. Per comparison x = f(A) - f(B), where A is the layout the current engine would ship; mirrored rows; columns scaled by their RMS; L2 logistic regression with C = 0.03 (chosen by an inner design-grouped CV on log-loss); margin 0.5 (the modal margin chosen by the nested procedure: largest net corrections with override precision >= 0.7). No pretrained components. Deterministic.

Evaluation

Held-out designs (repeated 10-fold GroupKFold by design, 20 shuffles; C and margin chosen on the training part). Agreement with the judge; "engine" = the engine's real pick (round 3) or its rule score plus readability tie-break re-implemented on the pair.

population	n	engine	critic	gain, 95% CI	overrides per pass / precision
all rounds, margin 0.5	618	80.9%	85.7%	+4.8 [+2.6, +7.1]	58 / 76%
all rounds, margin chosen by the nested procedure	618	80.9%	85.2%	+4.3 [+2.2, +6.4]	53 / 75%
round 3 only (current engine)	370	83.5%	86.8%	+3.3 [+0.5, +6.0]	31 / 70%
rule-score gap < 0.01	181	68.0%	75.8%	+7.9 [+3.0, +13.1]	25 / 79%
organizer samples (3 designs)	77	88.3%	91.7%	+3.4 [-2.1, +9.6]	6 / 72%
holdout designs (40)	349	75.6%	81.4%	+5.8 [+2.5, +8.9]	40 / 75%

Other splits: leave one template family out +3.4 [+1.1, +5.5]; train on rounds 1-2, test round 3 +2.7 [0.0, +5.5]; leave the holdout set out +4.3 [+1.4, +7.2] at margin 0.5. Comparison with simpler models (pooled, nested): the engine's 12 numbers alone +1.9; plus the complaint features +2.9; the larger-smallest-text rule +0.9; an 8-unit MLP +2.5. When enabled in the engine on 1,162 evaluation tasks it changes 130 picks, keeps every layout valid and costs 2 ms per task. Ten rendered changes: six clearly or moderately better, two marginally better, one neutral, one worse (critic_report.md section 7).

Out-of-sample check on the learned layout family (2026-10-28, critic_report.md section 10). Judge round 7 compares the engine's pick without and with the learned layout family on 59 designs that no earlier round used (141 decided comparisons; this critic had never seen a learned layout). The rule pick alone agrees with the judge in 57.4% of them, this critic in 71.6% (+14.2 points, design-bootstrap 95% CI +6.2 to +23.4). A critic retrained on rounds 1-6 with the same margin and window also agrees in 71.6%, so these weights were kept. A version trained on all seven judge rounds is available as tools/judge/data/critic_r1_7.npz (margin 0.25); it is not enabled.

Limitations and risks

• Distilled from a single vision model's verdicts on about 600 decided pairs; its biases (larger type, reflowed layouts) are the model's.
• Round 1 and 2 pairs come from earlier engine versions; on the current engine the gain is smaller (+3.3).
• One uniform-stretch pair is preferred to the engine's pick in every held-out run; layouts far from the engine's candidate distribution can fool it. The finalist window (0.06 rule-score) limits this in the engine.
• About thirty model and recipe configurations were compared on the same labels; the reported numbers carry some selection optimism.
• Complaint features use heuristics (roles from geometry, an ink-width estimate, ids containing "cta" / "logo" when present); other design languages may break them silently. Report numbers (soft metrics) differ slightly between CoreText and FreeType.
• Not a safety, fairness or content model; it sees geometry and colours only, never text content.

Citation

Chameleon hackathon team, Dev 3: learned critic distilled from a strong vision judge, 2026-10-26. Report: tools/judge/critic_report.md.