NagaYu/sludge-small

Sludge — 12 M UI-tree dark-pattern detector

This model does not determine legality.

It reports provisions that may be implicated and the screen elements that are the factual basis for looking at them. Whether a provision is actually engaged depends on facts no UI tree contains — the purposes of processing, the legal basis relied on, the audience, the rest of the journey, prior consent, sector rules — and is an assessment for a qualified human.

It has no feature that labels a named company's product as unlawful, and it is not built to acquire one. It is a tool for reviewing interfaces, not for publishing verdicts about them.

Sludge reads a UI tree — an accessibility tree, a DOM plus computed style, an AX hierarchy — and returns, for each finding, a three-tuple:

provision that may be implicated	the element of that provision at issue	evidence
Directive 2011/83/EU Art. 22	express_consent_not_default_option	cook_consent_marketing_9

…plus what to change, measured against the screen itself:

Render 'Reject all' at 200×52 px with 16 px type, matching 'Accept all' (currently 95×29 px, 9 px).

• Dataset: NagaYu/sludge-ui-counterfactuals
• Demo: NagaYu/sludge

Why a UI tree instead of a screenshot

On-device the tree is already there, and it gives exact strings, geometry, colours, default selection state, hierarchy and link targets. A screenshot pipeline has to recover all of that, and gets some of it wrong.

That is the project's central claim, and the benchmark tests it rather than assuming it: the same rule analyser was run on the exact tree and on a real rendered screenshot (OpenCV element detection, colour and checkbox-state recovery from pixels, text through an OCR channel).

same analyser, different input	micro-F1	ms/screen
on the exact UI tree	0.767	1.36
on a rendered screenshot	0.228	21.14

Swapping the tree for a screenshot costs 0.539 micro-F1. Of that, 0.233 is text-recovery error alone; the rest is lost hierarchy, guessed roles, and link targets that pixels simply do not contain.

Results

Held-out layouts (test) and unseen class combinations (hard). Split is by UI template, never by sample, so no layout is shared across the boundary.

system	micro-F1 (test)	macro-F1	localisation	ms/screen	micro-F1 (hard)
Rules on the exact tree	0.767	—	—	1.36	0.847
Screenshot CV+OCR (same analyser)	0.228	—	—	21.14	0.326
Sludge	0.917 (0.908–0.926)	0.894	1.000	6.77	0.849

Localisation = the returned evidence intersects the ground-truth element set, over correctly classified pairs. Ranges are 95% percentile bootstrap intervals over records.

The two F1 columns are not comparable to each other — hard screens carry more patterns each, and micro-F1 rises with the positive rate. Compare systems within a column.

Size / accuracy / cost

checkpoint	parameters	epochs	micro-F1	localisation	ms/screen
sludge-tiny	0.9 M	8	0.863	1.000	1.48 ms
sludge-small	12.0 M	8	0.917	1.000	6.44 ms
sludge-base	101.7 M	4	0.892	1.000	33.55 ms

Not a clean scaling study — the larger checkpoint got fewer epochs on a contended machine, so a lower score at a larger size means undertrained, not bigger is worse. sludge-small is the published default because it is the accuracy/cost optimum here and meets the millisecond-per-screen requirement that sludge-base does not.

What is in this repo: sludge-small (root, plus every export format) and sludge-tiny/. sludge-base weights are not published — it is undertrained and sludge-small dominates it on every axis, so shipping 388 MB would be misleading rather than useful. Its row above is a real measurement; reproduce it with python scripts/train.py --preset base --epochs 8.

It reacts to the edit, not the template

The corpus is built from counterfactual pairs: a fair screen, and the same screen with one thing changed. On held-out layouts, 24 of 27 classes cross their own decision threshold on that single edit.

It can be moved to an operating point

profile	target FPR	rules	screenshot	Sludge
triage	10%	0.928	0.394	0.987
review	5%	0.928	0.394	0.981
audit	1%	0.928	0.076	0.967
distinct scores available		7	8	1356

The rule engine returns the same recall at 1% and 10% because its scores take a single-digit number of distinct values — there is no threshold between those points that separates anything. That is the operational difference between a rule engine and a calibrated model, independent of F1.

Exported runtimes, measured

Forward pass, one screen, 1 CPU thread on arm. Every format is exported and run, then compared to PyTorch on whether it makes the same decisions.

format	ms/screen	size	decisions matching PyTorch
PyTorch	10.70 ms	48.0 MB	—
ONNX	33.24 ms	48.3 MB	1.0000
ONNX int8	13.34 ms	12.4 MB	0.9898 (11/40 screens flip)
Core ML (fp16)	2.36 ms	24.2 MB	1.0000
GGUF (f16)	—	24.0 MB	1.0000

End to end — featurise, forward, decode — 4.91 ms p50, 5.96 ms p95 per screen. That is the number the benchmark uses.

Two things worth knowing before you pick a runtime.

The exported graphs have a static node axis (fixed at max_nodes; the caller pads and masks). A dynamic node axis looks like it works — it matches PyTorch on the screen it was traced with — and then raises on any screen of a different size, because the attention reshapes bake in the traced node count. So ONNX pads every screen to 128 elements whether it has 13 or 128, which is why its wall time here exceeds PyTorch's. Re-export at a smaller fixed axis if your screens are small; the padding is verified output-preserving either way.

This model does not survive int8. Dynamic quantisation — per-tensor, per-channel, MatMul-only, QInt8 and QUInt8 alike — flips at least one screen-level decision on 11 of 40 sample screens, with probability differences up to 0.998. An earlier version of this card quoted 0.995 agreement for int8; that figure was measured over the raw per-node logit tensor, where almost every entry is confidently negative and agrees, and it hid the flips. The table above now reports agreement on the screen-level decision the detector actually thresholds. fp16 is not an alternative either: onnxruntime-web's WASM build rejects float16 attention ops. Use fp32 unless you have measured that quantisation is safe for your screens.

On GGUF, plainly: the file is a valid GGUF v3 container and round-trips through sludge.gguf, but llama.cpp cannot execute it — this is not one of its graph architectures. It is a portable weight container for the accompanying loader, not a drop-in llama.cpp model.

Usage

pip install "sludge[all] @ git+https://github.com/<you>/sludge"
sludge check tree.json --profile review --jurisdiction eu

from huggingface_hub import snapshot_download
from sludge.model import SludgeDetector
from sludge.provisions import ProvisionMapper
from sludge.remediate import RemediationSuggester
from sludge.uitree import tree_from_html

detector = SludgeDetector.load(snapshot_download("NagaYu/sludge-small"))
tree = tree_from_html(open("page.html").read())

detections = detector.detect(tree)                       # class + evidence element ids
hits = ProvisionMapper.load("eu").map(detections)         # provisions that MAY be implicated
fixes = RemediationSuggester().suggest(tree, detections)  # what to change, with target values

for h in hits:
    print(h.citation, [e["id"] for e in h.elements], h.evidence)

The taxonomy (intermediate representation)

7 categories / 27 classes, all from published work — nothing here is coined by this project. Per-class provenance ships in sludge/taxonomy.py and prints with sludge taxonomy.

Category (Mathur et al. 2019)	Classes
Sneaking	sneak_into_basket · hidden_costs · hidden_subscription · bait_and_switch · hidden_information
Urgency	countdown_timer · limited_time_message
Misdirection	confirmshaming · visual_interference · false_hierarchy · preselection · trick_question · pressured_selling · disguised_ad
Social Proof	activity_message · testimonials_uncertain_origin
Scarcity	low_stock_message · high_demand_message
Obstruction	hard_to_cancel · price_comparison_prevention · intermediate_currency · privacy_maze · dead_end
Forced Action	forced_enrollment · forced_registration · privacy_zuckering · nagging

Sources: Mathur et al. (2019) CSCW · Gray et al. (2018) CHI · Brignull, deceptive.design · EDPB Guidelines 03/2022 · OECD (2022) DEP No. 336 · Gray et al. (2024) CHI. Where this project had to place a class its source work organises differently, the class records that as an aggregation decision rather than presenting it as a citation.

Provision mapping is configuration

Classes map onto the constituent elements of provisions, from swappable YAML tables shipped for eu, us_federal and us_ca. Point --config-dir at your own directory and every output changes with no code change.

The shipped us_federal.yaml carries the worked example: the FTC Negative Option Rule (16 C.F.R. Part 425) was finalised and then vacated in its entirety by the Eighth Circuit in July 2025, while ROSCA beneath it stayed in force. Here that is one enabled: false line plus a status note. A tool with the table compiled in would still be asserting it.

Training

Trained only on synthetic screens. No screenshot or UI tree from a real application is used for training. Public dark-pattern datasets are for evaluation only, under their own terms, via scripts/eval_real_data.py, which refuses to run without an explicit licence acknowledgement.

Selection and per-class thresholds use a dev split of two held-out training templates, never the test set. Node order is shuffled during training so the positional embedding cannot fingerprint a layout.

Limitations

• The corpus is synthetic, with bounded lexical and layout variation. Read the numbers as relative between systems, not as an estimate of field accuracy.
• Flow-scope classes need a flow. Given a single screen, hard_to_cancel, privacy_maze, nagging, dead_end and forced_registration have no evidence available; Sludge reports nothing for them rather than guessing.
• A quiet report is not a clean bill of health. It means nothing crossed the threshold the chosen profile fixes.
• The provision tables are a starting point, written against operative articles, not recitals or national transpositions. Verify against consolidated text before relying on them.
• The multimodal-LLM comparison arm was not run here — without an API key it contributes no scores, and the figures mark its published reference point with an open marker rather than inventing numbers.

Citation

@software{sludge2026,
  title  = {Sludge: deceptive design pattern detection on UI trees},
  year   = {2026},
  url    = {https://huggingface.co/NagaYu/sludge-small},
  note   = {Reports provisions that may be implicated; does not determine legality.}
}

Licence: Apache-2.0. The provision tables are configuration, not legal advice.