MediaAttention-Ads v1

An attention model for ads, posters and interfaces. Given an image, it predicts where people will look in the first seconds, as a fixation-density heatmap. It is the default image model of MediaAttention, an open-source toolkit that turns the heatmap into ad metrics: focus, cognitive demand, and attention on the brand, product, headline and CTA.

📄 Paper: Predicting Where People Look at Ads: An Open Benchmark and a Compact Attention Model Fine-Tuned on Design Eye Tracking (arXiv submission pending).

On held-out posters it correlates with human eye tracking at CC 0.752, vs 0.496 for the best other open model (+52%), with non-overlapping 95% confidence intervals. It has 3.7M parameters and runs in 0.04 s per image on a laptop.

Use

pip install "mediaattention[all]"
mediaattention ad.png --aoi auto        # downloads this model on first use
mediaattention ui                       # local web UI

from PIL import Image
from mediaattention import score
from mediaattention.saliency import ads

heat = ads(Image.open("ad.png"))              # (H, W) fixation density, sums to 1
heat, report = score(Image.open("ad.png"))    # heatmap + ad metrics

Files:

• ads-v1.pth: state dict.
• unisal_config.json: constructor arguments (UNISAL's, unchanged).
• The model code is vendored in mediaattention/unisal/.

Input is RGB resized to one of UNISAL's fixed-aspect sizes and ImageNet-normalised; use the SALICON source branch with static=True. Output is a log fixation density at the input size.

Training

• Architecture: UNISAL (Droste, Jiao & Noble, ECCV 2020). It has a MobileNetV2 encoder, 16 learned Gaussian priors, a ConvGRU bypass (used only for video), a decoder with skip connections, and domain-specific adaptation and smoothing.
• Starting point: UNISAL's SALICON weights (pranikz/MediaAttention).
• Data: the UEyes training split (Jiang et al., CHI 2023, CC BY 4.0). It has 1,872 posters, mobile UIs, desktop UIs and webpages, with eye tracking from 62 participants (7 s free viewing, Gazepoint GP3). 10% of it was held out to choose the epoch.
• Never seen: the UEyes test split (108 images) and FiWI.
• Loss: KL(ground truth ‖ prediction) − CC, against fixations blurred by ≈1° of visual angle.
• Optimiser: AdamW, lr 3e-5, weight decay 1e-4, batch size 1, batch norm frozen.
• Run: 8 epochs, ~25 min on an Apple M3 Pro (MPS). The best epoch was 5, with held-out CC going from 0.502 to 0.769.
• Script: benchmarks/finetune_unisal.py.

Results

Metrics are from pysaliency, averaged per image. Scripts and per-image scores: benchmarks/.

Posters (UEyes test split, n = 27)

Model	CC ↑	95% CI	NSS ↑	KLD ↓
MediaAttention-Ads v1 (ours)	0.752	[0.695, 0.809]	1.55	0.354
TranSalNet-Dense	0.496	[0.440, 0.551]	0.98	0.792
DeepGaze IIE	0.458	[0.392, 0.524]	0.90	0.894
UNISAL (stock)	0.454	[0.389, 0.518]	0.90	0.913
DeepGaze MSDB	0.416	[0.351, 0.481]	0.86	1.079
Spectral residual	0.285	[0.209, 0.361]	0.58	1.082
Centre-bias baseline	0.281	[0.218, 0.344]	0.55	0.949

UEyes test split, all categories (108 images)

Model	CC ↑	NSS ↑	sAUC ↑	SIM ↑	KLD ↓	poster CC	mobile CC	desktop CC	web CC
MediaAttention-Ads v1 (ours)	0.735	1.53	0.731	0.680	0.399	0.752	0.820	0.692	0.674
TranSalNet-Dense	0.514	1.00	0.646	0.559	0.771	0.496	0.601	0.482	0.476
UNISAL (stock)	0.467	0.92	0.640	0.543	0.864	0.454	0.504	0.456	0.452
DeepGaze IIE	0.454	0.92	0.667	0.522	0.871	0.458	0.460	0.453	0.445
DeepGaze MSDB	0.452	0.94	0.690	0.499	1.039	0.415	0.530	0.465	0.397
Centre-bias baseline	0.219	0.40	0.499	0.447	1.021	0.281	0.189	0.190	0.215
Spectral residual	0.214	0.44	0.616	0.433	1.125	0.285	0.131	0.270	0.169

All 1,980 UEyes images (models as released; this model is left out because it trained on the train split)

Model	CC ↑	NSS ↑	sAUC ↑	SIM ↑	KLD ↓	poster CC	mobile CC	desktop CC	web CC
TranSalNet-Dense	0.534	1.01	0.655	0.586	0.700	0.523	0.586	0.543	0.483
UNISAL (stock)	0.502	0.97	0.656	0.573	0.779	0.494	0.521	0.526	0.468
DeepGaze MSDB	0.485	0.99	0.704	0.525	0.921	0.463	0.516	0.517	0.443
DeepGaze IIE	0.485	0.95	0.684	0.548	0.777	0.483	0.478	0.513	0.465
Spectral residual	0.232	0.45	0.623	0.463	1.006	0.257	0.198	0.290	0.185
Centre-bias baseline	0.220	0.37	0.497	0.468	0.963	0.257	0.201	0.198	0.225

Webpages: FiWI (149 pages, out of domain for every model)

Model	CC ↑	NSS ↑	sAUC ↑	SIM ↑	KLD ↓	pictorial CC	mixed CC	text CC
DeepGaze MSDB	0.676	1.54	0.748	0.642	0.462	0.751	0.686	0.592
DeepGaze IIE	0.649	1.50	0.729	0.629	0.504	0.726	0.680	0.541
MediaAttention-Ads v1 (ours)	0.631	1.27	0.685	0.617	0.500	0.653	0.643	0.597
UNISAL (stock)	0.616	1.31	0.661	0.613	0.575	0.676	0.641	0.531
TranSalNet-Dense	0.608	1.25	0.646	0.605	0.603	0.650	0.642	0.532
Centre-bias baseline	0.345	0.62	0.500	0.476	0.899	0.305	0.373	0.358
Spectral residual	0.209	0.40	0.562	0.449	1.052	0.220	0.249	0.159

Compared with other attention models

Model	Architecture	Params	Trained on	Video	Licence	s / image (M3 Pro)	UEyes test CC	FiWI CC
MediaAttention-Ads v1 (ours)	UNISAL: MobileNetV2 encoder, Gaussian priors, ConvGRU (video), decoder with skips	3.7M	SALICON → fine-tuned on UEyes (posters, ads, UIs)	yes (recurrent)	Apache-2.0	0.04	0.735	0.631
UNISAL (stock)	same as above	3.7M	SALICON, DHF1K, Hollywood-2, UCF Sports	yes (recurrent)	Apache-2.0	0.04	0.467	0.616
TranSalNet-Dense	DenseNet-161 CNN + transformer encoder + CNN decoder	76.6M	SALICON	no	MIT	0.10	0.514	0.608
DeepGaze IIE	ensemble of 4 backbones (DenseNet-201, ResNeXt-50, EfficientNet-B5, ShapeNet-ResNet-50) + readout	104.0M	SALICON → MIT1003	no	no licence file	0.67	0.454	0.649
DeepGaze MSDB	CLIP RN50x64 + DINOv2 ViT-B/14 at several scales + readout, dataset-bias parameters	451.6M	MIT1003, CAT2000, COCO-Freeview, DAEMONS, FIGRIM	no	no licence file	2.90	0.452	0.676
Neurons (commercial, for reference)	CNN encoder–decoder (ConvNeXt / ResNet-50 components), per its public tech paper	not disclosed	proprietary eye tracking (~20k participants)	separate video model	closed	–	not runnable	not runnable

Where it wins:

• It leads every UEyes category by 0.19–0.26 CC, and is best of all models on text-heavy webpages.
• It is 20–120× smaller and 17–70× faster than DeepGaze.
• It handles images and video in one network.
• Code and weights are Apache-2.0.

Where it doesn't: DeepGaze MSDB, at 120× the size, is still ~0.05 CC ahead on FiWI, and further ahead on photo-led pages.

Limitations

• One lab panel. UEyes is 62 participants, free viewing for 7 s. A one-second glance in a scrolling feed is different.
• Images only. MediaAttention's video mode keeps the stock UNISAL video weights (DHF1K), because there is no open ad-video eye-tracking set to tune them on yet.
• Bottom-up prediction. It predicts free-viewing attention, not task-driven search.

Roadmap

• Ad-video attention: tune the recurrent branch on ad video eye tracking.
• Feed context and 1–2 s exposures.
• v2: distil DeepGaze MSDB on photographs while keeping the design data.
• Panel-trained engagement and emotion models, from data collected with MediaAttention's open study kit.

Credits and licence

Apache-2.0. A fine-tune of UNISAL (Apache-2.0) on UEyes (CC BY 4.0). No UEyes images are redistributed; see NOTICE.

@misc{mahapatra2026mediaattention,
  title={MediaAttention: open predicted attention for ads},
  author={Mahapatra, Pratyush}, year={2026}, doi={10.5281/zenodo.23540409}
}
@inproceedings{droste2020unisal,
  title={Unified Image and Video Saliency Modeling},
  author={Droste, Richard and Jiao, Jianbo and Noble, J. Alison}, booktitle={ECCV}, year={2020}
}
@inproceedings{jiang2023ueyes,
  title={UEyes: Understanding Visual Saliency across User Interface Types},
  author={Jiang, Yue and Leiva, Luis A. and Rezazadegan Tavakoli, Hamed and Houssel, Paul R. B. and Kylm{\"a}l{\"a}, Julia and Oulasvirta, Antti},
  booktitle={CHI}, year={2023}, doi={10.1145/3544548.3581096}
}

By Pratyush Mahapatra · Site · Paper (PDF) · Report (PDF)