DOI: 10.5061/dryad.3fs556j
title: Predicting human complexity perception of real-world scenes

description: Perceptual load is a well-established determinant of attentional
 engagement in a task. So far, perceptual load has typically been
 manipulated by increasing either the number of task-relevant items or the
 perceptual processing demand (e.g. conjunction vs. feature tasks). The
 tasks used often involved rather simple visual displays (e.g. letters or
 single objects). How can perceptual load be operationalised for richer,
 real-world images? A promising proxy is the visual complexity of an image.
 However, current predictive models for visual complexity have limited
 applicability to diverse real-world images. Here we modelled visual
 complexity using a deep convolutional neural network trained to learn
 perceived ratings of visual complexity. We presented 53 observers with
 4000 images from the PASCAL VOC dataset, obtaining 75,020 2AFC paired
 comparisons across observers. Image visual complexity scores were obtained
 using the TrueSkill algorithm. A CNN with weights pre-trained on an object
 recognition task predicted complexity ratings with r=0.83. In contrast,
 feature-based models as used in the literature, working on image
 statistics such as entropy, edge density and JPEG compression ratio, only
 achieved r = 0.70. Thus, our model offers a promising method to quantify
 the perceptual load of real-world scenes through visual complexity.
description: Open data provided with
 Nagle & Lavie 2019 (Predicting human complexity perception of
 real-world scenes).This readme describes
 the open data provided with Nagle & Lavie 2019 (Predicting human
 complexity perception of real-world scenes). The PASCAL VOC images are
 available at
 http://host.robots.ox.ac.uk/pascal/VOC/voc2012/index.html#devkit under
 training/validation data. The notebooks folder holds Python notebooks used
 for analysis, including some helper files. - Complexity - Inception model:
 training and evaluation of the predictive model of visual complexity -
 Corchs replication: evaluation on our images of the model of Corchs et al
 - Feature integration: integration of pre-calculated image features and
 correlation against complexity The data folder contains data: - measures:
 Python dictionaries of various image statistics indexed by image filename,
 saved as Python Pickle objects. - mus_by_url.pik: dictionary of image
 complexity values output (mu) from TrueSkill, indexed by image filename -
 all_comparisons.pik: list of dictionaries showing all comparisons'
 images and outcomes - Visual search difficulty data from Ionescu 2017 is
 included The models folder contains the final trained model, saved as an
 .h5 file which is loadable by Keras. The Inception notebook shows how to
 load and run this model. The futils folder contains some utilities
 necessary for visualisation. This code is intended for use with Python 2
 on a GPU-equipped system (we used a Titan X card with 12 GB of VRAM). Deep
 learning models are implemented with Keras; we used the Theano backend,
 which is now retired. Some installation of libraries and changing of paths
 will be required to run this code on your system. For further information,
 please contact fintan.nagle@ucl.ac.uk.open-data.zip