Linguistic-Mediation of Visual Saliency: Using ROC to Compare Multi-Dimensional Datasets
Channel: Berkeley Institute for Data Science (BIDS)
Date: 21/04/2017

BIDS Data Science Lecture Series | April 21, 2017 | 1:10-2:30 p.m. | 190 Doe Library, UC Berkeley

Speaker:  Stephanie Huette, Assistant Professor, University of Memphis

Models of visual saliency have been used for decades to predict eye movements, in many cases very accurately (Itti, Koch, & Niebur, 1998). However, eye movements change as a function of task (Yarbus, 1967), which often includes instruction or questions delivered through speech or text.There are an infinite number of combinations of visual scenes and linguistic input, so characterizing every situation is impossible. Instead, principles of how language might mediate the salience of an image should be developed so more dynamic, interactive models of eye-movement patterns can be established. Two experiments, receiver operating characteristics (ROC) analyses, and potential applications to other areas that work with multi-dimensional datasets will be discussed. The ROC analysis is useful for evaluating data transformations, the accuracy of classification algorithms, individual or strategic differences in human-generated data, and much more.