Visual Attention is Attracted by Text Features Even in Scenes without Text

Visual Attention is Attracted by Text Features Even in Scenes without Text Hsueh-Cheng Wang (hchengwang@gmail.com) Shijian Lu (slu@i2r.a-star.edu.sg) Joo-Hwee Lim (joohwee@i2r.a-star.edu.sg) Marc Pomplun (marc@cs.umb.edu) Department of Computer Science, University of Massachusetts at Boston, 100 Morrissey Boulevard, Boston, MA 02125 USA Institute for Infocomm Research, A*STAR, Singapore 1 Fusionopolis Way, Singapore 138632 Abstract Previous studies have found that viewers’ attention is disproportionately attracted by texts, and one possible reason is that viewers have developed a “text detector” in their visual system to bias their attention toward text features. To verify this hypothesis, we add a text detector module to a visual attention model and test if the inclusion increases the model’s ability to predict eye fixation positions, particularly in scenes without any text. A model including text detector, saliency, and center bias is found to predict viewers’ eye fixations better than the same model without text detector, even in text- absent images. Furthermore, adding the text detector – which was designed for English texts – improves the prediction of both English- and Chinese-speaking viewers’ attention but with a stronger effect for English-speaking viewers. These results support the conclusion that, due to the viewers’ everyday reading training, their attention in natural scenes is biased toward text features. Keywords: real-world scenes; text detector; eye movements; visual attention. Introduction When inspecting real-world scenes, human observers continually shift their gaze to retrieve information. Viewers’ attention has been found to be biased toward visually salient locations, e.g., high-contrast areas, during scene viewing or search (Itti & Koch, 2001) or toward the center of the screen when viewing scenes on computer monitors (Tatler, 2007). Since it is also known that viewers pay a disproportionate amount of attention to faces (Cerf, Frady, & Koch, 2009), Judd, Ehinger, Durand, and Torralba (2009) equipped their model of visual saliency with a face detector (Viola & Jones, 2004) and a person detector (Felzenszwalb, McAllester, & Ramanan, 2008). In those images that contained depictions of people, their model with all features combined outperformed models trained on typical saliency features such as color, orientation, intensity, and contrast. Cerf et al. (2009) refined the “standard” saliency model by adding a channel of manually-defined regions of faces, texts, and cellphones, and demonstrated that the enhancement of the model significantly improved its ability to predict eye fixations in natural images. Besides depictions of people, texts in natural scenes are usually important pieces of information, which could be shown on depictions of signs, banners, advertisement billboards, license plates, and other objects. Human text detection in natural scenes is critically important for people to survive in everyday modern life, for example, by drawing attention to traffic signs or displays showing directions to a hospital or grocery store. Our previous studies (Wang & Pomplun, 2011; under revision) suggested that attention seems disproportionately attracted by texts but that the specific visual features of texts, e.g., edge density, rather than typically salient features such as color, orientation, intensity, or contrast, are the main attractors of attention. This finding was in line with the results in Baddeley and Tatler (2006) that high spatial frequency edges, not contrasts, predict where we fixate. Automatic text detection has been a hot topic in the fields of computer vision and pattern recognition for its practical applications. The special features of texts, e.g., the small variation of the stroke width (see Epshtein, Ofek, & Wexler, 2010; Jung, Liu, & Kim, 2009) or edge density (Lu, submitted) have been used to develop text detectors. Although many text detection techniques, i.e., texture- based, region-based, and stroke-based methods, have been reported, many non-text objects, such as windows, fences, or brick walls, easily cause false alarms (see Lu, submitted; Ye, Jiao, Huang, & Yu, 2007, for a review). Furthermore, many established text detectors are restricted under commercial patents. Therefore, only few text detectors are freely available or have been tested in visual attention studies. Lu, Wang, Lim, and Pomplun (submitted) developed specialized text features, e.g., histograms of edge width and edge density, trained with Support Vector Machine (SVM) classifiers. The study reported better performance compared with earlier studies (e.g., Epshtein, et al., 2010; Jung, et al., 2009) on public text-detecting competition datasets (ICDAR2003 and ICDAR2005). In the present study, we used the automatic text detector developed by Lu et al. (submitted) to test whether it can improve the prediction of viewers’ fixations. This detector employs contrast of strokes over background, width of strokes, joints of horizontal and vertical strokes, and stroke structure as key variables Although manually-defined regions of texts were shown to improve the prediction of eye fixations in text-present images (Cerf et al., 2009), it is unclear if viewers’ attention is biased toward any non-text objects which share some features of texts, particularly in text-absent images. In the present study, two eye-movement datasets obtained in our previous investigations (Wang & Pomplun, under revision)