## 2 Theoretical Background
### Comparing Human and CNN Attention Maps
#### 2.2.2 Influence of Tasks  
We use the term \"task\" to refer to the procedures of eliciting human attention maps. These procedures differ between studies in two nested ways: how directly they assess attention and how this assessment is implemented in specific cognitive activities. Concerning the first distinction, a rather direct assessment approach is to track people's eye movements during scene categorization, while a less direct approach is to let people manually select the image areas they consider relevant. Within these two general approaches, previous studies have used various cognitive activities: different categorization procedures or different means of manual selection. The following section is organized by the general assessment approaches and reviews the variety of specific activities within them. **Eye movements**. Eye tracking is often considered the gold standard for eliciting human attention maps and thus most studies have applied this method [8; 10; 48; 53; 54; 56; 57; 53]. In these studies, people's eye movements were tracked while they performed a wide variety of cognitive activities that differed on several dimensions. One such dimension is the _amount of experimental control_. On the one hand, there have been unrestricted tasks such as free verbal descriptions during routine radiological image reading [48; 53] or driving in natural environments [50]. On the other hand, there have been highly controlled tasks such as performing saccades to briefly presented images [10]. In between these extremes, various categorization procedures have been used such as verbal labelling [49; 56], keypress responses to choose between categories [8; 52; 57] or textual explanations of why the image matches a category label [51]. A second dimension on which previous tasks differed is the degree to which they encouraged people to _focus their eyes_ on a specific area or to broadly explore the image. Some tasks required attention to small details, such as the fine-grained classification of birds [8; 52] or the explicit description of features responsible for classification [51]. Other tasks diverted eye movements more broadly, such as naming as many objects as possible [49] or reading radiological images [48; 53]. A third dimension pertains to _restrictions of viewing time_. This ranged from extremely short times (e.g., 150 ms) that only permit one fixation [10] to medium times (e.g., 3 s) that allow for a brief inspection [8; 49; 56] up to unlimited time that enables people to thoroughly investigate the image [48; 50; 51; 53; 57]. However, not only the variability of tasks but also the _method of eye tracking per se_ must be evaluated critically. First, eye movements convey some information that is not relevant for scene categorization due to systematic viewing biases. Second, they fail to convey other information that actually is relevant because basic scene categorization can proceed without any need for eye movements. Moreover, attention maps generated from eye movements reflect people's search processes (instead of their decision as CNN attention maps do) and thus fixations may land on areas where the target might be located (based on contextual constraints) but actually is not: good guesses that upon closer inspection turned out to be wrong [20]. These tendencies might cast doubts on the suitability of eye movements to indicate which image areas people need for categorization. Thus, it is worthwhile to contrast them with other means of eliciting attention maps, which do not suffer from the same problems. **Manual selection**. Compared to eye tracking studies, the number of studies that elicited human attention maps via manual selection is smaller [9; 11; 45; 58; 57]. At the same time, their diversity is even larger. Manual selection studies differ in how people defined the areas relevant for categorization and in their degrees of freedom. One type of task is to _define the outlines of relevant areas_, either by drawing polygons [45] or by lassoing them [58].