Collective indexing of emotions in images. A study in emotional information retrieval

ion Levels of Tags, Prototypes, and Cognitive Reference Points User-generated tags can be located on different levels of abstraction. In empirical psychological studies, Eleanor Rosch and her colleagues (Mervis & Rosch, 1981; Rosch, 1975a; Rosch, 1975b; Rosch & Mervis, 1975; Rosch, Mervis, Gray, Johnson, & Boyes-Braem, 1976) could show that there are three abstraction levels and that the middle level— the basic level—is most frequently used because here “the information value of attribute clusters is maximized” (Mervis & Rosch, p. 92). Concerning Rosch, the three levels of abstraction are called subordinate, basic, and superordinate: Suppose that basic objects (e.g., chair, car) are the most inclusive level at which there are attributes common to all or most members of the category. Then total cue validities are maximized at that level of abstraction at which basic objects are categorized. That is, categories one level more abstract will JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY—May 2009 865 DOI: 10.1002/asi be superordinate categories (e.g., furniture, vehicle) whose members share only a few attributes among each other. Categories below the basic level will be subordinate categories (e.g. kitchen chair, sports car) which are also bundles of predictable attributes and functions, but contain many attributes which overlap with other categories (for example, kitchen chair shares most of its attributes with other kinds of chairs). (Rosch et al., 1976, p. 385) Basic level concepts are most useful in human life: Universally, basic object categories should be the basic classifications made during perception, the first learned and the first named by children, and the most codable, most coded, and most necessary in the language of any people (Rosch et al., 1976, p. 435) In the context of image indexing, Rorissa (2008, p. 1744) states: “Among these three levels, people, in general, prefer to use the basic level when they are thinking about the world around them. ” Here, one research question arises: Are in fact most of the user-created tags that describe images basic-level terms? A problem in practical research is arising:What are the conditions in which a given term can be assigned to the basic level (or superordinate or subordinate)? Green (2005) applies WordNet (Fellbaum, 1998), which is a lexical database for English terms that arranges the concepts (nouns and verbs, but not adjectives) in a hierarchical way. She is able to show that there is some regularity for the indication of the abstraction level of a noun, e.g., “if a word is longer than 15 characters, it is unlikely that it names a basic level category; if a lexical unit is a phrase, . it is unlikely that it is names a basic level category; . if a concept has more than four levels beneath it, it probably names a superordinate level category; if a concept has no children, it is unlikely to be basic level, but is much more likely to be a subordinate concept” (Green, 2005). Rorissa and Iyer (2008, p. 1392) add the following criteria besides WordNet: “terms for which one can think of several manifestations (examples)” for superordinate concepts; “convey information on the parts and components of objects” for basic concepts; and “describes a particular kind of a basic level object” for subordinate concepts. Green’s, Rorissa’s, and Iyer’s analyses do not deal with emotional concepts, so further research is necessary to adjust their findings to emotions. For a given concept, there are some objects that can be seen as prototypes. A prototype is the clearest case or a very good example for a concept. Prototypical members of a category are “those with most attributes in common with other members of that category and are those with least attributes with other categories” (Rosch & Mervis, 1975, p. 576). For example, a robin is a prototype for the concept bird, while a penguin is not (penguins fail to fly, which is common for most birds; Jörgensen, 2003, p. 40). So, we meet the problem relating to image tagging: Given a basic emotion, do prototypical images exist? Is it possible to measure prototypicality of images relative to fundamental emotions? If there are indeed good fitting images or even prototypical images, then it would be an interesting analysis to separate those cognitive reference points (Rosch, 1975b) that lead the users to tag a certain term or to adjust the scroll bar on a high value for the intensity of a basic emotion. Eleanor Rosch (1975b, p. 545) asks: “Do other natural categories, less obvious than colors, lines, or numbers, have stimuli which serve as reference points? ” An example of a reference point is a landmark that people use to navigate through cities. Here, another research area becomes visible: Which are the landmarks of happiness, sadness, etc., on prototypical images? If we are able to find such cognitive reference points, these points can be used as a base for content-based image retrieval. But this research is far behind the study presented here and has to be considered elsewhere.