# Inferring Emotion Tags from Object Images Using Convolutional Neural Network
## 1 Introduction
CNN-based models tend to produce a more accurate and robust result as compared to the standard machine learning algorithms in many computer vision applications. These applications include digit recognition, speech/pronunciation and audio recognition, semantic analysis, and scene recognition [30; 31; 32; 33; 34; 35]. The research study in [28] demonstrated that deep learning-based CNN architecture is the most astonishing and off-the-shelf tool for feature extraction. Generally, the information contained in the images includes diverse kinds of living object categories, such as flowers, birds, animals, and insects, etc. In contrast, the categories of the non-living objects included in the images contain household items, vehicles, buildings, sceneries, etc. According to [36], these objects divulge various types of emotions on human beings, and a person might have some emotional reactions when interacting with these objects. For instance, an image that contains some eatable items, like pizza, sushi spaghetti, etc., generally perceives emotions, such as excitement, amusement, or disgust. Likewise, a person may perceive an emotion such as contentment, excitement, or amusement while watching an image of a flower, tree or beautiful scenery. Similarly, a person can regard an emotion tag as either amusement by viewing a picture of a snowmobile or a heavy motorbike. Hence, the exclusive information about a person's behavior can be acquired by interpreting their emotion perception towards different real-life object categories. In this aspect, user demographic information can also be taken into account for emotion recognition. The psychological and behavioral studies proved that the human perception of emotion varies concerning user demographics [8]. Only a few research studies have shown interest in classifying emotions based on user demographics that generally refer to user attributes such as age, gender, marital status, and profession. In [37; 38], the authors reported a gender discrimination scheme for image-based emotion recognition. [PERSON] et al. [39] analyzed the demographics that are widely used to characterize customer's behaviors. Likewise, [PERSON] et al. [38] reported that diverse genders and age groups have different emotion perceptions. Besides, men invoke powerful emotions (such as anger), and women have powerless emotions (such as sadness and fear). Likewise, the authors in [37] mentioned that females are more expressive in emotions as compared to males. Therefore, it can be realized that multiple emotion classification based on object-related features and demographic analysis is a challenging task. For addressing the challenges discussed above, in this study, we explore the implications of inferring emotion tags from images based on object analysis and categorization. Human-object interaction (either in real-life or through object images) entails varying emotion perceptions for different users, as shown in Figure 1. Different objects may produce varying emotions on the viewers depending upon their judgment. Hence, it seems advantageous to infer a viewer's emotions based on visual object interaction through images. In this aspect, based on the majority voting principle, we can establish a standard way to annotate emotions associated with different object categories. Thus, our proposed scheme is envisaged on the hypothesis that we can accurately infer emotion tags from the images using object-related features to alleviate the existing challenges associated with visual features (such as image contrast, brightness, and color information). Thus, to validate the proposed idea, we used a publicly available Caltech-256 dataset that consists of a hierarchy of object categories, including 20 top-level and 256 bottom-level object categories. We pre-processed this dataset for assigning emotion tags to these object categories. To the best of our knowledge, no one else has reported such research work that infers the emotion tags from images based on object analysis.