# Inferring Emotion Tags from Object Images Using Convolutional Neural Network
## 4 Conclusions  
In this research work, we presented an efficient scheme for inferring emotion tag from object images. We pre-processed the Caltech-256 dataset and applied subjective evaluation from emotion labeling, in which we adopted the CNN model (ResNet-50) to extract deep features for emotion classification. A novel method is devised to infer emotion tags from visual stimuli based on image object categories rather than color and texture features. We trained and inferred nine emotion classes, including amusement, anger, awe, boredom, contentment, disgust, excitement, fear and, sadness, from object images. Moreover, we used gender-based analysis to see how emotion perception varies from males to females. Overall, the proposed method achieved an accuracy of approximately 85% for top-level and 79% for bottom-level emotion tag recognition, which showed the effectiveness of the proposed idea. In future, the proposed scheme can be expanded to take into consideration more user demographic information to better model and infer the human emotions. The proposed study and the output results can be utilized in the future studies for a wide-ranging application, such as recommender systems and emotion-based content generation and retrieval. Furthermore, considering the adaptability of the proposed mechanism, it can be used to depict or infer the mental state of a person, which in turn can be quite useful in medical fields for patient's treatment. Moreover, the proposed scheme can be used to model human behavior based on human-object interactions and emotion \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Study**} & \multirow{2}{*}{**Technique**} & \multirow{2}{*}{**Dataset**} & \multicolumn{2}{c}{**No. of**} & \multirow{2}{*}{**Performance**} & \multirow{2}{*}{**Results**} \\  & & & & **Categories** & & \\ \hline \multirow{4}{*}{[12]} & Noisy fine-tuned & \multirow{4}{*}{\begin{tabular}{c} CNN, ImageNet \\ CNN, fine-tuned \\ CNN \\ \end{tabular} } & Flickr dataset & \multirow{4}{*}{8} & \multirow{4}{*}{Accuracy} & 58\% with fine tunes, 32\% with \\  & & (23,000 images) & & & ImageNet and 46\% with Noisy \\  & & CNN & & & fine-tuned \\ \hline \multirow{4}{*}{[23]} & \multirow{4}{*}{CNN, PCNN} & Flicker data set & \multirow{4}{*}{2} & Precision, & \multirow{4}{*}{Accuracy is 72\% with CNN} \\  & & (half a million & & & Accuracy, F1 & and 78\% with PCCN \\  & & images) & & & measure & \\ \hline \multirow{4}{*}{[14]} & Off the shelf CNN, & \multirow{4}{*}{\begin{tabular}{c} Att photo (806 \\ CNN with \\ multi-scale pooling \\ \end{tabular} } & \multirow{4}{*}{8} & Art photo (806 \\  & & images) + Flickr \\  & & dataset (11,575 \\  & & images) & & positive Rate \\ \hline \multirow{4}{*}{Proposed} & Off the shelf CNN & \multirow{4}{*}{\begin{tabular}{c} Att photo (806 \\ CNN with \\ multi-scale pooling \\ \end{tabular} } & \multirow{4}{*}{9} & Accuracy, & \multirow{4}{*}{\begin{tabular}{c} Bottom level Accuracy is 79\% \\ Precision is 0.789 The recall is \\ 0.790, F1-score is 0.789. \\ \end{tabular} } \\  & & & & & \\ \cline{1-1}  & Off the shelf CNN & \multirow{4}{*}{\
\begin{tabular}{c} Caltech-256 (30,608 \\ images) \\ \end{tabular} } & \multirow{4}{*}{9} & Accuracy, & \multirow{4}{*}{
\begin{tabular}{c} Top-level Accuracy is 85\%, \\ precision is 0.851, The recall is \\ 0.850, F1-score is 0.850. \\ \end{tabular} } \\  & & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Comparison of the proposed method with some well-known deep learning-based inferring emotion tag schemes. recognition. With the increasing use of visual contents on social media platforms, the proposed scheme can help in analyzing the behavior of different communities towards some particular problem. For research articles Conceptualization, A.M., M.E.-u.-H., A.H.; methodology, A.M., M.E.-u.-H.; software, A.M., A.H.; validation, W.A., M.E.-u.-H.; investigation, M.U.A., M.E.-u.-H., M.A.K. ; resources, A.M.A. ; writing-original draft preparation, A.M., M.E.-u.-H., A.H., W.A. ; writing-review and editing, M.E.-u.-H., A.H., M.A.K.