Recognition of human emotion induced by visual contents of images
Creators: name: MOHSENI, Seyed Abdolreza
Publisher: name: RMIT University
Year: 2024
Subjects: classificationCode: 400607
subject: Signal processing
classificationCode: 460304
subject: Computer vision
classificationCode: 460306
subject: Image processing

description: In the past two decades, the number of images on the Internet and storage devices has greatly increased and people are more inclined to express and share their emotions using images than in the past. This has increased the demands in processing, searching, and indexing images based on the affective contents. Manually annotating the huge number of images based on their affective content is cumbersome, time-consuming and seemingly impractical. Therefore, there is a potential demand for automated analysis and annotation of images based on their affective content: Recognition of human Emotion induced by Visual Contents of images (REVC). The focus of this research is REVC, and it should not be confused with other emotion recognition systems such as facial emotion recognition.<br><br>

Subjectivity and affective gap are two main challenges in this field. Different approaches such as constructing standard emotion image datasets and emotion representation models have sorted out, to some extent, the issue of subjectivity. The focus of the REVC research to fill the affective gap has been mainly on identifying features that induce emotions and designing REVC systems that predict the induced emotions based on the content of images and to do so with higher accuracy and less complexity. <br><br>

The REVC systems that have been proposed are complex and/or rely more on using combined complex features. Simpler, faster and more efficient REVC systems are needed. Moreover, identifying the key-features for the REVC system is important. Although different levels of features in the REVC area have been proposed and used, it remains unknown which levels of features are more effective in the REVC area. <br><br>

The research focus so far has been on color images; however, it is possible to consider other types of images (e.g. binary images) for REVC systems. The importance and effect of classifiers and color spaces on the performance of REVC systems also need to be investigated. Identifying the relationship between each individual visual feature and the induced emotion must be examined. <br><br>

This thesis aims to investigate the effect of different numbers of bins in color histogram on the performance of REVC systems. The optimum number of bins in color histogram, the better-suited color space and classifier to detect the induced emotions from images are identified. This is examined using two different metrics to ensure generalization of the findings.<br><br>

Finding key-features in the field of REVC systems, and developing simpler and more efficient methods to recognize emotions, is another aim of this thesis. Moreover, this thesis shows that combining different complex features in REVC does not lead to better performance. Further, a new optimization method is proposed to enhance the performance of the REVC system. This optimization method is used to propose a novel REVC system, so called, CORT. The CORT system is simpler and more efficient than the existing REVC systems. This thesis shows that the CORT is robust enough to be used for different image datasets from different domains. <br><br>

Another new REVC system (based on color moments) is proposed in this thesis. The proposed color moment-based REVC systems use only single, four, or seven features, while the-state-of-the art uses over 100 features. The proposed color moment-based REVC systems are shown to be simpler and more efficient than the-state-of-the-art. <br><br>

This thesis also explores the effect of binary features and binary images in inducing emotions and then develops a binary-based REVC system to detect emotions. It is found that the binary features offer much more than what other researcher have realized. <br><br>

Finally, this thesis also explores the importance of different shallow features in REVC systems such as low-level, mid-level and high level features and challenges our current understanding about the effect of different level of features on the performance of REVC systems.<br><br>
This thesis paves the path for future research including the importance of binary images and binary features in the recognition of induced emotions.