# Multimodal Emotion Recognition Using Visual, Vocal and Physiological Signals: A Review
## 2 Materials and Methods
### Study Selection  
After removing duplicates, we proceeded to the screening phase, where we applied inclusion and exclusion criteria to assess the relevance of each record to this review study. We retained only those records that directly addressed the research question. The inclusion/exclusion criteria focused on relevance, study type, completeness, and publication language. We excluded records related to textual modality when this modality was part of a bimodal emotion recognition system, as this review does not address textual inputs. Additionally, we removed editorial reviews, internal reviews, incomplete studies, extended abstracts, and white papers. To capture relevant developments and recent advancements, we set the publication range from 2012. This time frame allowed us to provide a comprehensive overview of current trends, methods, and findings while maintaining the review's relevance and up-to-date status. It also facilitated the identification of current research gaps.  
Figure 2 presents the PRISMA flow diagram demonstrating the study selection process used in this review. The query initially yielded 15,371 records without duplications across all the databases. This number is skewed due to the number of SpringerLink records being high, because this query searches the full text. During the screening phase based on titles and abstracts, the abstracts underwent scrutiny. we excluded non-English articles, studies published before 2012, incomplete studies, and studies that utilize textual modality in a bimodal framework, reducing the record count to 983 for detailed examination. Upon a thorough full-text review of these papers, we further excluded 716 records. The exclusion criteria included the absence of significant insights (often found in short conference papers) or exclusive focus on unimodal techniques. In the end, this study selection process led to the inclusion of 267 records in the final study. Our meticulous selection ensures that the records included in our review are highly relevant to our research questions and objectives, providing a comprehensive understanding of multimodal emotion recognition and addressing any existing gaps in the literature.  
Finally, we thoroughly examined the selected papers to extract essential insights, identify trends, and draw conclusions. This examination explored various modalities, data collection methods, fusion techniques, and machine learning approaches related to multimodal emotion recognition. By doing so, we effectively addressed our research questions and better understood the field's current state. This process also revealed potential gaps or shortcomings in the existing literature, paving the way for future research. It enabled us to delve deeper into the recognition of subtle expressions, spontaneous emotions, and dynamic displays and assess method accuracy and robustness. Examining various study types provided valuable insights into the practical implementation of multimodal emotion recognition, ensuring that our research remains relevant and applicable to real-world scenarios.