## 6 Conclusions and Future Work  
This study addresses several fundamental challenges in human-robot interaction (HRI), particularly in the domains of emotional understanding, contextual reasoning, and decision making. By integrating multimodal emotion recognition capabilities into the Pepper humanoid robot, our approach combines visual, auditory, and textual data to provide a more holistic understanding of user emotions. This enables Pepper to respond with greater sensitivity and contextual awareness, representing a significant advancement in creating robots capable of empathetic and meaningful interactions. Compared to previous studies, which often rely on single-modal approaches or focus on individual components such as speech or facial recognition, our work introduces a unified framework that synthesizes data from multiple sources. This integration not only enhances the robustness and adaptability of emotion recognition systems but also  
Figure 5: Accuracy graph of the fine-tuned auditory emotion recognition model with the personalized dataset. addresses broader scientific problems related to the perception and reasoning capabilities of robots. By leveraging an LLM, the system dynamically interprets multimodal inputs and contextual cues, demonstrating the potential for robots to act empathetically and intelligently in real-world scenarios. Despite these advancements, several limitations must be acknowledged. First, the hardware constraints of Pepper, including outdated sensors and limited computational power, pose challenges to achieving real-time performance and scalability. Second, the process of integrating new users requires manually creating personalized datasets, which can be labor-intensive and difficult for users to replicate, especially when mimicking specific emotions. Lastly, while the system performs well with mid-range devices, its generalizability to broader demographics and diverse use cases remains to be tested. These limitations highlight areas for improvement and guide future research directions. To address these shortcomings, future work should focus on integrating the pipeline into more advanced humanoid robots equipped with state-of-the-art sensors and enhanced hardware capabilities. Additionally, incorporating automated dataset generation tools and leveraging hybrid datasets--combining personalized and standardized data--could improve both scalability and generalization. The adoption of advanced vision and language models, such as large language models (LLMs) and more sophisticated scene description systems, will also be crucial for maintaining state-of-the-art performance as technology evolves. Furthermore, this study contributes to understanding how robots can use emotional information to guide decision making, such as by adjusting tone, posture, or conversational content based on a user's emotional state. By addressing the complexity of recognizing and reasoning about human emotions in context, this work highlights the broader scientific implications for designing robots capable of human-like interaction. In conclusion, this study lays the groundwork for the next generation of emotionally intelligent robots, bridging the gap between human-like interaction and real-world applicability. The integration of multimodal emotion recognition not only advances the current state of HRI but also underscores the potential of robots to become empathetic collaborators in diverse environments. As the field of humanoid robotics continues to evolve, the innovations demonstrated in this work represent a critical step toward a future where robots seamlessly coexist with humans, offering meaningful and personalized support in daily life. Conceptualization, M.C. and F.G.-D.; methodology, F.G.-D.; software, S.G.; validation, S.G.; formal analysis, F.G.-D.; investigation, F.G.-D. and S.G.; resources, M.C. ; writing--original draft preparation, S.G. and F.G.-D.; writing--review and editing, F.G.-D. and M.C. ; supervision, M.C. ; project administration, F.G.-D.; funding acquisition, M.C.