# Robot Control Platform for Multimodal Interactions with Humans Based on ChatGPT
## 2 Related Research  
The research at Opole University of Technology [11] has focused on enhancing Pepper's capabilities for front-of-house applications. In this study, advanced modules, including an external video module and a speech to text recognition module, were integrated to facilitate human-computer interaction. The research demonstrated how these enhancements improved Pepper's communication skills and enabled the implementation of an automatic reception control system, addressing challenges such as mechanical noise interference. This work highlights the significance of external speech recognition modules in enabling natural language interactions, emphasizing the need for noise interference mitigation. This study, therefore, lays a foundation for complex human-machine collaboration systems, demonstrating the potential for integrating robots into everyday tasks. Similarly, a collaboration between Scaled Foundations and Microsoft Autonomous Systems and Robotics Research [12], introduced an innovative strategy that emerges to seamlessly integrate ChatGPT into robotics workflows. This research utilized language label embedded models (LLMs) and multimodal modeling for tasks like visual language navigation and human-robot interaction. The deployment pipeline incorporated diverse prompting technologies supported by a high-level function library, illustrating ChatGPT's proficiency in various robotic tasks. The iterative development cycle, which involved user feedback, ensured continuous refinement and optimization for effective deployment. This approach suggests a significant leap toward integrating advanced AI tools into practical robotic applications. Furthermore, a study from Stanford University [13] examined the use of generative agents as platforms for rehearsing interpersonal communication. These agents, which engage with users through avatar-based interactions, were found to improve communication skills and offer significant potential for applications in social science research. By interacting with users in natural language, generative agents can store and recall experiences, enabling them to dynamically plan and adapt their behavior. This dual function--both as a tool for testing social theories and as a practical aid for communication skill development--positions generative agents as valuable resources for studying and navigating complex social dynamics. Their ability to facilitate more natural dialogue in human-computer interactions also suggests broader applications for enhancing communication and understanding between humans and intelligent systems across various contexts. In the era of Industry 4.0 [14], the integration of AI, IoT, and big data is transforming manufacturing into \"smart factories.\" However, the adoption of AI in these environments faces significant challenges. One of the main obstacles is access to large, diverse datasets that are necessary to effectively train AI models [15]. The complexity of industrial processes complicates data collection, leading to challenges in data volume and diversity [16]. In addition, the introduction of AI into industrial environments raises concerns about the reliability and robustness of these systems, as modern industrial control systems are inherently complex, less deterministic, and therefore, more difficult to predict and control [17]. These issues highlight the need for rigorous testing and validation to ensure the safe and effective deployment of AI technologies in industrial environments. ChatGPT can be used on many different levels and different ways in robotics and human-robot interaction. It can be used as natural language processing tool with powerful question and answer (Q&A) capabilities available to end users as an interface, similar to our approach. On the other hand, in [18], [PERSON] et al. propose using ChatGPT as a tool for programmers to streamline robot programming and, at the end goal, support non-technical users to program robots.