## 3 Development and Results of the Multi-Party Dialogue Management  
### Integration of Multi-Party Person Detection  
Although interactions with individual users are generally robust, group dynamics can significantly affect the robot's ability to detect and engage with humans [24]. The Furhat robot features person detection that is based on facial recognition. However, in scenarios where users face or obstruct each other without making eye contact with the robot, there can be challenges, as this often results in disruptions during multi-party interactions, where  
Figure 3: Architectural sketch of our social robot, with the corresponding subsystems. Own illustration. individuals may obscure each other or engage in side conversations, leading to errors in person detection. Therefore, we used the YOLOv5m model [40]. In contrast to facial recognition systems, the YOLOv5m excels in detecting individuals, whether fully or partially visible, from various camera angles (i.e., egocentric and exocentric from the robot [31]). Using CNN architecture, the YOLO models are capable of making bounding-box and class probability predictions in a single network pass. This design feature renders YOLO models particularly well suited to edge computing applications, where the minimization of latency and the processing of data locally are of paramount importance. Among the various YOLO models, the YOLOv5m model emerged as the most suitable option due to its ability to deliver optimal performance at a processing speed of 60 frames per second (FPS) [40]. This decision was also influenced by hardware constraints, specifically the limitation to edge hardware with only 4 GB of GPU VRAM. The YOLOv5m model could effectively operate within these parameters while prioritizing human detection. Expanding upon the YOLOv5m architecture, we developed a real-time person detection system tailored to accurately discern the number of people within the robot's interaction space and determine their participation in the conversation. To further enhance the effectiveness of the social robot, the person detection simultaneously evaluates three camera angles to accurately determine the presence of individuals. In initial testing, person detection errors were primarily due to people in the background, who were not involved in the conversation, being mistakenly counted as users. In compliance with the General Data Protection Regulation (GDPR), we carefully selected a location within the city administration to effectively minimize any background presence. We rigorously adhered to the principle of data minimization, aligning both with legal requirements and ethical guidelines. Although it was not always possible to completely remove background elements, we made every effort to centralize them to the lowest extent possible, incorporating this approach into our framework. Therefore, adjustments were required depending on the deployment environment, mainly involving fine-tuning the interaction radius and adapting the person detection to ambient lighting conditions. To address lighting issues, we adjusted the brightness, gain, and exposure settings of each camera to ensure consistent image quality, minimizing over- and underexposure across different lighting conditions. This iterative calibration process improved the detection accuracy. These modifications ensured optimal performance, maintained high accuracy in person detection in different settings, and contributed to the overall effectiveness of multi-party interactions. This capability strengthens the robustness of the social robot, especially in scenarios involving groups where individuals may be partly obstructed by others. Our system analyzes the area of the bounding boxes of detected individuals to assess whether they are actively involved in the interaction. Adjustable threshold parameters define the minimum area size required for a person to be considered part of the conversation. Figure 4 illustrates the multi-camera person detection system.