## 4 Experimental Results
### Experimental Verification of the System  
The initial practical tests at the robotic workplace were conducted using the YOLACT model for object detection. These tests began with simulations in the CoppeliaSim software (4.2), employing the Franka Emika Panda collaborative robot, which had been already preconfigured. To simulate a real-world scenario, the simulation environment was modified by integrating 3D tool models in DAE format, positioning a camera parallel to the work table, and adding a shelf model where the robot would place the tools during the simulation. The results of tool detection can be observed in Figure 11, and the complete simulated robotic workplace is depicted in Figure 12.  
Despite these preparations, the YOLACT model did not consistently achieve reliable tool detection, necessitating careful rotation of tools to improve accuracy. The objective of this simulation was to evaluate the feasibility of creating an accurate sampling position for the robot using the object detection algorithm. At this stage, operator gesture detection was not yet operational, so tools were selected manually by inputting corresponding numbers. The simulation was successful, leading us to proceed with testing in a real robotic workplace.  
The tool feeding test at the physical workplace with the Franka Emika Panda collaborative robot was conducted in several phases. Unlike the simulation, where the camera was positioned parallel to the work table, in the physical setup, the camera was mounted directly on the robot. In the end, these experiments revealed the need for a more effective tool detection model, leading us to adopt the previously trained YOLOv8 model. Following these initial tests, the process advanced to the test phase using the ABB Yumi collaborative robot. Experimental testing with ABB Yumi is shown in Figure 13.  
Figure 11: Successful tool detection in CoppeliaSim.  
Figure 12: Simulated workplace in CoppeliaSim.  
Next, after training the YOLOv7 model for gesture detection and integrating it into ROS 2, practical tests at the workplace were conducted. Figure 14 shows experimental results of object detection and tool recognition. Additionally, a GUI was created to visualize the list of detected tools, detect gestures, and recalculate the take-off position for the robot. In Figure 15, the individual elements of the graphical interface can be seen.  
Following the selection of a work tool through hand gestures, the robot calculates the tool's orientation and grasping position. The orientation is determined from the segmentation mask of the detected tool, identifying the handle or gripping side. To enhance workflow efficiency, the robot presents the selected tool to the operator with the handle facing outward.  
The final procedure for work tool feeding was as follows:  
1. Initially, the presence of work tools on the worktable is detected, and these work tools are arranged into a numbered list.
2. Work tools are then selected from the list using gestures.
3. The feeding process is started by a specific gesture.
4. The selected work tool is then picked from the worktable by the robot.
5. The work tool is oriented with its handle facing towards the operator.
6. Handover of the work tool to the operator.  
This scenario repeats until the feeding process is stopped by a stop gesture or until at least one work tool model remains on the table.  
Figure 14: PC screen displaying the graphical user interface with tool detection and gesture recognition.  
Figure 13: Successful tool feeding test by ABB Yumi robot.