# Improving Robotic Bin-Picking Performances through Human-Robot Collaboration
## 6 Results and Discussion  
The experimental test revealed that robot collaborative functions can be effectively exploited when performing bin-picking tasks, and they allow a wide variety of possible applications. For example, manual hand guidance of the robot end effector can be exploited when clearing object-recognition faults, and the ability of a cobot to detect contacts can be used to inform the robot that the fault has been cleared by simply touching it. In addition, the experimental tests proved that it is, in fact, possible to use the same vision system for object detection and human recognition in collaborative human-robot bin-picking tasks. This is done by implementing a multitasking program that manages the two vision processes: the one for object recognition is a 3D vision process, while the one for human recognition is a 2D vision process. Three-dimensional map acquisition takes a significant amount of time, around 2 s, while object recognition takes \(650\pm 50\) ms. Two-dimensional image acquisition and human detection takes \(914\pm 6\) ms. It may seem a high value compared to the time required for 3D object recognition, but this depends on the searching area: reduced to the bin in 3D object recognition, extended to the whole field of view of the camera in human recognition. However, considering that the maximum TCP speed when  
Figure 8: Example of a perception failure: (**a**) results of object recognition and magnification of the 3D point map acquired (it has too few points due to reflectance proprieties of the surface). (**b**) Fault resolution by manual hand guidance to the pick pose. Figure 7: Example of a failure because of constraints in robot motion: (**a**) results of object recognition (the highlighted nut is recognized but cannot be picked), (**b**) constraints in robot motion (bin faces), and (**c**) resolution of the fault by manual pick of the part. working in collaborative mode is 250 mm/s, the time required to detect the operator was short enough to ensure safety during the tests, as no collisions occurred. The results obtained suggest that human-robot collaboration when performing bin-picking tasks increases flexibility and improves fault tolerance by combining human perception skills and robots' ability to perform repetitive tasks and lift heavy objects. In [6; 7; 8], human perception skills were already exploited to solve object recognition failures in bin-picking. In this case, the robot initiates a call to a remotely located human operator to ask for help in resolving perception system failures during bin-picking operations. This requires a user interface that allows information exchanges between the robot and the human operator that detects the objects that need to be picked from the bin. In the proposed layout, the human operator is already working alongside the robot, and he or she can more quickly resolve the fault by manually moving the robot end effector above the grasping position without the need for an interface. Manual hand guidance of the robot to the pick position can be particularly useful when resolving perception failures of heavy objects that are difficult to pick manually. Moreover, in this work, human perception skills are exploited not only for fault resolution, but also for fault prediction. In fact, the operator can predict when a potential failure may occur and autonomously disentangle entangled objects or manually pick a part that cannot be picked due to constraints in robot motion. Tests have shown that it is useful, as suggested in [9], that the robot's speed is adjusted according to the relative distance between the robot end effector and the operator. However, in [9], a laser sensor was used, while in this work the bracelet allows a more accurate and almost costless location of the operator. The experimental tests carried out showed that it is possible to effectively use the same robot-integrated vision sensor for both object and human recognition, proving the inexpensiveness of the proposed strategy.