# Improving Robotic Bin-Picking Performances through Human-Robot Collaboration
## 3 Collaborative Human-Robot Bin-Picking
### Possible Collaborative Bin-Picking Applications  
Implementing a collaborative human-robot bin-picking cell allows a wide variety of possible applications, such as:  
* When perception failures occur (i.e., the computer vision algorithm recognizes that the bin is not empty, but object recognition fails), the human operator can manually move the robot above the grasping pose by pushing it directly, exploiting zero gravity torque control [23]. Once the robot has been moved to the pick position, the operator notifies the cobot, for example, by touching it, exploiting collaborative (i.e., the robot ability to detect contacts) features, or by pushing a button. The robot then proceeds to approach and pick the part. The bin-picking execution task is restored and the fault is quickly and easily cleared;
* When bin-picking fails because of pose-estimation failures or because of constraints in robot motion, the cobot seeks external help from the operator. The human operator can manually perform bin-picking and subsequent placement of the part;
* When the computer vision algorithm recognizes that two or more objects are entangled with each other, the robot asks for external help, and the human operator can manually pick the entangled objects, separate them, and either put them back in the bin or place them where they need to be placed;
* The operator working alongside the robot can also autonomously predict possible bin-picking failures. For example, the operator can notice that two or more objects are entangled and disentangle them, without waiting for the cobot to ask for help. In addition, the operator can notice that an object near the edges of the bin cannot be picked and pick it manually. In these situations, the cobot must interrupt the execution of normal bin-picking tasks and behave accordingly: it might move away from the bin or decrease its speed;
* Human-robot interaction in collaborative bin-picking is not limited to fault handling and resolution. Since bin-picking is often the preliminary task in kitting or assembly applications, the robot can perform bin-picking and subsequently place heavier objects with a relatively simple shape, while the operator can perform bin-picking and subsequently place the objects with complex shapes that can easily become entangled and are difficult to separate;
* The workspace can be divided into two or more zones with bins or heaps of objects that need to be picked and placed, and the operator and the robot can work together, each on a different bin/heap of objects. To give an example, the operator might switch from an area to the area where a cobot is already working: in this case, the cobot moves away from that area and starts working in another one;
* The operator can also manually deposit more items inside the bin or replace an empty bin with a full one: in this scenario, the system recognizes the presence of the operator  
Figure 1: Human–robot collaboration when resolving bin-picking failures.  
and the robot can move away from the bin and resume its work once the operator has finished the operation;
* When an human operator enters the workcell, the velocity of the robot can be adapted to its relative distance from the operator. In particular, the robot's speed can be reduced proportionally to its distance from the operator. The robot path can also be modified to prevent collisions by exploiting collision avoidance algorithms.  
In particular, the great advantage of collaborative bin-picking applications is that operators can work alongside robots and intervene only when necessary, increasing system flexibility, productivity, and fault tolerance. Moreover, collaborative bin-picking allows handling a wide assortment of parts, without the need for any change in the hardware structure and design.