# A Collaborative Control Method of Dual-Arm Robots Based on Deep Reinforcement Learning
## 4 Experiments
### Simulation Experiments
(**a**) The task is to move a block placed on the table in front of the robot to the target position on the table. (**b**) The learning curve of Push scene—the abscissa is the number of training rounds and the ordinate is the average success rate of 10 tests. Figure 2: Reach scene. (**a**) The left and right target positions are randomly chosen in 3D space. Control the left and right end effectors of the robot to reach the left and right targets as quickly as possible. (**b**) The learning curve of Reach scene—the abscissa is the number of training rounds and the ordinate is the average success rate of 10 tests. **Pick up**: This task is similar to the Push scenario but the target positions are in certain areas above the table and the fingers are not locked, as shown in Figure 4a. The reward function is set based on the distance from the object to the target positions. An episode is successful if the distance between the gripper and the desired target is less than 5 cm. The robot cannot learn knowledge in the initial exploration stage, but can quickly accumulate experience after it changes the position of the object. However, this task is more complicated than the first two. The agents need to control the grippers to move over the two ends of the object, and then control the fingers to open and close for grasping. During the ascending process of median success rate, the relative motion of the two robot arms cannot drastically change, and the grippers cannot be released. Otherwise, the task will fail. Due to the addition of random noise, the dual-arm robot will appear uncoordinated. From Figure 4b, it is clear that the success rate fluctuates at the beginning of training, but the results eventually converge. According to the analysis of the experimental results, due to the difficulty of the task being different, the time for the algorithm to terminate training varies greatly. We conducted experiments using the DADDPG algorithm in the simulation environment. For each training epoch in the experiment, 50 episode cycles (20 episode cycles in the Reach scene) were executed by two parallel agents in the environment and stored in the replay buffer, followed by 40 steps of optimization of the policy performed by [PERSON]. After each training epoch, 10 test episodes were executed and the success rate was calculated. For the Reach, Push, and Pick up tasks, the difficulty of the tasks leads to different training rounds, but the DADDPG algorithm finally converges to the optimal result. The gripping pot experiment was performed in order to verify the performance of the proposed algorithm, as shown in Figure 5a. A circle protrudes from the top of the object. The robot arms are trained based on the proposed algorithm. The robot can touch the reasonable point to grip the pot, and put the pot in a specified area. Figure 5b shows the robot arms are exploring to search the target object at the beginning stage. When they can touch the pot, the learning curve gradually increases as time goes on. However, gripping the pot is more difficult than touching it. The learning curve begins to oscillate with a small amplitude during the learning process because the arms try to grip the pot. With the refinement of the action strategy, the algorithm converges to the optimal solution. The simulation results show the model can be convergent for different tasks and has better generalization ability. Figure 4: Pick up scene. (**a**) This task is similar to the Push scene, but the target position is in a specific area above the table. The robot needs to hold the block on the target position. (**b**) The learning curve of Pick up scene—the abscissa is the number of training rounds and the ordinate is the average success rate of 10 tests.