# Monocular Camera-Based Robotic Pick-and-Place in Fusion Applications
## 4 Experiment  
We carried out our experiment on a computer with a Nvidia RTX 2080 Ti GPU (12 Gigabyte Memory), Intel i7-10700k CPU, and 64 GB of RAM. We created 30 sub  
Figure 3: Some important positions in our task, in this picture, \(P_{a}\) and \(P_{b}\) are the center points of the two cubes, \(P_{c}\) and \(P_{d}\) are the tips of two fingers and \(P_{e}\) is the CPE. environments in the simulation, as shown in Figure 4. Our code is available at here ([[https://github.com/rolandying/Monocular-based-pick-and-place](https://github.com/rolandying/Monocular-based-pick-and-place)]([https://github.com/rolandying/Monocular-based-pick-and-place](https://github.com/rolandying/Monocular-based-pick-and-place)), accessed on 29 March 2023). The implementation of some algorithms depends on the RL_games [25] package. Figure 5 illustrates the reward values during the training process. Following an initial bottleneck phase, the reward value experiences a remarkable surge at approximately 4000 epochs and continues to increase thereafter. This corresponds to the experimental findings that the robot is able to proficiently grasp a target placed randomly on the table following the first 4000 epochs of training. Subsequently, the alignment and stacking processes commence, which take longer, primarily because an eye-in-hand camera is utilized. When the robot arm grasps the white cube, the camera's limited field of view prevents direct observation of the purple cube. Consequently, the robot must rely on historical observation data to ascertain the subsequent destination, resulting in a lengthier learning process. The losses incurred during training are depicted in Figure 6. Analysis of the figure indicates that the value of actor losses (AL) consistently oscillates within a confined range,  
Figure 4: The simulation environment, the number of the sub-environment is depend on the size of GPU Memory. Figure 5: The reward during the training. signifying a stable policy change ratio. This underscores the efficacy of the clipping operation. The value of critic losses (CL) exhibits a continuous decline and ultimately stabilizes, except for a single abrupt change in the middle. At roughly 4 million steps, both AL and CL values experience a peak. This aligns precisely with the moment in Figure 5 when the robot makes a significant breakthrough and adeptly grasps the target around the 4000 th epoch. At this point in time, both the value and policy undergo a rapid transformation. We conducted a comparative analysis between our monocular method and the multi-layer perception machine (MLP) based method. In the latter, the 3D coordinates of the centers of the two cubes are employed as input data, rendering it immensely advantageous, albeit impractical to implement in reality. The outcomes are documented in Table 1. Our method demonstrated that the agent encountered a total of three failures during the testing phase, with one of them occurring during the grasping phase and the remaining during the stacking phase. The failure during the grasping phase can be attributed to the cubes being too close to each other, thereby rendering it arduous for the robot to identify an optimal grasping pose. On the other hand, during the stacking phase, the agent primarily failed due to the target cube rolling off the destination cube, owing to poor posing when the gripper was released. For the MLP method, the agent failed once in the alignment phase and once in the stacking phase. The reason for the failure during the stacking phase remained consistent with our method, whereas the failure during the alignment phase was due to the agent unintentionally pushing the purple cube off the table. The comparison indicates that the performance of our method is marginally inferior to the MLP-based method. However, it is still remarkable, particularly considering that our method relies solely on a monocular camera.