## 4 Experimental Section
### Simulation Experiments  
#### 4.2.1 Verification of Model Effectiveness  
The visual pushing and grasping (VPG) system provided by Zeng was employed to verify the performance of the model proposed in this study. Data evaluated by the VPG system were collected, and some similarities and differences between the settings of this study's model and the VPG system were noted:  
1. In the simulation, the VPG system used a UR5 robotic arm installed with an RG2 gripper, whereas the system of this study used a Hiwin robotic arm with a Xeg_32 electric gripper. Both of these robotic arms have six degrees of freedom.
2. In the VPG system, pushes of 10 cm length were made, whereas in this study's model, pushes of 5 cm length were made because the objects employed were smaller and a longer push was thus not required.
3. When the VPG system simulated a push, the push was performed with the end of the closed gripper. In this study, the same setting was employed, and the width of the closed gripper was the same as that of the gripper in the VPG system.  
These slight differences would not affect the model verification because the change in push distance did not require changes in the reinforcement learning algorithm. Moreover, to verify the effectiveness of the mask proposed in this paper, we compare it with the general binary mask. The normal binary mask will change the pattern according to the object geometry, but cannot find the center of the object. as shown in Figure 7. Training was  
\begin{table}
\begin{tabular}{l l l l l l} \hline \hline
**Scenario** & **Number of Objects** & **Success Rate (\%)** & **Avg. Number of Pushes** & **Avg. Number of Grasps** & **Time Taken (seconds)** \\ \hline Static Objects (Easy) & 4 & 95 & 2 & 1 & 12 \\ Static Objects (Medium) & 6 & 89 & 3 & 2 & 20 \\ Static Objects (Hard) & 10 & 75 & 5 & 3 & 35 \\ Dynamic Objects (Easy) & 4 & 90 & 3 & 1 & 15 \\ Dynamic Objects (Medium) & 6 & 82 & 4 & 2 & 28 \\ Dynamic Objects (Hard) & 10 & 68 & 7 & 4 & 45 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Performance metrics across different scenarios.  
performed over 1500 steps, and the rate of successful grasping when using the proposed system was discovered to be 82%. The grasping performance and number of training steps are illustrated in Figure 8.  
The overall grasping performance was calculated by determining the success rate in the preceding 200 training steps. The solid line in Figure 7 is a running average of the rate of grasp success over the preceding 200 grasps. The rate of successful object grasping achieved using the model proposed in this study was thus similar to that achieved using the VPG system [13], confirming that the model can be applied in reinforcement learning algorithms to ensure that objects are effectively pushed and grasped. Recent studies, such as those by [30; 31], support these findings by showing that machine learning steps, including the use of Mask R-CNN and other approaches, can significantly improve object grasping optimization. These machine learning-based methods, through sophisticated visual modeling and adaptive decision-making strategies, enable the system to more accurately identify and respond to different types of objects, as well as different environmental conditions. This is in line with previous studies showing that reinforcement learning algorithms with max-based object detection support can ensure that grasping is performed more effectively and efficiently, increasing the overall success rate [32].