# Sim-Real Mapping of an Image-Based Robot Arm Controller Using Deep Reinforcement Learning
## 5 Discussion  
The paper proposed a sim-real and a corresponding real-sim model to transfer the learnt controller of a robotic arm from the simulation environment to real space. A custom-made 4 DoF robot arm was designed for testing the proposed system. The simulation was conducted in the MuJoCo virtual environment, with careful design consideration to mirror the physical setup in an attempt to reduce the reality gap. In the study, image processing and mapping techniques were applied to reduce the RG between the simulated and actual environment. For image processing, dimensionality reduction and filtering techniques were investigated with the end goal of bridging the reality gap. The advantage of using grayscaled images as opposed to raw images was demonstrated to reduce the computational cost and enhance the accuracy of the network [25]. In the sim-real model, the robot was trained and effectively attained gripping operations successfully for all of the image processing techniques. This is as reported in Table 6 with an average success rate of 100% for all target positions. However, upon deploying the model to the actual robot, the average successful grasps dropped to 15% for the left, 57% for the center, and 43% for the right target positions. This paints a clear picture of the severity of RG, where both the environment and the target objects ought to be similar for \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Image Processing** & **Left [\%]** & **Center [\%]** & **Right [\%]** & **Total [\%]** \\ \hline \hline Grayscale & 100 & 100 & 100 & 100 \\ Binarization & 100 & 100 & 100 & 100 \\
10-gradation gray scale & 100 & 100 & 100 & 100 \\
4-gradation gray scale & 100 & 100 & 100 & 100 \\
10-gradation gray scale w/median filter & 100 & 100 & 100 & 100 \\
4-gradation gray scale w/median filter & 100 & 100 & 100 & 100 \\ \hline \hline \end{tabular}
\end{table} 
Table 10: Experimental results in real space with the proposed scheme. Figure 11: Sample training and validation curves of the CNN mapping operation. (**a**) Grayscale operation. (**b**) Binarization operation. the accurate transfer of knowledge. Approaches of a synthetically generated simulation environment based off the real environment, akin to real-sim-real systems and GAN-related studies, can offer a potential solution in this regard [11; 16; 17; 26]. The challenge would be in complexity in the training environment. As noted by [6], these approaches are inherently data-intensive, thus limiting the practical applicability of such methods. The training time of the approaches is in the range of 100,000 episodes with 1,000,000+ episodes being typical even with high-end computing hardware as reported in [17; 26]. The alternative approach considered in this paper is that of mapping the real environment to approximate the simulated environment. In this approach, similar to other reported methods, a search algorithm is introduced to mediate or adapt the real environment to the simulated environment [9; 18; 28; 29]. In the study, a CNN layer is introduced with simulation data to map the real environment to a potentially similar environment in an already learnt simulation environment. With this adjustment, the DRL model attained a 100% grasping success rate in an actual robot arm. Since the grasping was successful in all cases, it can be said that the deep learning CNN was able to form a correspondence between the real and virtual space images. From this, real-sim mapping was confirmed to be more suitable than enhancing the similarity of the two environments. The advantage of the proposed scheme is the integration with a trained simulation model without requiring any additional RL updates. Further, the image processing method adopted reduces the amount of data to be processed, allowing for a quick and efficient learning process with minimal data requirement as reported in the learning curves (Figures 8 and 11). In the proposal, the mapping model is introduced after a trained policy is attained.