# Sim-Real Mapping of an Image-Based Robot Arm Controller Using Deep Reinforcement Learning
## 1 Introduction  
Artificial intelligence is a promising venture that seeks to revolutionize automation and autonomous operations as witnessed in growing interest in the study. Deep reinforcement learning (DRL), one of the many algorithms utilized in artificial intelligence, affords machines the ability to interact with an environment and extract/learn an optimal behavior from the interactions [1]. Learning is arrived at through a trial-and-error process with a corresponding reward for every action taken. This feature of trial-and-error learning is very appealing as training data do not have to be prepared in advance [2]. Reinforcement learning (RL) has been applied in many fields with varying success such as robotics, autonomous driving, and chess games, among others. This paper will focus on the application of DRL in robotics, particularly robot arms. Some of the challenges facing DRL as far as robotics is concerned include but are not limited to the training time/cost and safety of operations during operations [3, 4]. In training, due to the trial-and-error nature of RL, performing such explorations in a real-world environment is costly. Besides that, during the exploration phase, the machine/robot is bound to perform unsafe actions before settling on an optimal action plan. As such, the safe and efficient training of robots is indispensable. Simulators offer a solution to the above issues. In theory, simulators allow the controller to be trained using diverse data, safer procedures, and an added advantage ofscaling up the learning process by parallel trainings, which might be unfeasible in an actual machine. Once the model (policy) is trained in the virtual environment, it is transferred to the actual system, which is referred to as sim-real transfer [5; 6]. Sim-real transfer is only effective if the simulator is given a suitably accurate representation of the actual robot and its environment, which in itself is a difficult task. The problem compounds when all dynamics of the environment/robot are captured as this raises the computational costs. As such, sim-real systems consist of a trade-off between the accurate representation of the robot environment and complexity. As a consequence of this, when such models are deployed in the real world, the controller learnt in the simulator degrades, a phenomenon referred to as the reality gap (RG). In the literature, several approaches are adopted to overcome the RG challenge. Some of these include meta learning [7], domain randomization [8; 9], adversarial RL [10; 11], and continual and transfer learning [12; 13], among others. These approaches can be broadly divided into domain randomization and adaptation strategies. Instead of meticulously simulating all of the real-world characteristics, the objective of domain randomization is to randomize the simulation environment in order to cover the whole distribution of the real world. A challenge with that is that prior knowledge is needed to determine how much randomization to incorporate in training for a policy robust to sim-real transfer. That is, deciding what randomization to add with a concrete parsimonious explanation for how and why the randomization works [2]. In domain adaptation, the idea is to unify the source domain with the target domain via adaptive strategies ingrained during or after training. In this regard, the adaptation focuses on the mismatch between the two environments and optimizes the transfer. The current paper focuses on adaptation strategies, specifically a mapping protocol from the real world to the virtual world, to deal with the RG that exists in sim-real systems in the gripping motion of a robot arm. Until now, conventional DRL studies (to distinguish between sim-real studies) have been conducted and applied in robot arm inquiries. [PERSON] et al. attempted to control a robot using only image information using the RL algorithm and Deep Q-Network [14]. The control target was a 3-DOF robot arm whose goal was to reach a target point with an end-effector.