Therefore, RL can be applied to robotic agents to learn high-precision assembly skills instead of only transferring human skills to the robot program [8]. Recent studies showed the importance of RL for robotic manipulation tasks [9; 10; 11], but none of these methods can be applied directly to high-precision industrial applications due to the lack of fine motion control. In [12], an RL technique was used to learn a simple peg-in-hole insertion operation. Similarly, [PERSON] et al. [13] proposed a robot skill-acquisition approach by training a recurrent neural network to learn a peg-in-hole assembly policy. However, these approaches use a finite number of actions by discretizing the action space, which has many limitations in continuous-action control tasks [14], as is the case for robot control, which is continuous and high-dimensional. [PERSON] et al. [15] proposed learning dual peg insertion by using the deep deterministic policy gradient [16] (DDPG) algorithm with a fuzzy reward system. Similarly, [PERSON] et al. [17] used DDPG combined with guided policy search (GPS) [18] to learn high-precision assembly tasks. [PERSON] et al. [19] also used GPS to learn peg-in-hole tasks on a deformable surface. Nevertheless, these methods learn policies that control the motion trajectory only, and they require the manual tuning of force control gains; therefore, they do not scale well to variations of the environment. [PERSON] et al. [20] proposed the use of DDPG to simultaneously control position and force control gains, but they assumed the geometric knowledge of the insertion task, which made the learned policies inflexible to be applied to different insertion tasks. To solve high-precision assembly tasks, our approach focused on learning policies that simultaneously control the robot's motion trajectory and actively tune a compliant controller to unknown geometric constraints. [PERSON] et al. [21] accomplished variable stiffness skill learning on robot manipulators by using an RL algorithm call-policy improvement with path integrals (PI2). However, the method was formulated for torque-control robots. Another similar approach was to use a flexible robot so as to focus only on the motion trajectory, as in [22]; however, rigid position-controlled robots are still more widely used. Therefore, we focus on industrial robot manipulators, which are mainly controlled in a position-based manner. [PERSON] et al. [23] proposed a learning method based on iterative learning control (ILC). Their method is focused on transferring manipulation skills from demonstrations that provide a reference trajectory and force profile. In this work, we present a method that can learn manipulation skills without prior knowledge of a reference trajectory or force profile. However, our method supports the use of such prior knowledge to speed up the learning phase. The main contribution of this work is a robust learning-based framework for robotic peg-in-hole assembly given an uncertain goal position. Our method enables a position-controlled industrial robot manipulator to safely learn contact-rich manipulation tasks by controlling the nominal trajectory, and at the same time, learning variable force control gains for each phase of the task. We built this on the basis of our previous work [24]. More specifically, the contributions of this work are:  
* A robust policy representation based on time convolutional neural networks (TCNs). * Faster learning of control policies via domain transfer-learning techniques (sim2 real) to greatly improve the training efficiency in real robots. * Improved generalization capabilities of the learned control policies via domain randomization during the training phase in simulations. Although the effects of domain randomization have been researched [25; 26], to the best of our knowledge, we are the first to study the effects of sim2 real with domain randomization on contact-rich, real-robot applications with position-controlled robots.