## 1 Introduction  
The robot operating contact environment is changeable and unpredictable. It is a challenge that the robot could quickly perform new tasks and precisely control the contact force in different environments. High-precision assembly is a typical contact operation [1, 2], and the assembly process needs to overcome the environmental model and controller errors. The peg-in-hole assembly process is usually divided into the search phase and the insertion phase [3], which is visual and tactile. In the insertion phase, the center axis of the peg-in-hole inserts into the bottom. When the axis deviation or force/torque is not appropriate, it can cause card resistance or wedge tightening. Due to the deformation error, friction and robot positioning error between assembly objects, it is difficult to establish an accurate physical model and find the optimal assembly strategy according to the model analysis.  
Robot assembly control strategies could be designed with forces and torques in the robot assembly based on mathematical models. Compared to the position feedback controller with high gain, impedance control ensures that the robot and environment are fully controllable. A natural mass-damping-spring relationship is maintained between the contact force and the position offset, and its force control characteristics depend on inertia, stiffness, and damping parameters [4]. The traditional method of adjusting parameters manually adjusts the control parameters according to the characteristics of the task. For the assembly of such complex tasks, it is difficult to set the impedance control method of fixed parameters to achieve the target task. If the parameters of impedance control could be adjusted to changes in the assembly tasks, the control performance is improved. The safety of the robot could also be guaranteed in the process of adjusting the impedance parameters.  
An important feature of reinforcement learning [5; 6] is that optimal performance could be achieved by designing incentive function guidance without understanding the robot and environmental system models. Therefore, this paper mainly studies how robots could react to contact forces by changing the stiffness motion in the robot peg-in-hole insertion task. The robot obtains the optimal control strategy in the assembly process through dynamic adjustment of the impedance parameters. Aiming at the soft control problem in the robot contact task, this paper combines the quality evaluation of fuzzy reward and the deep deterministic strategy gradient to achieve variable impedance. The robot could adapt to the different contact force changes in the assembly process.  
The main contributions are as follows:  
1. The reward function with fuzzy logic based on assembly quality evaluation was designed in the actual industrial environment to improve the system learning performance;
2. The assembly process model was constructed and the robot completed the stiffness workpieces assembly;
3. The robot combined with the variable impedance in the course of operation to avoid the damage of the workpiece during the learning process;
4. A framework of learning robot assembly skills is proposed, and it is verified in the peg-in-hole assembly of weak stiffness workpieces.
5. The proposed method in this paper combines deep reinforcement learning with robotics technology, which provides theoretical support for the improvement of complex manipulation skills of a new generation of robots. It also provides a new idea for the application of artificial intelligence algorithms in the industrial field.  
The remainder of this paper is organized as follows. Section 2 introduces the related work. The assembly system and formulation of the problem to be solved are in Section 3. Section 4 contains the description of the proposed method. Experiments were performed to validate the proposed method, and the results are presented and discussed in Section 5. Finally, in Section 6, we summarize the results of the current work and discuss future directions.