## 4 Method
### Reward Function  
For peg-in-hole assembly, the assembly quality evaluation is one of the most important factors affecting the success rate. The fuzzy logic system could handle multiple parameters, so it could evaluate the quality of the assembly system very well. Therefore, this paper sets up a quality evaluation with fuzzy logic.  
In this paper, four typical parameters are used as the parameters of fuzzy quality evaluation, i.e., contact force \(F_{Y}\) of t-moment y axis, contact force \(F_{Z}\) of z-axis, assembly depth \(Z\) of t-moment shaft and assembly action amount \(D_{Z}\):  
\[F_{Y}=|F_{y}-f_{y}| \tag{11}\]  
\[F_{Z}=|F_{z}-f_{z}| \tag{12}\]  
\[Z=s_{Z} \tag{13}\]  
\[D_{Z}=d-s_{Z} \tag{14}\]  
where \(F_{z}\) is the force of the \(z\) direction, \(f_{z}\) is the initial force of the \(z\) direction, \(F_{y}\) is the force of the \(y\) direction, \(f_{y}\) is the initial force of the \(y\) direction, \(s_{Z}\) is the \(Z\) axis coordinates in the current state of the robot arm, and \(d\) is the \(Z\) axis coordinate value at the bottom of the hole under the same coordinate system as the robot arm.  
If you blur these four parameters, 625 fuzzy rules are needed if you use only one layer of fuzzy logic system. In order to simplify the design of the reward system, the double-layer fuzzy logic structure of Figure 4 is adopted, the first layer has two fuzzy logic systems, the \(F_{y}-F_{z}\) fuzzy logic system takes the contact force \(F_{y}\) and z-axis contact force \(F_{z}\) as input, and the \(Z-D_{Z}\) fuzzy logic system takes the assembly depth \(Z\) and assembly action \(D_{Z}\) of the t-moment axis as the input. The output of the two systems serves as input to the second layer of fuzzy logic systems, and finally the value of the reward required is output by the second layer of fuzzy logic systems. Thus 75 fuzzy rules need to be made, greatly reducing the difficulty of fuzzy rules.  
For the above parameter input system, using triangular membership function for fuzzy processing, each parameter is blurred into 5 fuzzy values: VG, G, M, B, and VB,  
Figure 4: The assembly quality evaluation with fuzzy logic.  
respectively, refer to very good, good, medium, bad, and very bad. Each fuzzy value has a membership range of (0, 1).  
\[f(x)=\left\{\begin{aligned} 0&& x\leq a\\ \frac{x-a}{b-a}&& a\leq x\leq b\\ \frac{c-x}{c-b}&& b\leq x\leq c\\ 0&& x\geq c\end{aligned}\right. \tag{15}\]  
where \(a,b,c\) represent the parameter value in the triangular membership function, \(a,c\) determines the width of the membership function, and \(b\) determines the location of the membership function.  
After blurring the parameters, fuzzy reasoning is based on the established rule library, and the rule library as shown in Table 1 is established according to the experience in peg-in-hole assembly. Additionally, use the AND operation for fuzzy reasoning.  
\[R_{i}(x)=min(\mu_{A(i)}(x),\mu_{B(i)}(x)) \tag{16}\]  
where \(A,B\) are fuzzy collections. The AND operation takes the minimum value of the membership of both and determines the membership of the fuzzy value of the output according to the rule library.  
Finally, the resulting fuzzy value is clearly processed, that is, the last part of the fuzzy logic system: de-fuzzing. Since there are many fuzzy values obtained after fuzzy reasoning, which cannot be used, the data need to be processed by the de-fuzzing method, and finally the clear value that meets our requirements is obtained.  
Discrete fuzzy values are used in this paper. Use the center of gravity method to defuse  
\[U^{*}=\frac{\sum_{i}(\text{C}^{*}_{i}w_{i})}{\sum_{i}(\text{C}^{*}_{i})} \tag{17}\]  
where \(U^{*}\) refers to the clear output after de-blurring, \(U^{*}\sim(-1,0)\) refers to the fuzzy collection of fuzzy reasoning, and \(w_{i}\) refers to the weight of each membership.