## 4 Simulation Experiments and Optimization
### Simulation Experiments  
1. Conventional stochastic gradient optimization is solved by using an activation function combining stochastic gradient descent (SGD) and hyperbolic tangent (tanh). The curves of the Sigmoid function and tanh function are shown in Figure 8. The advantage is that it is smooth and easy to derivate and can map a real number to the interval [0, 1]. The disadvantage is that the exponential operation has a large amount of calculation, slow descent speed, and there is division in the derivation when the backpropagation is solving the error gradient. In the process of backpropagation, saturated neurons will lead to the disappearance of the gradient, so that the training of the deep network cannot be completed. The fitting situation of SGD plus tanh solution is shown in Figures 9 and 10. 2. The activation function combining Nesterov adaptive moment estimation (Nadam) and hyperbolic tangent (tanh) is used to complete the experiment. Nadam has a stronger constraint on the learning rate and a more direct influence on the gradient update. In this case, the gradient disappearance still exists. The main advantage of Nadam over Adam is better performance in the case of disappearing gradients. In general, where you want to use RMSprop or Adaptive Moment Estimation (Adam), \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**i** & \(\theta_{1}\) & \(\theta_{2}\) & \(\theta_{3}\) & \(\theta_{4}\) & \(\theta_{5}\) & \(\theta_{6}\) \\ \hline
1 & 169.152 & \(-\)74.474 & 100.077 & \(-\)250.680 & \(-\)19.168 & 291.015 \\
2 & 108.117 & \(-\)10.469 & 60.329 & \(-\)325.002 & 85.537 & 303.795 \\
3 & 66.132 & \(-\)37.705 & 84.361 & \(-\)75.441 & 38.092 & \(-\)230.169 \\
4 & 76.237 & \(-\)135.703 & \(-\)43.846 & \(-\)317.680 & \(-\)98.703 & 226.420 \\
5 & \(-\)140.971 & \(-\)72.721 & 143.930 & \(-\)111.730 & 20.891 & \(-\)193.332 \\
6 & \(-\)92.969 & \(-\)105.562 & 19.138 & 139.354 & 95.771 & 321.504 \\
7 & 17.470 & \(-\)121.286 & \(-\)78.944 & \(-\)169.744 & 83.476 & \(-\)172.002 \\
8 & 116.285 & \(-\)107.124 & 135.547 & \(-\)105.011 & \(-\)74.334 & \(-\)174.241 \\
9 & 42.937 & \(-\)76.106 & \(-\)23.294 & 231.580 & 20.890 & 34.807 \\
10 & 154.362 & \(-\)101.412 & 88.230 & 177.610 & \(-\)29.291 & 47.475 \\ ⋮ & ⋮ & ⋮ & ⋮ & ⋮ & ⋮ & ⋮ \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Joint angles sample data. The row represents joint angles, the column represents the sample number. Figure 7: The data acquisition platform. most can use Nadam for better results. The fitting situation of the Nadam plus tanh solution is shown in Figures 11 and 12. 3. The activation function combining Nesterov adaptive moment estimation (Nadam) and improved ReLU function (ELU) is used to complete the experiment. The curve of the ELU function is shown in Figure 13. Compared with the ReLU function, there is a certain number of outputs in the case of negative input, and this part of the output has a certain anti-interference ability, which can eliminate the problem of ReLU dying. However, there are still problems with gradient saturation and exponential operation, and the calculation intensity is high. The fitting situation of the Nadam plus ELU solution is shown in Figures 14 and 15. The loss function used in this paper is mean_squared_error function, which is the mean square error loss function. This function can accurately describe the angle difference when calculating the joint angles, which is equivalent to the accuracy difference of the FK. The experiment results of the above three solution methods are shown in Figure 16 and Table 5. According to the analysis of data results, the Nadam solver plus ELU function has the fastest solution speed. A higher accuracy is obtained with the increasing number of iterations. Figure 11: The loss function curve of Nadam + tanh solution method. Figure 12: The precision for the solution of Nadam + tanh. Figure 13: The curves of ELU function. Figure 16: The solving speed comparison of the three solution methods. Figure 14: The loss function for the solution of Nadam + ELU.