## 3 Experiments and Results
### Experiments
#### 3.2.3 Deep Neural Network Models  
To compare the performance of all the proposed models, a dataset is generated with 100,000 samples using quaternions to represent orientation. The dataset is preprocessed and normalized using robust scaling, then fed to all the models for comparison. In every case, the learning rate is reduced using the callback described in Algorithm 1, starting at 0.1 and ending at 0.0001. Only one curriculum is generated. The total data generation time is 1 min and 40 s.  
Table 4 collects the results for each metric after training, testing the models on a previously unseen set of data. In Figure 12, the MSE loss, position error, and orientation error for every model can be seen. For the position error, the Euclidean distance is used, the units being meters. For the quaternion error, the chosen metric is the pseudometric given by Equation (5), meaning the results are in radians and, in particular, in the \([0,\sqrt{2}]\) range.  
Figure 11: Test run on the same set of 10,000 samples, using different preprocessing normalization techniques.  
All three plots represent metrics evaluated on the dataset apart from validation at the end of each epoch. Rather than attempting to provide statistical relevance, the results intend to be descriptive.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **MSE** & **Position Error** & **Orientation Error** \\ \hline SimpleMlp & 0.1286 & **0.1395** & 0.2992 \\ DeepMlp & 1.4451 & 0.7872 & 1.0508 \\ SubnetMlp & **0.1155** & 0.1429 & **0.2949** \\ CNN & 1.3380 & 0.9288 & 1.0193 \\ SimpleRnn & 0.1308 & 0.1468 & 0.3048 \\ LSTM & 0.3578 & 0.5711 & 0.9405 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Comparison of the different metrics for each model, obtained from the test set split from the same 100,000 samples dataset, normalized using robust scaling.  
Figure 12: Test run on the same set of 100,000 normalized samples, using different models. In the first image, the y-axis is limited to 1.7 to reduce the effects of the SubnetMlp outliers.  
The variance of the output columns in the dataset is 1.2284, which means that most models have an acceptable MSE according to the R\({}^{2}\) metric, where R\({}^{2}\) is defined as \(R^{2}=1-\frac{MSE}{\text{norm}(Y)}\), except for the CNN and DeepMlp models. This, in opposition to the performance of the SimpleMlp and SimpleRnn models, both with only one hidden layer, points to the conclusion that increasing the depth of the DNNs used does not necessarily lead to better performance when dealing with the IK problem.  
It can be seen, both in the plot and in the results, that the best-performing model is the SubnetMlp one (Figure 6). It is additionally the most affected by the learning rate decreasing callback (Algorithm 1), since every other model, although decreasing slightly with the change in learning rate, mostly seems to stabilize quickly, while the descent in this case is steeper.  
The performance of both RNNs and the SubnetMlp highlight the importance of considering the dependencies between joints when dealing with the IK problem. In the RNN case, this is implicit in its memory as it takes into account past results while it is learning. In the SubnetMlp, it is explicit in the model architecture, as each subnet takes the results from the previous one as an input.  
Regarding training time, it is to be noted that simpler architectures such as SimpleMlp and SimpleRnn took three times less time to train than the more complex models.