# Semi-Supervised Manifold Alignment Using Parallel Deep Autoencoders
## 6 Discussion and Conclusions  
While previous research had shown that deep autoencoders are capable of manifold learning, this study introduced a parallel deep autoencoder model for manifold alignment. The present study focused on semi-supervised approaches where parts of the data were labelled by correspondence information. Future studies may address the general case of data without correspondence information and a wider range of datasets. In the new parallel model, two deep autoencoder networks were trained in parallel to minimise the sum of their reconstruction errors and a correspondence error. The minimisation of the reconstruction errors led the deep autoencoders to extract manifolds intrinsic to the datasets. The minimisation of the correspondence error aligned the extracted manifolds at the code layer. The activations at the 3D code layers allowed visualising the aligned manifolds or sections of them in three dimensions. First, PDAE was evaluated on a well-known protein structure benchmark dataset, where it performed comparably to traditional manifold alignment methods. Then, in the main part of our study, PDAE was tested on a suite of new 6-dimensional, 9-dimensional, and 11-dimensional datasets that we generated by simulating the non-linear dynamics of two double pendulums and adding various levels of noise. It should be noted that manifold learning and manifold alignment can be computationally very expensive (Table 6). Hence, the experiments of our comparative study were restricted to two pendulums with similar arm-length proportions. With this data, the expected resulting manifolds were of sufficient complexity to challenge the methods, but could still be visualised either in full as two-dimensional tori in three-dimensional space (Figure 5) or using snapshots or sections of the resulting 3-manifolds (Figure 6) or 4-manifolds (Figure 7). Interestingly, the visualisation of the aligned manifold obtained from the two three-dimensional pendulum motion datasets on the right in Figure 7 together with the numerical evaluation in Table 4 indicated that, even in this unexampled case of representing a latent 4-manifold, PDAE produced the expected outcome and performed better than all other tested methods. In fact, all methods failed except PDAE and MAGG-feat. The quantitative performance evaluation in Table 4 shows that the alignment using PDAE was closer and more stable than that of MAGG. Moreover, due to the involvement of high-dimensional matrix multiplication, MAGG required a significantly higher execution time than the other methods (Table 6). Table 6 summarises the execution times for all methods used in our study when applied to the clean and complete versions of our simulated datasets. While training of the PDAE took a long time, the inference, that is the execution time using the trained neural model, was significantly faster than running the other methods. It is important to note that if new data points are included in the dataset, the trained model of PDAE can still be executed while the other methods require recalculation. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Methods** & **Detail** & \multicolumn{3}{c}{**Execution Times**} \\ \hline \hline  & & **2-Manifold Alignment** & **3-Manifold Alignment** & **4-Manifold Alignment** \\ \hline \multirow{2}{*}{**MAPA**} & feat & 0.63 s & 2.2 min & 6.6 min \\  & inst & 0.71 s & 4.2 min & 13.9 min \\ \hline \multirow{2}{*}{**MALG**} & feat & 0.32 s & 1.6 min & 5.3 min \\  & inst & 0.44 s & 3.7 min & 11.7 min \\ \hline \multirow{2}{*}{**MAGG**} & feat & 3 s & 9.5 min & 11 h \\  & inst & 8 s & 12.3 min & 13 h \\ \hline \multirow{2}{*}{**PDAE**} & training & 18 (3.88) min & 22 (5.56) min & 9.3 (5.22) h \\  & inference & 0.33 (0.04) s & 0.61 (0.08) s & 1.6 (0.12) s \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Shown are the execution times of the different manifold alignment methods when processing our data. The dataset sizes were 1296 \(\times\) 6, 1728 \(\times\) 9, and 20,736 \(\times\) 11 for the 2-, 3- and 4-manifold data, respectively.