# Semi-Supervised Manifold Alignment Using Parallel Deep Autoencoders
## 1 Introduction  
Circles and lines are one-dimensional manifolds. Two-dimensional manifolds include surfaces such as spheres, tori and pretzels. Higher-dimensional manifolds include curved spaces that locally look like a Euclidean space. The mathematical definition of a topological \(n\)-dimensional manifold, or \(n\)-manifold, \(M\), requires that \(M\) is a second countable Hausdorff space, and each each point in \(M\) has a neighbourhood that is homeomorphic to an open subset of the Euclidean space \(\mathbb{R}^{n}\)[1; 2; 3]. Manifold learning refers to techniques of non-linear dimensionality reduction that can extract latent low-dimensional manifolds from high-dimensional data [4; 5; 6; 7]. For example, we can consider the frames of a video where each frame can be regarded as a pixel vector of a few thousand dimensions. If the video shows a rotating object or if the camera rotates in a circle, then manifold learning techniques such as isomap [5] can extract a circle from the video frame sequence, which shows that the essential information contained in the high-dimensional data is that of a circular rotation, which can be visualised as a 1-manifold homeomorphic to \(S^{1}\) in two dimensions [8]. If the object in the image sequence is not only rotated, but also translated, we obtain a manifold homeomorphic to \(S^{1}\times I\), that is a cylinder [9]. A torus homeomorphic to \(S^{1}\times S^{1}\) was obtained by [10] from an image set of robot heads that turn left and right in one circle or not up and down in another circle. Two major issues that can occur in the context of manifold learning are that there are not enough data or that the data are too noisy. Both issues can prevent the success of manifold learning, that is the extraction of an intact latent low-dimensional manifold from high-dimensional input data without it collapsing [4; 11; 12]. In this project, we focus on a variation of manifold learning called manifold alignment [13; 14] that can extract manifolds even under suboptimal conditions. If there are not enough samples in the dataset, then manifold alignment can combine the information from two or more datasets to provide sufficient data. The datasets can be different high-dimensional representations of the same underlying non-linear manifold. Manifold alignment can also be used for data matching and information transfer between these datasets. There are several examples of application studies, for example, in face processing [14; 15; 16; 17; 18; 19; 20], graph matching [21; 22], bioinformatics [23], and in image clustering and classification [24; 25; 26]. With respect to the issue of noise, autoencoder neural networks are a possible solution. It is known that neural networks are not only relatively robust to noise, but that the addition of noise to the data can even improve performance [27]. Basic autoencoders were already employed for dimensionality reduction in the 1980s, but had limitations to learn non-linear data patterns [28]. [PERSON] and [PERSON] [29] proposed deep autoencoders to reduce the dimensionality of images and to embed image sequences in a manifold. Several later studies demonstrated the usefulness of deep autoencoders for dimensionality reduction and manifold learning in various applications [30; 31; 32]. The ability of autoencoder neural networks to deal with noisy data was further established in several recent studies that developed deep autoencoders to denoise data, for example [33; 34]. The present study combines the concept of deep neural networks with their robustness to noise and the concept of manifold alignment with its ability to utilise the information from two or more datasets to achieve successful manifold learning and proposes a Parallel Deep Autoencoder (PDAE) for manifold alignment. The approach can be regarded as semi-supervised because it is a combination of unsupervised learning for dimensionality reduction and supervised learning for aligning the manifolds using correspondence information in two input datasets.