Year: 2019

Abstract:
Unsupervised landmark learning is the task of learning semantic keypoint-like representations without the use of expensive input keypoint-level annotations. A popular approach is to factorize an image into a pose and appearance data stream, then to reconstruct the image from the factorized components. The pose representation should capture a set of consistent and tightly localized landmarks in order to facilitate reconstruction of the input image. Ultimately, we wish for our learned landmarks to focus on the foreground object of interest. However, the reconstruction task of the entire image forces the model to allocate landmarks to model the background. This work explores the effects of factorizing the reconstruction task into separate foreground and background reconstructions, conditioning only the foreground reconstruction on the unsupervised landmarks. Our experiments demonstrate that the proposed factorization results in landmarks that are focused on the foreground object of interest. Furthermore, the rendered background quality is also improved, as the background rendering pipeline no longer requires the ill-suited landmarks to model its pose and appearance. We demonstrate this improvement in the context of the video-prediction task.

text: Unsupervised landmark learning is the task of learning semantic keypoint-like representations without the use of expensive input keypoint-level annotations. A popular approach is to factorize an image into a pose and appearance data stream, then to reconstruct the image from the factorized components. The pose representation should capture a set of consistent and tightly localized landmarks in order to facilitate reconstruction of the input image. Ultimately, we wish for our learned landmarks to focus on the foreground object of interest. However, the reconstruction task of the entire image forces the model to allocate landmarks to model the background. This work explores the effects of factorizing the reconstruction task into separate foreground and background reconstructions, conditioning only the foreground reconstruction on the unsupervised landmarks. Our experiments demonstrate that the proposed factorization results in landmarks that are focused on the foreground object of interest. Furthermore, the rendered background quality is also improved, as the background rendering pipeline no longer requires the ill-suited landmarks to model its pose and appearance. We demonstrate this improvement in the context of the video-prediction task.

heading: 1 INTRODUCTION
text: Pose prediction is a classical computer vision task that involves inferring the location and configuration of deformable objects within an image. It has applications in human activity classification, finding semantic correspondences across multiple object instances, and robot planning to name a few. One of the caveats of this task is that annotation is very expensive. Individual object “parts” need to be carefully and consistently annotated with pixel-level precision. Our work focuses on the task of unsupervised landmark learning, which aims to find unsupervised pose representations from image data without the need for direct pose-level annotation.
A good visual landmark should be tightly localized, consistent across multiple object instances, and grounded on the foreground object of interest. Tight localization is important because many objects (such as persons) are highly deformable. A landmark localized to a smaller, rigid area of the object will offer more precise pose information in the event of object motion. Consistency across multiple object instances is also important, as we wish for our landmarks to apply to all instances within a visual category. Finally, and most relevant to our proposed method, we want our landmarks to focus on the foreground objects. A landmark that fires on the background is a wasted landmark, as the background is constantly changing, and yields little information regarding the pose of our foreground object of interest.
Many unsupervised landmark learning methods perturb an input training image with various transformations, then require the model to learn semantic correspondences across the transformed variants to piece together the unaltered input image. The primary issue with this approach is it penalizes the entire image reconstruction when we care only about the foreground, resulting in landmarks being allocated to the background. This poses a number of issues, including increased memory requirements (more landmarks required to capture the foreground) and lower landmark reliability (landmarks assigned to background are unstable). Our proposed method aims to reduce the likeli-
hood of landmarks being allocated to the background, thereby improving overall landmark quality and reducing the number of landmarks required to achieve state-of-the-art performance.
Our work builds upon existing methods in image-reconstruction-guided landmark learning techniques (Jakab et al., 2018; Lorenz et al., 2019). We explicitly encourage our model to factorize the reconstruction task into separate foreground and background reconstructions, where only the foreground reconstruction is conditioned on learned landmarks. Our contributions are as follows:
1. We propose an improvement to reconstruction-guided unsupervised landmark learning that allows the landmarks to better focus on the foreground.
2. We demonstrate through empirical analysis that our proposed factorization allows for state-ofthe-art landmark results with fewer learned landmarks, and that fewer landmarks are allocated to modeling background content.
3. We demonstrate that the overall quality of the reconstructed frame is improved via the factorized rendering, and include an application to the video-prediction task.

heading: 2 RELATED WORKS
text: Our work builds upon prior methods in unsupervised discovery of image correspondences (Thewlis et al., 2017b; Zhang et al., 2018b; Suwajanakorn et al., 2018; Thewlis et al., 2017a; Kanazawa et al., 2016; Jakab et al., 2018; Lorenz et al., 2019). Most relevant here are Jakab et al. (2018) and Lorenz et al. (2019), which learn the latent landmark activation maps via an image factorization and reconstruction pipeline. Each image is factored into pose and appearance representations and a decoder is trained to reconstruct the image from these latent factors. The loss is designed such that accurate image reconstruction can only be achieved when the landmarks activate at consistent locations between an image its TPS-warped variant. Lorenz et al. (2019) specifically improves upon the method proposed by Jakab et al. (2018) such that instead of representing the appearance information as a single vector for the entire image, there is a separate appearance encoding for each landmark in the pose representation. One limitation of these works is that the appearance and pose vectors also need to encode background information in order to reconstruct the entire image. Our work attempts to resolve this limitation by introducing unsupervised foreground-background separation into the pipeline, using the pose and appearance vectors for only the foreground rendering.
There are few other works that propose to separate foreground and background in image rendering tasks. Balakrishnan et al. (2018) separates foreground and background for image synthesis in an unseen pose, but their method relies on supervised 2D keypoints. Rhodin et al. (2018) and Rhodin et al. (2019) separate background from foreground for single and multi-person pose-estimation. In both works, the background images are computed by taking the median pixel value across all frames, and therefore require video sequence data with perfectly static backgrounds. Instead, our approach trains a network to synthesize a clean background from any input frame. It is therefore more forgiving with respect to background variation, and can even handle thin-plate-spline warped backgrounds after overfitting to the training data. This allows us to use our method on non-video datasets such as CelebA faces (Liu et al., 2015).

heading: 3 METHOD
text: Our method extends the pipeline proposed by Lorenz et al. (2019). At a high-level, it reconstructs an image from two perturbed variants: one where the appearance (color, lighting, texture) information is perturbed, and one where the pose (position, orientation) of the object is perturbed. The model must learn to extract the pose information from the appearance-perturbed image, and appearance information from the pose-perturbed image. The model will learn a set of landmarks in the process as a means to spatially-align the information extracted from the two sources in order to reconstruct the original image.
Our extension to Lorenz et al. (2019) factorizes the final reconstruction into separate foreground and background renders, where only the foreground is rendered conditioned on the landmark positions. The background will be inferred directly from the pose-perturbed input image with a simple UNet (Ronneberger et al., 2015). We want our UNet to have a limited capacity for handling complex changes in pose. The remaining complex pose changes (e.g. limb motion, object rotations) will
then be captured by the more flexible landmark representations. During training, this factorization is guided by a translating foreground/static background assumption, though we demonstrate that landmark quality is improved even when this assumption is held weakly.

heading: 3.1 MODEL COMPONENTS
text: Our full pipeline comprises five components: the pose and appearance encoders, foreground decoder, background reconstruction subnet, and foreground mask subnet.
The goal of the pose encoder Φpose = Encpose(x) is to take an input image x and output a set of unsupervised part activation maps. Critically, we want these part activation maps to be invariant to changes in local appearance, as well as to be consistent across deformations. A heatmap that activates on a person’s right hand should be invariant across varying skin tones and lighting conditions, as well track the right hand’s location across varying deformations and translations.
The appearance encoder Φapp = Encapp(x;Encpose(x)) extracts local appearance information, conditioned on the pose-encoder’s activation maps. Given an input image x, the pose encoder will first provide K × H × W part activation maps Φpose. To extract local appearance vectors, the appearance encoder projects the image to a C×H×W appearance feature mapMapp. We compute the appearance vector for the kth pose activation map as:
Φappk,c = H∑ i W∑ j Φposek,i,jM app c,i,j for c = 1...C, (1)
giving us K C-dimensional appearance vectors. Here, each activation map in Φpose is softmaxnormalized.
The method pipeline attempts to reconstruct the original input image by combining the pose information from the K activation maps with the pooled appearance vectors for each of the K parts. As in Lorenz et al. (2019), we fit a 2D Gaussian to each activation of the K activation maps by estimating their respective means and either estimating or using a pre-determined covariance. Each part is represented by Φ̃posek = (µk,Σk), where µk ∈ R2 and Σk ∈ R2×2. The 2D Gaussian approximation forces each part activation map into a unimodal representation with a simple parameterization, thereby enforcing that each landmark appears in at most one location per image.
The foreground decoder (FGDec) and background reconstruction subnet (BGNet) are networks that attempt to reconstruct the foreground and background respectively. Our foreground decoder is based on the architecture proposed in SPADE (Park et al., 2019). In SPADE, semantic maps are used to predict spatially-aware affine transformation parameters for normalization schemes such as InstanceNorm. Herein, we project the 2D Gaussian parameters from Φ̃pose to a heatmap of the target output width and height to use as semantic maps in the SPADE architecture. Following Lorenz et al. (2019), we use the formula:
s(k, l) = 1
1 + (l − µk)TΣ−1k (l − µk) (2)
where s(k, l) is the heatmap value for part map k at coordinate location l. In addition to feeding s(k, l) as a semantic map to SPADE, individual appearence vectors are also projected onto their respective heatmap to create a localized appearance encoding to be fed into the decoder. Please see section 3.4 of Lorenz et al. (2019) for details on this projection.
Unlike the foreground decoder, which is conditioned on bottlenecked pose-appearance representation, the BGNet is given direct access to image data, albeit the pose-perturbed variant of the input. Given a static background video sequence, we assume it is easier for the BGNet to learn to directly copy background pixels (and remove the foreground when necessary) than it is for the pose-appearance factorization to learn to model the background. In the absence of a BGNet-like module, several landmarks will be allocated to capture the “pose” of the background, despite being ill-suited for such a task.
The final module is the foreground mask subnet (MaskNet), which infers the blending mask to composite the foreground and background renders. It can be interpreted as a foreground segmentation mask and is conditioned on Φ̃pose.

heading: 3.2 TRAINING PIPELINE
text: All network modules are jointly trained in a fully self-supervised fashion, using the final image reconstruction task as guidance. We follow the training method as detailed in Lorenz et al. (2019), with the addition of our proposed factorized rendering pipeline in the reconstruction phase. An illustration of this pipeline is depicted in Fig. 1.
Training involves reconstructing an image from its appearance and pose perturbed variants, learning to extract the un-perturbed element from each variant. As with Lorenz et al. (2019), we use color jittering to construct the appearance-perturbed variant Tcj(x). When training from video data, we temporally sample a frame 3 to 60 timesteps apart from the same scene to attain the pose-perturbed variant Ttemp(x). However, in the absence of video data, we use thin-plate-spline warping to perturb pose Ttps(x). In general, our method is able to work with both Ttemp(x) and Ttps(x), though TPS-warping has the downside of also warping the background pixels, making the task of BGNet more difficult. Let Φ̃pose be the gaussian-heatmap fitted to the raw activation map Φpose, and let represent element-wise multiplication. Our training procedure can be expressed as follows:
Φposecj = Enc pose(Tcj(x)) and Φ pose temp = Enc pose(Ttemp(x)) (3) Φapp = Encapp(Ttemp(x);Enc pose(Ttemp(x))) (4)
Mcj = MaskNet(Φ̃posecj ) andMtemp = MaskNet(Φ̃ pose temp) (5)
x̃fg = FGDec(Φ̃posecj ,Φ app) and x̃bg = BGNet((1−Mtemp) Ttemp(x)), (6)
x̃ =Mcj x̃fg + (1−Mcj) x̃bg (7)
where the goal is to minimize the reconstruction loss between the original input x and the reconstruction x̃. As can be seen, neither the shape encoder nor the appearance encoder are ever given direct access to the original image x. The pose information feeding into the foreground decoder FGDec(·, ·) is based on the color-jittered input image, where only the local appearance information is perturbed. The appearance information is captured from Ttemp(x) (or Ttps(x)), where the pose information is perturbed. Notice the shape encoder is also executed on both the pose-perturbed and color-jittered input images. This is necessary to map the localized appearance information for a particular landmark from its location in the pose-perturbed image to its unaltered position in Tcj(x).
Finally, the predicted foreground-background masks are computed for both the appearance and pose perturbed variants:Mcj andMtemp respectively.Mcj should have a foreground mask corresponding to the original foreground’s pose, and is used to blend the foreground and background renders in the final step. Mtemp is the foreground mask for the pose-perturbed input image, and assists the BGNet in removing foreground information from its background render. Refer to Appendix A for architecture, loss, and training parameters.

heading: 4 EXPERIMENTS
text: Here, we analyze the effect of introducing foreground-background separation into an unsupervisedlandmark pipeline. Through empirical analysis, we demonstrate that the learned landmarks less used for capturing background information, thereby improving overall landmark quality. Landmark quality is evaluated by using linear regression to map the unsupervised landmarks to annotated keypoints, with the assumption that well-placed, spatially consistent landmarks lead to low regression error. Finally, we include an additional application of our method in the video prediction task, demonstrating how the factorized rendering pipeline improves the overall rendered result.

heading: 4.1 DATASETS
text: We evaluate our method on Human3.6M (Ionescu et al., 2013), BBC Pose (Charles et al., 2013), CelebA (Liu et al., 2015), and KTH (Schuldt et al., 2004). Human3.6M is a video dataset that features human activities recorded with stationary cameras from multiple viewpoints. BBC Pose dataset contains video sequences featuring 9 unique sign language interpreters. Individual frames are annotated with keypoint annotations for the signer. While most of the motion is from the hand gestures of the signers, the background features a constantly changing display that makes clean background separation more difficult. CelebA is an image-only dataset that features keypoint-annotated celebrity faces. As with prior works, we separate out the smaller MAFL subset of the dataset, train our landmark representation on the remaining CelebA training set, and perform the annotated regression task on the MAFL subset. The KTH dataset comprises videos of people performing one of six actions (walking, running, jogging, boxing, handwaving, hand-clapping). We use KTH for our video prediction application. Additional preprocessing details are given in Appendix A.

heading: 4.2 UNSUPERVISED LANDMARK EVALUATION
text: As with prior works (Jakab et al., 2018; Thewlis et al., 2017a), we fit a linear regressor (without intercept) to our learned landmark locations from our pose representation to supervised keypoint coordinates. Following Jakab et al. (2018), we create a loose crop around the foreground object using the provided keypoint annotations, and evaluate our landmark learning method within said crop. Importantly, most prior methods have not released their evaluation code for all datasets, thus we were not able to control for cropping parameters and coordinate space. The former affects the relative size and aspect ratio of the foreground object to the input frame, whereas the latter affects the regression results in the absence of a bias term. As such, external comparisons on this task should be interpreted as a rough comparison at best, and that the reader focus on the comparison against our internal baseline, which is our rough implementation of Lorenz et al. (2019). We include our cropping details in Appendix A.
We report our regression accuracies on Human3.6M, BBC, and CelebA/MAFL, with the first two being video-based datasets and the last being image only. Results are shown in Tables 1a, 1b, and 2 respectively. For the video datasets, we found it best to use only Ttemp(x) to sample perturbed poses from future frames during training. Only Ttps(x) was possible for CelebA/MAFL. Our primary baseline is our model without the explicit foreground-background separation. For this baseline, we report results using Ttemp-only (Baseline (temp)) as well as both Ttemp and Ttps (Baseline (temp,tps)). In all cases, we demonstrate that including factorized foreground-background rendering improves landmark quality compared to the controlled baseline model. We also believe our performance is competitive if not state-of-the-art based on our best-attempt at matching cropping and regression protocols for external comparisons. The results on CelebA demonstrate that our method works even given very weak static background assumptions. This is because Ttps(x) indiscriminately warps the entire image, creating a pose-perturbed variant with a heavily deformed background. Further discussion in Appendix B.
Table 2: Landmark evaluation on MAFL using 10 landmarks. Prediction error is scaled by inter-ocular distance. While we can only use Ttps to sample pose perturbations on this nonvideo dataset, we still see strong improvements over the baseline. We also ablate the use of the masksM in Ours (No Mask), where no masks are predicted and the predicted x̃fg and x̃bg are directly elementwise-added.
MAFL Error Thewlis et al. (2017b) 6.32 Zhang et al. (2018a) 3.46 Lorenz et al. (2019) 3.24 Jakab et al. (2018) 3.19 Baseline (tps) 4.34 Ours (No Mask) 2.88 Ours 2.76
Next, we analyze how factorizing out the background rendering influences landmark quality. In Fig. 2a, we present an ablation study where we measure the regression-toannotation accuracy against the number of learned landmarks. Compared to our baseline models, we can see that the background-factorization allows us to achieve better accuracy with fewer landmarks, and that the degradation is less steep. Further, in Table 2, we include a No Mask baseline which is our proposed model but sans predicted blending masks. Here, we combine foreground and background directly with: x̃ = x̃fg + x̃bg . This variant also improves over the unfactorized baseline, though the full pipeline still performs best.
One of our primary claims is that by factorizing foreground and background rendering in the training pipeline, we allow the landmarks to focus on modeling the pose and appearance of the foreground objects, leaving the background rendering task to a less expressive, but easier to learn mechanism. We attempt to validate this claim on the Human3.6M dataset, as they provide foregroundbackground segmentation masks. If the landmarks truly focus more on modeling the foreground more, then underlying activation heatmaps for each unsupervised landmark should be more contained within the provided segmentation masks in the factorized case. In Fig. 2b, we compare the percentage of the normalized activation maps contained within the provided segmentation masks against our baseline model for 8, 12, and 16 landmark models. For each learned landmark, we first compute its average activation mass contained within the foreground
segmentations. We then sort the landmarks in ascending order of containment (horizontal axis of Fig. 2b) and plot the models’ landmark-containment curve.
The results in Fig. 2b demonstrate that the foreground-background factorization noticeably improves the least containment of the least-contained landmarks. Note that the lowest containment percentages for the baseline are 0.05, 0.4, and 0.3, whereas the factorized containment percentages are an order of magnitude larger at 8.2, 4.8, and 2.5 for 16, 12, and 8 landmarks respectively. It is safe to say that the least-contained landmarks for the baseline model are nearly completely utilized for modeling the image background (99%+ of the activation mass is on the background). While the proposed factorization does not eliminate the problem, we believe this difference is a contributing factor to the improvements over our baseline.
We show qualitative results of our regressed annotated keypoint predictions, as well as landmark activation and foreground mask visualizations in Fig. 3. From top to bottom, we show our regressed annotated keypoint predictions, our predicted foreground mask, and the underlying landmark activation heatmaps. Datasets are BBC Pose, Human3.6M, and CelebA/MAFL respectively. Notice that the degree of binarization in the predicted mask is indicative of the strength of the static background assumption on the data. Human3.6M features a strongly static background, whereas BBC Pose has a constantly updating display on the left, and CelebA was trained with Ttps which indiscriminately warps both foreground and background. Nevertheless, our method still shows improves over the baseline despite imperfectly binarized foreground-background separation.

heading: 4.3 APPLICATION TO VIDEO PREDICTION
text: Lorenz et al. (2019) applied their model to video-to-video style transfer on videos of BBC signers, indicating that the rendered images from the landmark model are temporally stable. One of the issues with these renders, however, is that the landmarks are not suited for modeling the background, resulting in low-fidelity rendered backgrounds. We demonstrate that our factorized formulation resolves this issue.
We evaluate our rendering on the video prediction task on the KTH dataset, and compare against external methods. The unsupervised landmark model factorizes image data into pose (landmarks parameterized as 2D Gaussians) and appearance information. We assume the appearance information remains constant throughout each video sequence, and use an LSTM to predict how the 2D Gaussians move through time conditioned on an initial set of seed-frames. Refer to Appendix D for implementation details. We show our qualitative and quantitative results in Fig. 4 and 5 and respectively. We report SSIM, PSNR, and the perceptual-feature based LPIPS (Zhang et al., 2018a) metric. Note that the background-factorized approach significantly outperforms the unfactorized baseline on all performance metrics, indicating better background reconstruction, as the foreground is a comparatively smaller portion of the frame. Our method is also competitive with state-of-the-art models such as Lee et al. (2018). In Fig. 4, we show our rendered foreground, mask, rendered background, and the corresponding composition. Our method assumes a fixed background for the entire sequence, but predicts a new foreground and blending mask for each extrapolated timestep. Both our baseline and proposed method maintain better structural integrity than other methods. However, due to the imperfect binarization of the predicted mask, the foreground in the composite image may
appear somewhat faded compared to that of other methods. Improved binarization of the predicted masks remains a topic of future work.

heading: 5 CONCLUSION
text: We propose and study the effects of explicitly factorized foreground and background rendering on reconstruction-guided unsupervised landmark learning. Our experiments demonstrate that by using UNet to learn a simpler copy mechanism to copy roughly static background pixels, the model do a better job of allocating landmarks to the foreground objects of interest. As such, we are able to achieve more accurate regressions to annotated keypoints with fewer landmarks, thereby reducing memory requirements. We also demonstrate applications of our pipeline to unsupervised-landmarkbased video manipulation tasks. For future work, we are interested in finding ways to improve binarization of the predicted foreground masks.

heading: B STATIC IMAGE REFACTORING DURING TRAINING
text: In Fig. 6, we show our model outputs from training on CelebA. We see that the pipeline has determined the center of the face to be foreground, with everything else as background. Importantly, the fourth column from the left shows that the BGNet is capable of memorizing how to rectify a
thin-plate-spline warped image. While this is a case of overfitting, it is arguably the reason our approach is able to perform foreground-background separation when the static background assumption is weak. A median-filtering based approach for background subtraction would be feasible only on video sequence data with perfectly still backgrounds.

heading: C ADDITIONAL QUALITATIVE LANDMARK PREDICTION RESULTS
text: Here, we show additional results for the pose-regression task on various datasets. The regression quality is generally very accurate, though notice that, as with other unsupervised landmark approaches, we cannot model keypoint visibility easily, nor can we distinguish between front and back facing subjects on the Human3.6M dataset.
D VIDEO PREDICTION WITH UNSUPERVISED LANDMARKS
As can be seen in Fig. 1, the rendering component of our pipeline is conditioned directly on an appearance encoding Φapp and a pose encoding Φ̃pose. Conveniently, the pose representation is stored as a set of 2D Gaussian distributions, which can can easily manipulate spatially by translating their means and updating their covariances. As such, we apply our model to the video prediction task by assuming Φapp to be constant throughout the prediction sequence, and using an LSTM to update Φ̃pose, conditioned on the first N frames. We extract Φapp from the N th frame in the input sequence.
Care must be taken to maintain positive definite covariance matrices during prediction. Thus we use the parameters of the Cholesky decomposition of the covariance matrices as the prediction targets. In practice, the LSTM may produce an estimate for L which is not a valid Cholesky factor, but even this case will still produce a valid covariance matrix when LLT is computed.
An illustration of our LSTM setup is depicted in Fig.8. In addition to the Cholesky parameterization, we also found that predicting the residual during extrapolation, rather than the state directly, was important to robust long term predictions. In practice. this helped improve both training performance and generalization. At each timestep, the LSTM takes a concatenation of the previous state and state residual to predict the next state residual.
Our LSTM comprises 3 LSTM layers and a final linear layer. Each LSTM layer has 256 channels. For the KTH results, we trained our landmark model with 40 landmarks. We also use a GAN loss term to improve image quality. Due to the KTH dataset being grayscale,
we had to predict foreground-background masks at half resolution to prevent the masks from encoding all the foreground details. The LSTM was trained to predict 10 future frames from an input sequence of 10.

heading: E ADDITIONAL VIDEO PREDICTION RESULTS

heading: F ADDITIONAL VIDEO PREDICTION EXPERIMENT ON BAIR PUSH DATASET
text: We additionally run video prediction experiments on the BAIR action-free dataset (Finn et al., 2016). This dataset consists of videos with robot arms moving randomly with a diverse set of objects on a table. The videos have spatial resolution of 64×64. For this dataset, our video prediction LSTM is trained with a 10 input 0 future setup (never conditioning on its own output during training). We show our qualitative and quantitative results on the BAIR dataset in Fig. 10 and 11 and respectively.

Review summaries:
The paper presents an unsupervised approach for learning landmarks in images or videos with single objects by separating the representation of the image into foreground and background and factorizing the representation of the foreground into pose and appearance. It builds upon previous work [Jakab 2018, Lorenz 2019] who proposed to train by reconstructing the original image from one version of the image with perturbed appearance and another with perturbed pose. It extends this approach by introducing an additional separation of foreground and background in the image. 

The paper presents an unsupervised method to get disentanglement of pose, appearance, background from both images domain and video domain. 5 sub-network are used to model pose, appearance, foreground, background, and decoders.  Their methods let the network focus more on the foreground to regress the landmark and improve state-of-the-art performance on landmark regression (unsupervised.), video prediction and image reconstruction.