Novel approaches for learning representations
Creators: name: Lotfi Rezaabad, Ali
name: 0000-0001-8526-2322
Publisher: name: The University of Texas at Austin
Year: 2022
Subjects: Representation learning
Graph embedding
Mutual information
Generative models

description: Newly developed machine learning algorithms are heavily dependent on the choice of data representation. Hereupon, the success of these methods generally relies on the quality and usefulness of data representation. For this reason, one significant body of deep learning research is dedicated to capturing/discovering useful representations, machine learning. As a result of this, there has been a surge in interest in unsupervised representation learning, which eliminates the requirement for the time-consuming and expensive labeling procedure. The emergence of variational autoencoders (VAEs) has opened a new avenue for unsupervised learning methods. While these techniques methods are classy in their approach, they are typically not useful for representation learning. One chapter of this dissertation aims to propose a simple and powerful framework based on VAEs that acts as encoder/decoder, and concurrently helps us discover more meaningful representations. To this end, my solution is to address the issue mentioned above by utilizing information-theoretic tools, which eventually promote amortized inference in VAEs. I call this approach InfoMax-VAE, as I propose to solve the problem based on mutual information. I also demonstrate that such an approach can significantly boost the quality of learned high-level representations. There are many interesting applications in physical, social, and information sciences that can be modeled based on relational data. In another chapter of this dissertation, I build off of semi-implicit graph variational auto-encoders to capture higher-order statistics in the latent features of a graph dataset. Indeed, I seek to address this by incorporating hyperbolic geometry in the latent space via a Poincaré embedding to efficiently represent graphs exhibiting hierarchical structure. To tackle the naive posterior latent distribution assumptions in variational inference, I utilize semi-implicit hierarchical variational Bayes to implicitly capture posteriors of given graph data, which may have multiple modes, skewness and highly correlated latent structures. Due to data bias and domain shift, the data used for training a neural network and those used for inference tend not to follow the same distribution. Unfortunately, it has been shown that the performance of neural networks is prone to slight differences between source and target distributions. Although, a variety of complex algorithms have lately been developed for this issue in classification, the similar challenge in unsupervised representation learning has seen very little investigations. I am particularly interested in the case where only a limited number of samples are available from the target domain due to the high cost of data collection. Therefore, to solve the issue of few-shot domain adaptation in unsupervised contrastive learning, I provide a novel method that requires neither samples from the source domain nor labels from the target domain are required.