user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Variational Inference of Disentangled Latent Concepts from Unlabeled Observations

Paper content:
Disentangled representations, where the higher level data generative factors are reflected in disjoint latent dimensions, offer several benefits such as ease of deriving invariant representations, transferability to other tasks, interpretability, etc.
We consider the problem of unsupervised learning of disentangled representations from large pool of unlabeled observations, and propose a variational inference based approach to infer disentangled latent factors.
We introduce a regularizer on the expectation of the approximate posterior over observed data that encourages the disentanglement.
We also propose a new disentanglement metric which is better aligned with the qualitative disentanglement observed in the decoder's output.
We empirically observe significant improvement over existing methods in terms of both disentanglement and data likelihood (reconstruction quality).
Feature representations of the observed raw data play a crucial role in the success of machine learning algorithms.
Effective representations should be able to capture the underlying (abstract or high-level) latent generative factors that are relevant for the end task while ignoring the inconsequential or nuisance factors.
Disentangled feature representations have the property that the generative factors are revealed in disjoint subsets of the feature dimensions, such that a change in a single generative factor causes a highly sparse change in the representation.
Disentangled representations offer several advantages -(i
) Invariance: it is easier to derive representations that are invariant to nuisance factors by simply marginalizing over the corresponding dimensions, (
ii) Transferability: they are arguably more suitable for transfer learning as most of the key underlying generative factors appear segregated along feature dimensions,
(iii) Interpretability: a human expert may be able to assign meanings to the dimensions,
(iv) Conditioning and intervention: they allow for interpretable conditioning and/or intervention over a subset of the latents and observe the effects on other nodes in the graph.
Indeed, the importance of learning disentangled representations has been argued in several recent works BID5 BID37 BID50 .Recognizing
the significance of disentangled representations, several attempts have been made in this direction in the past BID50 . Much of the
earlier work assumes some sort of supervision in terms of: (i) partial
or full access to the generative factors per instance BID48 BID58 BID35 BID33 , (ii) knowledge
about the nature of generative factors (e.g, translation, rotation, etc.) BID29 BID11 , (iii) knowledge
about the changes in the generative factors across observations (e.g., sparse changes in consecutive frames of a Video) BID25 BID57 BID21 BID14 BID32 , (iv) knowledge
of a complementary signal to infer representations that are conditionally independent of it 1 BID10 BID41 BID53 . However, in most
real scenarios, we only have access to raw observations without any supervision about the generative factors. It is a challenging
problem and many of the earlier attempts have not been able to scale well for realistic settings BID51 BID15 BID13 ) (see also, ).Recently, BID9 proposed
an approach to learn a generative model with disentangled factors based on Generative Adversarial Networks (GAN) BID24 , however implicit generative models like GANs lack an effective inference mechanism 2 , which hinders its applicability to the problem of learning disentangled representations. More recently, proposed
an approach based on Variational AutoEncoder (VAE) BID34 for inferring disentangled factors. The inferred latents using
their method (termed as β-VAE ) are empirically shown to have better disentangling properties, however the method deviates from the basic principles of variational inference, creating increased tension between observed data likelihood and disentanglement. This in turn leads to poor
quality of generated samples as observed in .In this work, we propose a
principled approach for inference of disentangled latent factors based on the popular and scalable framework of amortized variational inference BID34 BID55 BID23 BID49 powered by stochastic optimization BID30 BID34 BID49 . Disentanglement is encouraged
by introducing a regularizer over the induced inferred prior. Unlike β-VAE , our approach does
not introduce any extra conflict between disentanglement of the latents and the observed data likelihood, which is reflected in the overall quality of the generated samples that matches the VAE and is much better than β-VAE. This does not come at the cost of
higher entanglement and our approach also outperforms β-VAE in disentangling the latents as measured by various quantitative metrics. We also propose a new disentanglement
metric, called Separated Attribute Predictability or SAP, which is better aligned with the qualitative disentanglement observed in the decoder's output compared to the existing metrics.

assistant: text: We propose a variational inference based approach for encouraging the inference of disentangled latents. We also propose a new metric for quantifying disentanglement.