Title: Inverse Problems in Computer Vision using  Adversarial  Imagination Priors
Authors: Hsiao-Yu Fish Tung, Katerina Fragkiadaki
Keywords: Unsupervised Learning, Deep learning
TL;DR: We present a model that given a visual image learns to generate imaginations of  complete scenes, albedo, shading etc, by using adversarial data driven priors on the imaginations spaces.
Abstract:
Given an image,  humans  effortlessly run the image formation process backwards in their minds: they can tell albedo from shading,  foreground from background, and imagine the occluded parts of the scene behind foreground objects. In this work, we propose a weakly supervised inversion machine trained to generate similar imaginations that when rendered using differentiable, graphics-like decoders, produce the original visual input. We constrain the imagination spaces by providing exemplar memory repositories in the form of foreground segmented objects, albedo, shading, background scenes and imposing adversarial losses on the imagination spaces. Our model learns to perform such inversion with weak supervision, without ever having seen paired annotated data, that is, without having seen the image paired with the corresponding ground-truth imaginations. We demonstrate our method by applying it to three  Computer Vision tasks: image in-painting, intrinsic decomposition and object segmentation, each task having its own differentiable renderer.  Data driven adversarial imagination priors  effectively guide inversion,  minimize the need for hand designed priors of smoothness or good continuation, or the need for paired annotated data.
Venue: Submitted to ICLR 2017