  View synthesis aims to produce unseen views from a set of views captured by
two or more cameras at different positions. This task is non-trivial since it
is hard to conduct pixel-level matching among different views. To address this
issue, most existing methods seek to exploit the geometric information to match
pixels. However, when the distinct cameras have a large baseline (i.e., far
away from each other), severe geometry distortion issues would occur and the
geometric information may fail to provide useful guidance, resulting in very
blurry synthesized images. To address the above issues, in this paper, we
propose a novel deep generative model, called Self-Consistent Generative
Network (SCGN), which synthesizes novel views from the given input views
without explicitly exploiting the geometric information. The proposed SCGN
model consists of two main components, i.e., a View Synthesis Network (VSN) and
a View Decomposition Network (VDN), both employing an Encoder-Decoder
structure. Here, the VDN seeks to reconstruct input views from the synthesized
novel view to preserve the consistency of view synthesis. Thanks to VDN, SCGN
is able to synthesize novel views without using any geometric rectification
before encoding, making it easier for both training and applications. Finally,
adversarial loss is introduced to improve the photo-realism of novel views.
Both qualitative and quantitative comparisons against several state-of-the-art
methods on two benchmark tasks demonstrated the superiority of our approach.
