: Our model allows user control over both semantic and style as synthesizing an image. The semantic (e.g., existence of a tree) is controlled via a label map (visualized in the top row), while the style is controlled via the reference style image (visualized in the leftmost column). Please visit our website for interactive image synthesis demos. We propose spatially-adaptive normalization, a simple but effective layer for synthesizing photorealistic images given an input semantic layout. Previous methods directly feed the semantic layout as input to the deep network, which is then processed through stacks o f convolution, normaliza tion, and nonlinearity layers. We show that this is subop timal as the normalization layers tend to "wash away" se mantic information. To address the issue, we propose using the input layout fo r modulating the activations in normal ization layers through a spatially-adaptive, learned trans formation. Experiments on several challenging datasets demonstrate the advantage o f the proposed method over ex isting approaches, regarding both visual fidelity and align-+Taesung Park contributed to the work during his NVIDIA internship. merit with input layouts. Finally, our model allows user control over both semantic and style as synthesizing images. Code is available at h t t p s : / / g i t h u b . com / NV1 a b s/ SPADE.
SPADE REF used the semantic label maps to predict affine transformation parameters for modulating the activations in normalization layers.
Semantic Image Synthesis With Spatially-Adaptive Normalization
2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Computer Science