Original: I NFINITY GAN: T OWARDS I NFINITE -P IXEL I MAGE S YNTHESIS
Revised: I NFINITY GAN: T OWARDS I NFINITE -P IXEL I MAGE S YNTHESIS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We present a novel framework, InﬁnityGAN, for arbitrary-sized image generation.
Revised: We present a novel framework, InﬁnityGAN, for arbitrary-sized image generation.

Original: The task is associated with several key challenges.
Revised: The task is associated with several key challenges.

Original: First, scaling existing models to an arbitrarily large image size is resource-constrained, in terms of both computation and availability of large-ﬁeld-of-view training data.
Revised: First, scaling existing models to an arbitrarily large image size is resource-constrained, in terms of both computation and availability of large-ﬁeld-of-view training data.

Original: InﬁnityGAN trains and infers in a seamless patch-by-patch manner with low computational resources.
Revised: InﬁnityGAN trains and infers in a seamless patch-by-patch manner with low computational resources.

Original: Second, large images should be locally and globally consistent, avoid repetitive patterns, and look realistic.
Revised: Second, large images should be locally and globally consistent, avoid repetitive patterns, and look realistic.

Original: To address these, InﬁnityGAN disentangles global appearances, local structures, and textures.
Revised: To address these, InﬁnityGAN disentangles global appearances, local structures, and textures.

Original: With this formulation, we can generate images with spatial size and level of details not attainable before.
Revised: With this formulation, we can generate images with spatial size and level of details not attainable before.

Original: Experimental evaluation validates that InﬁnityGAN generates images with superior realism compared to baselines and features parallelizable inference.
Revised: Experimental evaluation validates that InﬁnityGAN generates images with superior realism compared to baselines and features parallelizable inference.

Original: Finally, we show several applications unlocked by our approach, such as spatial style fusion, multimodal outpainting, and image inbetweening.
Revised: Finally, we show several applications unlocked by our approach, such as spatial style fusion, multimodal outpainting, and image inbetweening.

Original: All applications can be operated with arbitrary input and output sizes.
Revised: All applications can be operated with arbitrary input and output sizes.

Original: [Figure]
Revised: [Figure]

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Generative models witness substantial improvements in resolution and level of details.
Revised: Generative models witness substantial improvements in resolution and level of details.

Original: Most improvements come at a price of increased training time (Gulrajani et al., 2017; Mescheder et al., 2018), larger model size (Balaji et al., 2021), and stricter data requirements (Karras et al., 2018).
Revised: Most improvements come at a price of increased training time (Gulrajani et al., 2017; Mescheder et al., 2018), larger model size (Balaji et al., 2021), and stricter data requirements (Karras et al., 2018).

Original: The most recent works synthesize images at 1024 × 1024 resolution featuring a high level of details and ﬁdelity.
Revised: The most recent works synthesize images at 1024 × 1024 resolution featuring a high level of details and ﬁdelity.

Original: However, models generating high resolution images usually still synthesize images of limited ﬁeld-of-view bounded by the training data.
Revised: However, models generating high resolution images usually still synthesize images of limited ﬁeld-of-view bounded by the training data.

Original: It is not straightforward to scale these models to generate images of arbitrarily large ﬁeld-of-view.
Revised: It is not straightforward to scale these models to generate images of arbitrarily large ﬁeld-of-view.

Original: Synthesizing inﬁnite-pixel images is constrained by the ﬁnite nature of resources.
Revised: Synthesizing inﬁnite-pixel images is constrained by the ﬁnite nature of resources.

Original: Finite computational resources (e.g., memory and training time) set bounds for input receptive ﬁeld and output size.
Revised: Finite computational resources (e.g., memory and training time) set bounds for input receptive ﬁeld and output size.

Original: A further limitation is that there exists no inﬁnitepixel image dataset.
Revised: A further limitation is that there exists no inﬁnitepixel image dataset.

Original: Thus, to generate inﬁnite-pixel images, a model should learn the implicit global structure without direct supervision and under limited computational resources.
Revised: Thus, to generate inﬁnite-pixel images, a model should learn the implicit global structure without direct supervision and under limited computational resources.

Original: Repetitive texture synthesis methods (Efros & Leung, 1999; Xian et al., 2018) generalize to large spatial sizes.
Revised: Repetitive texture synthesis methods (Efros & Leung, 1999; Xian et al., 2018) generalize to large spatial sizes.

Original: Yet, such methods are not able to synthesize real-world images.
Revised: Yet, such methods are not able to synthesize real-world images.

Original: Recent works, such as SinGAN (Shaham et al., 2019) and InGAN (Shocher et al., 2019), learn an internal patch distribution for image synthesis.
Revised: Recent works, such as SinGAN (Shaham et al., 2019) and InGAN (Shocher et al., 2019), learn an internal patch distribution for image synthesis.

Original: Although these models can generate images with arbitrary shapes, in
Revised: Although these models can generate images with arbitrary shapes, in

Original: Section 4.1, we show that they do not infer structural relationships well, and fail to construct plausible holistic views with spatially extended latent space.
Revised: Section 4.1, we show that they do not infer structural relationships well, and fail to construct plausible holistic views with spatially extended latent space.

Original: A different approach, COCO-GAN (Lin et al., 2019), learns a coordinate-conditioned patch distribution for image synthesis.
Revised: A different approach, COCO-GAN (Lin et al., 2019), learns a coordinate-conditioned patch distribution for image synthesis.

Original: As shown in
Revised: As shown in

Original: Figure 4, despite the ability to slightly extend images beyond the learned boundary, it fails to maintain the global coherence of the generated images when scaling to a 2 × larger generation size.
Revised: Figure 4, despite the ability to slightly extend images beyond the learned boundary, it fails to maintain the global coherence of the generated images when scaling to a 2 × larger generation size.

Original: How to generate inﬁnite-pixel images?
Revised: How to generate inﬁnite-pixel images?

Original: Humans are able to guess the whole scene given a partial observation of it.
Revised: Humans are able to guess the whole scene given a partial observation of it.

Original: In a similar fashion, we aim to build a generator that trains with image patches, and inference images of unbounded arbitrary-large size.
Revised: In a similar fashion, we aim to build a generator that trains with image patches, and inference images of unbounded arbitrary-large size.

Original: An example of a synthesized scene containing globally-plausible structure and heterogeneous textures is shown in Figure 1.
Revised: An example of a synthesized scene containing globally-plausible structure and heterogeneous textures is shown in Figure 1.

Original: We propose InﬁnityGAN, a method that trains on a ﬁnite-pixel dataset, while generating inﬁnitepixel images at inference time.
Revised: We propose InﬁnityGAN, a method that trains on a ﬁnite-pixel dataset, while generating inﬁnitepixel images at inference time.

Original: InﬁnityGAN consists of a neural implicit function, termed structure synthesizer , and a padding-free StyleGAN2 generator, dubbed texture synthesizer .
Revised: InﬁnityGAN consists of a neural implicit function, termed structure synthesizer , and a padding-free StyleGAN2 generator, dubbed texture synthesizer .

Original: Given a global appearance of an inﬁnite-pixel image, the structure synthesizer samples a sub-region using coordinates and synthesizes an intermediate local structural representations.
Revised: Given a global appearance of an inﬁnite-pixel image, the structure synthesizer samples a sub-region using coordinates and synthesizes an intermediate local structural representations.

Original: The texture synthesizer then seamlessly synthesizes the ﬁnal image by parts after ﬁlling the ﬁne local textures to the local structural representations.
Revised: The texture synthesizer then seamlessly synthesizes the ﬁnal image by parts after ﬁlling the ﬁne local textures to the local structural representations.

Original: InﬁnityGAN can infer a compelling global composition of a scene with realistic local details.
Revised: InﬁnityGAN can infer a compelling global composition of a scene with realistic local details.

Original: Trained on small patches, InﬁnityGAN achieves high-quality, seamless and arbitrarily-sized outputs with low computational resources—a single TITAN X to train and test.
Revised: Trained on small patches, InﬁnityGAN achieves high-quality, seamless and arbitrarily-sized outputs with low computational resources—a single TITAN X to train and test.

Original: We conduct extensive experiments to validate the proposed method.
Revised: We conduct extensive experiments to validate the proposed method.

Original: Qualitatively, we present the everlastingly long landscape images.
Revised: Qualitatively, we present the everlastingly long landscape images.

Original: Quantitatively, we evaluate InﬁnityGAN and related methods using user study and a proposed ScaleInv FID metric.
Revised: Quantitatively, we evaluate InﬁnityGAN and related methods using user study and a proposed ScaleInv FID metric.

Original: Furthermore, we demonstrate the efﬁciency and efﬁcacy of the proposed methods with several applications.
Revised: Furthermore, we demonstrate the efﬁciency and efﬁcacy of the proposed methods with several applications.

Original: First, we demonstrate the ﬂexibility and controllability of the proposed method by spatially fusing structures and textures from different distributions within an image.
Revised: First, we demonstrate the ﬂexibility and controllability of the proposed method by spatially fusing structures and textures from different distributions within an image.

Original: Second, we show that our model is an effective deep image prior for the image outpainting task with the image inversion technique and achieves multi-modal outpainting of arbitrary length from arbitrarily-shaped inputs.
Revised: Second, we show that our model is an effective deep image prior for the image outpainting task with the image inversion technique and achieves multi-modal outpainting of arbitrary length from arbitrarily-shaped inputs.

Original: Third, with the proposed model we can divide-and-conquer the full image generation into independent patch generation and achieve 7.2 × of inference speed-up with parallel computing, which is critical for high-resolution image synthesis.
Revised: Third, with the proposed model we can divide-and-conquer the full image generation into independent patch generation and achieve 7.2 × of inference speed-up with parallel computing, which is critical for high-resolution image synthesis.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Latent generative models.
Revised: Latent generative models.

Original: Existing generative models are mostly designed to synthesize images of ﬁxed sizes.
Revised: Existing generative models are mostly designed to synthesize images of ﬁxed sizes.

Original: A few methods (Karras et al., 2018; 2020) have been recently developed to train latent generative models on high-resolution images, up to 1024 × 1024 pixels.
Revised: A few methods (Karras et al., 2018; 2020) have been recently developed to train latent generative models on high-resolution images, up to 1024 × 1024 pixels.

Original: However, latent generative models generate images from dense latent vectors that require synthesizing all structural contents at once.
Revised: However, latent generative models generate images from dense latent vectors that require synthesizing all structural contents at once.

Original: Bounded by computational resources and limited by the learning framework and architecture, these approaches synthesize images of certain sizes and are non-trivial to generalize to different output size.
Revised: Bounded by computational resources and limited by the learning framework and architecture, these approaches synthesize images of certain sizes and are non-trivial to generalize to different output size.

Original: In contrast, patch-based GANs trained on image patches (Lin et al., 2019; Shaham et al., 2019; Shocher et al., 2019) are less constrained by the resource bottleneck with the synthesis-by-part approach.
Revised: In contrast, patch-based GANs trained on image patches (Lin et al., 2019; Shaham et al., 2019; Shocher et al., 2019) are less constrained by the resource bottleneck with the synthesis-by-part approach.

Original: However, (Shaham et al., 2019; Shocher et al., 2019) can only model and repeat internal statistics of a single image, and (Lin et al., 2019) can barely extrapolate few patches beyond the training size.
Revised: However, (Shaham et al., 2019; Shocher et al., 2019) can only model and repeat internal statistics of a single image, and (Lin et al., 2019) can barely extrapolate few patches beyond the training size.

Original: ALIS (Skorokhodov et al., 2021) is a concurrent work that also explores synthesizing inﬁnite-pixel images.
Revised: ALIS (Skorokhodov et al., 2021) is a concurrent work that also explores synthesizing inﬁnite-pixel images.

Original: It recursively inbetweens latent variable pairs in the horizontal direction.
Revised: It recursively inbetweens latent variable pairs in the horizontal direction.

Original: We further discuss the method in Appendix A.1.
Revised: We further discuss the method in Appendix A.1.

Original: Finally, autoregressive models (Oord et al., 2016;
Revised: Finally, autoregressive models (Oord et al., 2016;

Original: Razavi et al., 2019; Esser et al., 2021) can theoretically synthesize at arbitrary image sizes.
Revised: Razavi et al., 2019; Esser et al., 2021) can theoretically synthesize at arbitrary image sizes.

Original: Despite (Razavi et al., 2019) and (Esser et al., 2021) showing unconditional images synthesis at 1024 ×resolution, their application in inﬁnite-pixel image synthesis has not yet been well-explored.
Revised: Despite (Razavi et al., 2019) and (Esser et al., 2021) showing unconditional images synthesis at 1024 ×resolution, their application in inﬁnite-pixel image synthesis has not yet been well-explored.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: Conditional generative models.
Revised: Conditional generative models.

Original: Numerous tasks such as image super-resolution, semantic image synthesis, and image extrapolation often showcase results over 1024 × 1024 pixels.
Revised: Numerous tasks such as image super-resolution, semantic image synthesis, and image extrapolation often showcase results over 1024 × 1024 pixels.

Original: These tasks are less related to our setting, as most structural information is already provided in the conditional inputs.
Revised: These tasks are less related to our setting, as most structural information is already provided in the conditional inputs.

Original: We illustrate and compare the characteristics of these tasks against ours in Appendix B.
Revised: We illustrate and compare the characteristics of these tasks against ours in Appendix B.

Original: Image outpainting.
Revised: Image outpainting.

Original: Image outpainting (Abdal et al., 2020; Liu et al., 2021; Sabini & Rusak, 2018;
Revised: Image outpainting (Abdal et al., 2020; Liu et al., 2021; Sabini & Rusak, 2018;

Original: Yang et al., 2019) is related to image inpainting (Liu et al., 2018a; Yu et al., 2019) and shares similar issues that the generator tends to copy-and-paraphrase the conditional input or create mottled textural samples, leading to repetitive results especially when the outpainted region is large.
Revised: Yang et al., 2019) is related to image inpainting (Liu et al., 2018a; Yu et al., 2019) and shares similar issues that the generator tends to copy-and-paraphrase the conditional input or create mottled textural samples, leading to repetitive results especially when the outpainted region is large.

Original: InOut (Cheng et al., 2021) proposes to outpaint image with GANs inversion and yield results with higher diversity.
Revised: InOut (Cheng et al., 2021) proposes to outpaint image with GANs inversion and yield results with higher diversity.

Original: We show that with InﬁnityGAN as the deep image prior along with InOut (Cheng et al., 2021), we obtain the state-of-the-art outpainting results and avoids the need of sequential outpainting.
Revised: We show that with InﬁnityGAN as the deep image prior along with InOut (Cheng et al., 2021), we obtain the state-of-the-art outpainting results and avoids the need of sequential outpainting.

Original: Then, we demonstrate applications in arbitrary-distant image inbetweening, which is at the intersection of image inpainting (Liu et al., 2018a; Nazeri et al., 2019; Yu et al., 2019) and outpainting research.
Revised: Then, we demonstrate applications in arbitrary-distant image inbetweening, which is at the intersection of image inpainting (Liu et al., 2018a; Nazeri et al., 2019; Yu et al., 2019) and outpainting research.

Original: Neural implicit representation.
Revised: Neural implicit representation.

Original: Neural implicit functions (Park et al., 2019; Mescheder et al., 2019; Mildenhall et al., 2020) have been applied to model the structural information of 3D and continuous representations.
Revised: Neural implicit functions (Park et al., 2019; Mescheder et al., 2019; Mildenhall et al., 2020) have been applied to model the structural information of 3D and continuous representations.

Original: Adopting neural implicit modeling, our query-by-coordinate synthesizer is able to model structural information effectively.
Revised: Adopting neural implicit modeling, our query-by-coordinate synthesizer is able to model structural information effectively.

Original: Some recent works (DeVries et al., 2021; Niemeyer & Geiger, 2021; Chan et al., 2021) also attempt to integrate neural implicit function into generative models, but aiming at 3D-structure modeling instead of extending the synthesis ﬁeld-of-view.
Revised: Some recent works (DeVries et al., 2021; Niemeyer & Geiger, 2021; Chan et al., 2021) also attempt to integrate neural implicit function into generative models, but aiming at 3D-structure modeling instead of extending the synthesis ﬁeld-of-view.

Original:  P ROPOSED M ETHOD 3. O VERVIEW
Revised:  P ROPOSED M ETHOD 3. O VERVIEW

Original: An arbitrarily large image can be described globally and locally.
Revised: An arbitrarily large image can be described globally and locally.

Original: Globally, images should be coherent and hence global characteristics should be expressible by a compact holistic appearance (e.g., a medieval landscape, ocean view panorama).
Revised: Globally, images should be coherent and hence global characteristics should be expressible by a compact holistic appearance (e.g., a medieval landscape, ocean view panorama).

Original: Therefore, we adopt a ﬁxed holistic appearance for each inﬁnite-pixel image to represent the high-level composition and content of the scene.
Revised: Therefore, we adopt a ﬁxed holistic appearance for each inﬁnite-pixel image to represent the high-level composition and content of the scene.

Original: Locally, a close-up view of an image is deﬁned by its local structure and texture.
Revised: Locally, a close-up view of an image is deﬁned by its local structure and texture.

Original: The structure represents objects, shapes and their arrangement within a local region.
Revised: The structure represents objects, shapes and their arrangement within a local region.

Original: Once the structure is deﬁned, there exist multiple feasible appearances or textures to render realistic scenes.
Revised: Once the structure is deﬁned, there exist multiple feasible appearances or textures to render realistic scenes.

Original: At the same time, structure and texture should conform to the global holistic appearance to maintain the visual consistency among the neighboring patches.
Revised: At the same time, structure and texture should conform to the global holistic appearance to maintain the visual consistency among the neighboring patches.

Original: Given these assumptions, we can generate an inﬁnite-pixel image by ﬁrst sampling a global holistic appearance, then spatially extending local structures and textures following the holistic appearance.
Revised: Given these assumptions, we can generate an inﬁnite-pixel image by ﬁrst sampling a global holistic appearance, then spatially extending local structures and textures following the holistic appearance.

Original: Accordingly, the InﬁnityGAN generator G consists of a structure synthesizer G S and a texture synthesizer G T .
Revised: Accordingly, the InﬁnityGAN generator G consists of a structure synthesizer G S and a texture synthesizer G T .

Original: G S is an implicit function that samples a sub-region with coordinates and creates local structural features.
Revised: G S is an implicit function that samples a sub-region with coordinates and creates local structural features.

Original: G T is a fully convolutional StyleGAN2 (Karras et al., 2020)
Revised: G T is a fully convolutional StyleGAN2 (Karras et al., 2020)

Original: modeling textural properties for local patches and rendering ﬁnal image.
Revised: modeling textural properties for local patches and rendering ﬁnal image.

Original: Both modules follow a consistent holistic appearance throughout the process.
Revised: Both modules follow a consistent holistic appearance throughout the process.

Original: Figure 2 presents the overview of our framework.
Revised: Figure 2 presents the overview of our framework.

Original: 3. S TRUCTURE S YNTHESIZER ( G S )
Revised: 3. S TRUCTURE S YNTHESIZER ( G S )

Original: Structure synthesizer is a neural implicit function driven by three sets of latent variables: A global latent vector z g representing the holistic appearance of the inﬁnite-pixel image (also called implicit image since the whole image is never explicitly sampled), a local latent tensor z l expressing the local structural variation of the image content, and a coordinate grid c specifying the location of the patches to sample from the implicit image.
Revised: Structure synthesizer is a neural implicit function driven by three sets of latent variables: A global latent vector z g representing the holistic appearance of the inﬁnite-pixel image (also called implicit image since the whole image is never explicitly sampled), a local latent tensor z l expressing the local structural variation of the image content, and a coordinate grid c specifying the location of the patches to sample from the implicit image.

Original: The synthesis process is formulated as:
Revised: The synthesis process is formulated as:

Original: [Equation]
Revised: [Equation]

Original: where z S denotes the structural latent variable that is later used as an input to the texture synthesizer.
Revised: where z S denotes the structural latent variable that is later used as an input to the texture synthesizer.

Original: We sample z
Revised: We sample z

Original: g ∈ R D z g from a unit Gaussian distribution once and inject z g into every layer and pixel in G S via feature modulation (Huang & Belongie, 2017; Karras et al., 2020).
Revised: g ∈ R D z g from a unit Gaussian distribution once and inject z g into every layer and pixel in G S via feature modulation (Huang & Belongie, 2017; Karras et al., 2020).

Original: As local variations are independent across the spatial dimension, we independently sample them from a unit Gaussian prior for each spatial position of z l ∈ R H
Revised: As local variations are independent across the spatial dimension, we independently sample them from a unit Gaussian prior for each spatial position of z l ∈ R H

Original: × W × D z l , where H and W can be arbitrarily extended.
Revised: × W × D z l , where H and W can be arbitrarily extended.

Original: We then use coordinate grid c to specify the location of the target patches to be sampled.
Revised: We then use coordinate grid c to specify the location of the target patches to be sampled.

Original: To be able to condition G S with coordinates inﬁnitely far from the origin, we introduce a prior by exploiting the nature of landscape images: (a) self-similarity for the horizontal direction, and (b) rapid saturation (e.g., land, sky or ocean) for the vertical direction.
Revised: To be able to condition G S with coordinates inﬁnitely far from the origin, we introduce a prior by exploiting the nature of landscape images: (a) self-similarity for the horizontal direction, and (b) rapid saturation (e.g., land, sky or ocean) for the vertical direction.

Original: To implement this, we use the positional encoding for the horizontal axis similar to (Vaswani et al., 2017; Tancik et al., 2020; Sitzmann et al., 2020).
Revised: To implement this, we use the positional encoding for the horizontal axis similar to (Vaswani et al., 2017; Tancik et al., 2020; Sitzmann et al., 2020).

Original: We use both sine and cosine functions to encode each coordinate for numerical stability.
Revised: We use both sine and cosine functions to encode each coordinate for numerical stability.

Original: For the vertical axis, to represent saturation, we apply the tanh function.
Revised: For the vertical axis, to represent saturation, we apply the tanh function.

Original: Formally, given horizontal and vertical indexes ( i x , i y ) of z l tensor, we encode them as c = (tanh( i y ) , cos( i x /T ) , sin( i x /T )) , where T is the period of the sine function and c controls the location of the patch to generate.
Revised: Formally, given horizontal and vertical indexes ( i x , i y ) of z l tensor, we encode them as c = (tanh( i y ) , cos( i x /T ) , sin( i x /T )) , where T is the period of the sine function and c controls the location of the patch to generate.

Original: To prevent the model from ignoring the variation of z l and generating repetitive content by following the periodically repeating coordinates, we adopt a mode-seeking diversity loss (Mao et al., 2019) between a pair of local latent variables
Revised: To prevent the model from ignoring the variation of z l and generating repetitive content by following the periodically repeating coordinates, we adopt a mode-seeking diversity loss (Mao et al., 2019) between a pair of local latent variables

Original: z l 1 and z l 2 while sharing the same z g and c :
Revised: z l 1 and z l 2 while sharing the same z g and c :

Original: [Equation]
Revised: [Equation]

Original: Conventional neural implicit functions produce outputs for each input query independently, which is a pixel in z l for InﬁnityGAN.
Revised: Conventional neural implicit functions produce outputs for each input query independently, which is a pixel in z l for InﬁnityGAN.

Original: Such a design causes training instabilities and slows convergence, as we show in Figure 36.
Revised: Such a design causes training instabilities and slows convergence, as we show in Figure 37.

Original: We therefore adopt the feature unfolding technique (Chen et al., 2021) to enable G S to account for the information in a broader neighboring region of z l and c , introducing a larger receptive ﬁeld.
Revised: We therefore adopt the feature unfolding technique (Chen et al., 2021) to enable G S to account for the information in a broader neighboring region of z l and c , introducing a larger receptive ﬁeld.

Original: For each layer in G S , before feeding forward to the next layer, we apply a k × k feature unfolding transformation at each location ( i, j ) of the origin input f to obtain the unfolded input f (cid:48) :
Revised: For each layer in G S , before feeding forward to the next layer, we apply a k × k feature unfolding transformation at each location ( i, j ) of the origin input f to obtain the unfolded input f (cid:48) :

Original: [Equation]
Revised: [Equation]

Original: where Concat( · ) concatenates the unfolded vectors in the channel dimension.
Revised: where Concat( · ) concatenates the unfolded vectors in the channel dimension.

Original: In practice, as the gridshaped z l and c are sampled with equal spacing between consecutive pixels, the feature unfolding can be efﬁciently implemented with CoordConv (Liu et al., 2018b).
Revised: In practice, as the gridshaped z l and c are sampled with equal spacing between consecutive pixels, the feature unfolding can be efﬁciently implemented with CoordConv (Liu et al., 2018b).

Original: 3. T EXTURE S YNTHESIZER ( G T )
Revised: 3. T EXTURE S YNTHESIZER ( G T )

Original: Texture synthesizer aims to model various realizations of local texture given the local structure z S generated by the structure synthesizer.
Revised: Texture synthesizer aims to model various realizations of local texture given the local structure z S generated by the structure synthesizer.

Original: In addition to the holistic appearance z g and the local structural latent z S , texture synthesizer uses noise vectors z n to model the ﬁnest-grained textural variations that are difﬁcult to capture by other variables.
Revised: In addition to the holistic appearance z g and the local structural latent z S , texture synthesizer uses noise vectors z n to model the ﬁnest-grained textural variations that are difﬁcult to capture by other variables.

Original: The generation process can be written as
Revised: The generation process can be written as

Original: [Equation]
Revised: [Equation]

Original: where p c is a generated patch at location c (i.e., the c used in Eq 1 for generating z S ).
Revised: where p c is a generated patch at location c (i.e., the c used in Eq 1 for generating z S ).

Original: We implement upon StyleGAN2 (Karras et al., 2020).
Revised: We implement upon StyleGAN2 (Karras et al., 2020).

Original: First, we replace the ﬁxed constant input with the generated structure z S .
Revised: First, we replace the ﬁxed constant input with the generated structure z S .

Original: Similar to StyleGAN2, randomized noises z n are added to all layers of
Revised: Similar to StyleGAN2, randomized noises z n are added to all layers of

Original: G T , representing the local variations of ﬁne-grained textures.
Revised: G T , representing the local variations of ﬁne-grained textures.

Original: Then, a mapping layer projects z g to a style vector, and the style is injected into all pixels in each layer via feature modulation.
Revised: Then, a mapping layer projects z g to a style vector, and the style is injected into all pixels in each layer via feature modulation.

Original: Finally, we remove all zero-paddings from the generator, as shown in Figure 3(b).
Revised: Finally, we remove all zero-paddings from the generator, as shown in Figure 3(b).

Original: Both zero-padding and G S can provide positional information to the generator, and we later show that positional information is important for generator learning in Section 4.2.
Revised: Both zero-padding and G S can provide positional information to the generator, and we later show that positional information is important for generator learning in Section 4.2.

Original: However, it is necessary to remove all zero-paddings from G T for three major reasons.
Revised: However, it is necessary to remove all zero-paddings from G T for three major reasons.

Original: First, zero-padding has a consistent pattern during training, due to the ﬁxed training image size.
Revised: First, zero-padding has a consistent pattern during training, due to the ﬁxed training image size.

Original: Such a behavior misleads
Revised: Such a behavior misleads

Original: Example 1 3x3 Conv2d
Revised: Example 1 3x3 Conv2d

Original: Example 2 3x

Revised: Example 2 3x

Original: Conv2d- Transpose
Revised: Conv2d- Transpose

Original: Local Latent & Coordinate
Revised: Local Latent & Coordinate

Original: ImageSpace (b) Padding-Free Generator
Revised: ImageSpace (b) Padding-Free Generator

Original: Pad
Revised: Pad

Original: Pad
Revised: Pad

Original: Ignored Ignored
Revised: Ignored Ignored

Original: Consistent
Revised: Consistent

Original: Y Z
Revised: Y Z

Original: XYZ XYZ
Revised: XYZ XYZ

Original: XYZ XYZ
Revised: XYZ XYZ

Original: X Y
Revised: X Y

Original: XYZ
Revised: XYZ

Original: XYZ XYZ
Revised: XYZ XYZ

Original: XYZ
Revised: XYZ

Original: Patch A Patch B
Revised: Patch A Patch B

Original: Features
Revised: Features

Original: Positional Info
Revised: Positional Info

Original: Conv2d-Transpose Dilation (a) Conventional Generator
Revised: Conv2d-Transpose Dilation (a) Conventional Generator

Original: X
Revised: X

Original: Z
Revised: Z

Original: Inconsistent
Revised: Inconsistent

Original: Z
Revised: Z

Original: XYZ (=A) XYZ(=A)
Revised: XYZ (=A) XYZ(=A)

Original: Patch A Patch B
Revised: Patch A Patch B

Original: X
Revised: X

Original: Z
Revised: Z

Original: AB
Revised: AB

Original: Pad
Revised: Pad

Original: YZP (=B)
Revised: YZP (=B)

Original: AB
Revised: AB

Original: PXY(=C)
Revised: PXY(=C)

Original: CA CA
Revised: CA CA

Original: A
Revised: A

Original: X Y
Revised: X Y

Original: A
Revised: A

Original: Y
Revised: Y

Original: B
Revised: B

Original: C
Revised: C

Original: B
Revised: B

Original: C
Revised: C

Original: B
Revised: B

Original: C
Revised: C

Original: [Figure]
Revised: [Figure]

Original: the generator to memorize the padding pattern, and becomes vulnerable to unseen padding patterns while attempting to synthesize at a different image size.
Revised: the generator to memorize the padding pattern, and becomes vulnerable to unseen padding patterns while attempting to synthesize at a different image size.

Original: The third column of Figure 4 shows when we extend the input latent variable of the StyleGAN2 generator multiple times, the center part of the features does not receive expected coordinate information from the paddings, resulting in extensively repetitive textures in the center area of the output image.
Revised: The third column of Figure 4 shows when we extend the input latent variable of the StyleGAN2 generator multiple times, the center part of the features does not receive expected coordinate information from the paddings, resulting in extensively repetitive textures in the center area of the output image.

Original: Second, zero-paddings can only provide positional information within a limited distance from the image border.
Revised: Second, zero-paddings can only provide positional information within a limited distance from the image border.

Original: However, while generating inﬁnite-pixel images, the image border is considered inﬁnitely far from the generated patch.
Revised: However, while generating inﬁnite-pixel images, the image border is considered inﬁnitely far from the generated patch.

Original: Finally, as shown in Figure 3, the existence of paddings hampers G T from generating separate patches that can be composed together.
Revised: Finally, as shown in Figure 3, the existence of paddings hampers G T from generating separate patches that can be composed together.

Original: Therefore, we remove all paddings from G T , facilitating the synthesisby-parts of arbitrary-sized images.
Revised: Therefore, we remove all paddings from G T , facilitating the synthesisby-parts of arbitrary-sized images.

Original: We refer to the proposed G T as a padding-free generator (PFG).
Revised: We refer to the proposed G T as a padding-free generator (PFG).

Original: 3. S PATIALLY I NDEPENDENT G ENERATION
Revised: 3. S PATIALLY I NDEPENDENT G ENERATION

Original: InﬁnityGAN enables spatially independent generation thanks to two characteristics of the proposed modules.
Revised: InﬁnityGAN enables spatially independent generation thanks to two characteristics of the proposed modules.

Original: First, G S , as a neural implicit function, naturally supports independent inference at each spatial location.
Revised: First, G S , as a neural implicit function, naturally supports independent inference at each spatial location.

Original: Second, G T , as a fully convolutional generator with all paddings removed, can synthesize consistent pixel values at the same spatial location in the implicit image, regardless of different querying coordinates, as shown in Figure 3(b).
Revised: Second, G T , as a fully convolutional generator with all paddings removed, can synthesize consistent pixel values at the same spatial location in the implicit image, regardless of different querying coordinates, as shown in Figure 3(b).

Original: With these properties, we can independently query and synthesize a patch from the implicit image, seamlessly combine multiple patches into an arbitrarily large image, and maintain constant memory usage while synthesizing images of any size.
Revised: With these properties, we can independently query and synthesize a patch from the implicit image, seamlessly combine multiple patches into an arbitrarily large image, and maintain constant memory usage while synthesizing images of any size.

Original: In practice, having a single center pixel in a z S slice that aligns to the center pixel of the corresponding output image patch can facilitate z l and c indexing.
Revised: In practice, having a single center pixel in a z S slice that aligns to the center pixel of the corresponding output image patch can facilitate z l and c indexing.

Original: We achieve the goal by shrinking the StyleGAN2 blur kernel size from 4 to 3 , causing the model to generate odd-sized features in all layers, due to the convolutional transpose layers.
Revised: We achieve the goal by shrinking the StyleGAN2 blur kernel size from 4 to 3 , causing the model to generate odd-sized features in all layers, due to the convolutional transpose layers.

Original: 3. M ODEL T RAINING
Revised: 3. M ODEL T RAINING

Original: The discriminator D of InﬁnityGAN is similar to the one in the StyleGAN2 method.
Revised: The discriminator D of InﬁnityGAN is similar to the one in the StyleGAN2 method.

Original: The detailed architectures of G and D are presented in Appendix D.
Revised: The detailed architectures of G and D are presented in Appendix D.

Original: The two networks are trained with the nonsaturating logistic loss L adv (Goodfellow et al., 2014), R 1 regularization L R 1 (Mescheder et al., 2018) and path length regularization L path (Karras et al., 2020).
Revised: The two networks are trained with the nonsaturating logistic loss L adv (Goodfellow et al., 2014), R 1 regularization L R 1 (Mescheder et al., 2018) and path length regularization L path (Karras et al., 2020).

Original: Furthermore, to encourage the generator to follow the conditional distribution in the vertical direction, we train G and D with an auxiliary task (Odena et al., 2017) predicting the vertical position of the patch:
Revised: Furthermore, to encourage the generator to follow the conditional distribution in the vertical direction, we train G and D with an auxiliary task (Odena et al., 2017) predicting the vertical position of the patch:

Original: [Equation]
Revised: [Equation]

Original: (5) where ˆ c y is the vertical coordinate predicted by D , and ¯ c y is either (for generated images)
Revised: (5) where ˆ c y is the vertical coordinate predicted by D , and ¯ c y is either (for generated images)

Original: c y = tanh( i y ) or (for real images) the vertical position of the patch in the full image.
Revised: c y = tanh( i y ) or (for real images) the vertical position of the patch in the full image.

Original: We formulate L ar as a regression task.
Revised: We formulate L ar as a regression task.

Original: The overall loss function for the InﬁnityGAN is:
Revised: The overall loss function for the InﬁnityGAN is:

Original: [Equation]
Revised: [Equation]

Original: where λ ’s are the weights.
Revised: where λ ’s are the weights.

Original: [Table]
Revised: [Table]

Original: F u ll
Revised: F u ll

Original: I m a g e
Revised: I m a g e

Original: T e s tti m e e x t e n s i on t o ×
Revised: T e s tti m e e x t e n s i on t o ×

Original:  E XPERIMENTAL R ESULTS
Revised:  E XPERIMENTAL R ESULTS

Original: Datasets.
Revised: Datasets.

Original: We evaluate the ability of synthesizing at extended image sizes on the Flickr-Landscape dataset consists of 450 , 000 high-quality landscape images, which are crawled from the Landscape group on Flickr.
Revised: We evaluate the ability of synthesizing at extended image sizes on the Flickr-Landscape dataset consists of 450 , 000 high-quality landscape images, which are crawled from the Landscape group on Flickr.

Original: For the image outpainting experiments, we evaluate with other baseline methods on scenery-related subsets from the Place365 (Zhou et al., 2017) ( 62 , 500 images) and
Revised: For the image outpainting experiments, we evaluate with other baseline methods on scenery-related subsets from the Place365 (Zhou et al., 2017) ( 62 , 500 images) and

Original: FlickrScenery (Cheng et al., 2021) ( 54 , 710 images) datasets.
Revised: FlickrScenery (Cheng et al., 2021) ( 54 , 710 images) datasets.

Original: Note that the Flickr-Scenery here is different from our Flickr-Landscape.
Revised: Note that the Flickr-Scenery here is different from our Flickr-Landscape.

Original: For image outpainting task, we split the data into 80% , 10% , 10% for training validation test
Revised: For image outpainting task, we split the data into 80% , 10% , 10% for training validation test

Original: All quantitative and qualitative evaluations are conducted on test set.
Revised: All quantitative and qualitative evaluations are conducted on test set.

Original: Hyperparameters.
Revised: Hyperparameters.

Original: We use λ ar = 1 , λ div = 1 , λ R 1 = 10 , and λ path =
Revised: We use λ ar = 1 , λ div = 1 , λ R 1 = 10 , and λ path =

Original: 2 for all datasets.
Revised: 2 for all datasets.

Original: All models are trained with 101 × 101 patches cropped from 197 × 197 real images.
Revised: All models are trained with 101 × 101 patches cropped from 197 × 197 real images.

Original: Since our InﬁnityGAN synthesizes odd-sized images, we choose 101 that maintains a sufﬁcient resolution that humans can still recognize its content.
Revised: Since our InﬁnityGAN synthesizes odd-sized images, we choose 101 that maintains a sufﬁcient resolution that humans can still recognize its content.

Original: On the other hand, 197 is the next output resolution if stacking another upsampling layer to InﬁnityGAN, which also provides 101 × 101 patches a sufﬁcient ﬁeld-of-view.
Revised: On the other hand, 197 is the next output resolution if stacking another upsampling layer to InﬁnityGAN, which also provides 101 × 101 patches a sufﬁcient ﬁeld-of-view.

Original: We adopt the Adam (Kingma & Ba, 2015) optimizer with β 1 = 0 , β 2 = 0 .
Revised: We adopt the Adam (Kingma & Ba, 2015) optimizer with β 1 = 0 , β 2 = 0 .

Original: 99 and a batch sizefor 800,000 iterations.
Revised: 99 and a batch sizefor 800,000 iterations.

Original: More details are presented in Appendix C.
Revised: More details are presented in Appendix C.

Original: Metrics.
Revised: Metrics.

Original: We ﬁrst evaluate Fr´echet Inception Distance ( FID ) (Heusel et al., 2017) at G training resolution.
Revised: We ﬁrst evaluate Fr´echet Inception Distance ( FID ) (Heusel et al., 2017) at G training resolution.

Original: Then, without access to real images at larger sizes, we assume that the real landscape with a larger FoV will share a certain level of self-similarity with its smaller FoV parts.
Revised: Then, without access to real images at larger sizes, we assume that the real landscape with a larger FoV will share a certain level of self-similarity with its smaller FoV parts.

Original: We accordingly propose a ScaleInv FID , which resizes larger images to the training data size with bilinear interpolation, then computes FID.
Revised: We accordingly propose a ScaleInv FID , which resizes larger images to the training data size with bilinear interpolation, then computes FID.

Original: We denote N × ScaleInv FID when the metric is evaluated with images
Revised: We denote N × ScaleInv FID when the metric is evaluated with images

Original: N × larger than the training samples.
Revised: N × larger than the training samples.

Original: Evaluated Method.
Revised: Evaluated Method.

Original: We perform the evaluation on Flickr-Landscape with the following algorithms: − SinGAN.
Revised: We perform the evaluation on Flickr-Landscape with the following algorithms: − SinGAN.

Original: We train an individual SinGAN model for each image.
Revised: We train an individual SinGAN model for each image.

Original: The images at larger sizes are generated by setting spatially enlarged input latent variables.
Revised: The images at larger sizes are generated by setting spatially enlarged input latent variables.

Original: Note that we do not compare with the super-resolution setting from SinGAN since we focus on extending the learned structure rather than super-resolve the high-frequency details.
Revised: Note that we do not compare with the super-resolution setting from SinGAN since we focus on extending the learned structure rather than super-resolve the high-frequency details.

Original: − COCO-GAN.
Revised: − COCO-GAN.

Original: Follow the “Beyond-Boundary Generation” protocol of COCO-GAN, we transfer a trained COCO-GAN model to extended coordinates with a post-training procedure.
Revised: Follow the “Beyond-Boundary Generation” protocol of COCO-GAN, we transfer a trained COCO-GAN model to extended coordinates with a post-training procedure.

Original: − StyleGAN2 (+NCI).
Revised: − StyleGAN2 (+NCI).

Original: We replace the constant input of the original StyleGAN2 with a z l of the same shape, we call such a replacement as “non-constant input (NCI)”.
Revised: We replace the constant input of the original StyleGAN2 with a z l of the same shape, we call such a replacement as “non-constant input (NCI)”.

Original: This modiﬁcation enables
Revised: This modiﬁcation enables

Original: StyleGAN2 to generate images at different output sizes with different z l sizes.
Revised: StyleGAN2 to generate images at different output sizes with different z l sizes.

Original: 4. G ENERATION AT E XTENDED S IZE .
Revised: 4. G ENERATION AT E XTENDED S IZE .

Original: Additional (unfair) protocols for fairness.
Revised: Additional (unfair) protocols for fairness.

Original: We adopt two additional pre- and post-processing to ensure that InﬁnityGAN does not take advantage of its different training resolution.
Revised: We adopt two additional pre- and post-processing to ensure that InﬁnityGAN does not take advantage of its different training resolution.

Original: To ensure InﬁnityGAN is trained with the same amount of information as other methods, images are ﬁrst bilinear interpolated into 128 × 128 before resized into 197 × 197.
Revised: To ensure InﬁnityGAN is trained with the same amount of information as other methods, images are ﬁrst bilinear interpolated into 128 × 128 before resized into 197 × 197.

Original: Next, for all testing sizes in Table 4, InﬁnityGAN generates at 1.54 × ( = 197 / 128) larger size to ensure ﬁnal output images share the same
Revised: Next, for all testing sizes in Table 4, InﬁnityGAN generates at 1.54 × ( = 197 / 128) larger size to ensure ﬁnal output images share the same

Original: FoV with others.
Revised: FoV with others.

Original: In fact, these corrections make the setting disadvantageous for InﬁnityGAN, as it is trained with patches of 50% FoV, generates 54% larger images for all settings, and requires to composite multiple patches for its 1 × ScaleInv FID.
Revised: In fact, these corrections make the setting disadvantageous for InﬁnityGAN, as it is trained with patches of 50% FoV, generates 54% larger images for all settings, and requires to composite multiple patches for its 1 × ScaleInv FID.

Original: Quantitative analysis.
Revised: Quantitative analysis.

Original: For all the FID metrics in Table 1, unfortunately, the numbers are not directly comparable, since InﬁnityGAN is trained with patches with smaller FoV and at a different resolution.
Revised: For all the FID metrics in Table 1, unfortunately, the numbers are not directly comparable, since InﬁnityGAN is trained with patches with smaller FoV and at a different resolution.

Original: Nevertheless, the trend in ScaleInv FID is informative.
Revised: Nevertheless, the trend in ScaleInv FID is informative.

Original: It reﬂects the fact that the global structures generated from the baselines drift far away from the real landscape as the testing FoV enlarges.
Revised: It reﬂects the fact that the global structures generated from the baselines drift far away from the real landscape as the testing FoV enlarges.

Original: Meanwhile, InﬁnityGAN maintains a more steady slope, and surpasses the strongest baseline after 4 × ScaleInv FID.
Revised: Meanwhile, InﬁnityGAN maintains a more steady slope, and surpasses the strongest baseline after 4 × ScaleInv FID.

Original: Showing that InﬁnityGAN indeed performs favorably better than all baselines as the testing size increases.
Revised: Showing that InﬁnityGAN indeed performs favorably better than all baselines as the testing size increases.

Original: InﬁnityGAN synthesize at 512 ×pixels.
Revised: InﬁnityGAN synthesize at 512 ×pixels.

Original: We provide more details and samples in Appendix H.
Revised: We provide more details and samples in Appendix H.

Original: [Figure]
Revised: [Figure]

Original: ere n t s t y l e s synthesizes diverse samples at the same coordinate with different local latent and styles.
Revised: ere n t s t y l e s synthesizes diverse samples at the same coordinate with different local latent and styles.

Original: More samples are shown in Appendix I
Revised: More samples are shown in Appendix I

Original: Qualitative results.
Revised: Qualitative results.

Original: In Figure 4, we show that all baselines fall short of creating reasonable global structures with spatially expanded input latent variables.
Revised: In Figure 4, we show that all baselines fall short of creating reasonable global structures with spatially expanded input latent variables.

Original: COCO-GAN is unable to transfer to new coordinates when the extrapolated coordinates are too far away from the training distribution.
Revised: COCO-GAN is unable to transfer to new coordinates when the extrapolated coordinates are too far away from the training distribution.

Original: Both SinGAN and StyleGAN2 implicitly establish image features based on position encoded by zero padding, assuming the training and testing position encoding should be the same.
Revised: Both SinGAN and StyleGAN2 implicitly establish image features based on position encoded by zero padding, assuming the training and testing position encoding should be the same.

Original: However, when synthesizing at extended image sizes, the inevitable change in the spatial size of the input and the features leads to drastically different position encoding in all model layers.
Revised: However, when synthesizing at extended image sizes, the inevitable change in the spatial size of the input and the features leads to drastically different position encoding in all model layers.

Original: Despite the models can still synthesize reasonable contents near the image border, where the position encoding is still partially correct, they fail to synthesize structurally sound content in the image center.
Revised: Despite the models can still synthesize reasonable contents near the image border, where the position encoding is still partially correct, they fail to synthesize structurally sound content in the image center.

Original: Such a result causes ScaleInv FID to rapidly surge as the extended generation size increases to 1024 × 1024.
Revised: Such a result causes ScaleInv FID to rapidly surge as the extended generation size increases to 1024 × 1024.

Original: Note that at the 16 × setting, StyleGAN2 runs out of memory with a batch size of 1 and does not generate any result.
Revised: Note that at the 16 × setting, StyleGAN2 runs out of memory with a batch size of 1 and does not generate any result.

Original: In comparison, InﬁnityGAN achieves reasonable global structures with ﬁne details.
Revised: In comparison, InﬁnityGAN achieves reasonable global structures with ﬁne details.

Original: Note that the 1024 × 1024 image from
Revised: Note that the 1024 × 1024 image from

Original: InﬁnityGAN is created by compositing 121 independently synthesized patches.
Revised: InﬁnityGAN is created by compositing 121 independently synthesized patches.

Original: With the ability of generating consistent pixel values (Section 3.4), the composition is guaranteed to be seamless.
Revised: With the ability of generating consistent pixel values (Section 3.4), the composition is guaranteed to be seamless.

Original: We provide more comparisons in
Revised: We provide more comparisons in

Original: Appedix E, a larger set of generated samples in Appendix F, results from models trained at a higher resolution in Appendix G, and a very-long synthesis result in Appendix J.
Revised: Appedix E, a larger set of generated samples in Appendix F, results from models trained at a higher resolution in Appendix G, and a very-long synthesis result in Appendix J.

Original: In Figure 5, we further conduct experiments on LSUN bridge and tower datasets, demonstrating
Revised: In Figure 5, we further conduct experiments on LSUN bridge and tower datasets, demonstrating

Original: InﬁnityGAN is applicable on other datasets.
Revised: InﬁnityGAN is applicable on other datasets.

Original: However, since the two datasets are object centric with a low view-angle variation in the vertical direction, InﬁnityGAN frequently ﬁlls the top and bottom area with blank padding textures.
Revised: However, since the two datasets are object centric with a low view-angle variation in the vertical direction, InﬁnityGAN frequently ﬁlls the top and bottom area with blank padding textures.

Original: In Figure 6, we switch different z l and G T styles (i.e., z g projected with the mapping layer) while sharing the same c .
Revised: In Figure 6, we switch different z l and G T styles (i.e., z g projected with the mapping layer) while sharing the same c .

Original: More samples can be found in Appendix I.
Revised: More samples can be found in Appendix I.

Original: The results show that the structure
Revised: The results show that the structure

Original: [Table] [Table]
Revised: [Table] [Table]

Original: and texture are disentangled and modeled separately by G S and G T .
Revised: and texture are disentangled and modeled separately by G S and G T .

Original: The ﬁgure also shows that G S can generate a diverse set of structures realized by different z l .
Revised: The ﬁgure also shows that G S can generate a diverse set of structures realized by different z l .

Original: User study.
Revised: User study.

Original: We ask users to select images with more favorable global structures from two sets of 16 images generated from two competing methods. The evaluation is conducted by pairing our InﬁnityGAN against each of the other baselines.
Revised: We use two-alternative forced choice (2AFC) between InﬁnityGAN and other baselines on the Flickr-Landscape dataset.

Original: We ask each user to compare 30 pairs and collect the results from 50 participants.
Revised: A total of 50 participants with basic knowledge in computer vision engage the study and we conduct 30 queries for each participant.

Revised: For each query, we show two separate grids of 16 random samples from each of the comparing methods and ask the participant to select “the one you think is more realistic and overall structurally plausible.”

Original: As presented in Table 1, the user study shows an over 90% of preference favorable to InﬁnityGAN against allother baselines.
Revised: As presented in Table 1, the user study shows an over 90% of preference favorable to InﬁnityGAN against all baselines.

Original: 4. A BLATION S TUDY : T HE P OSITIONAL I NFORMATION I N G ENERATOR
Revised: 4. A BLATION S TUDY : T HE P OSITIONAL I NFORMATION I N G ENERATOR

Original: As discussed in Section 3.3, we hypothesize that StyleGAN2 highly relies on the positional information from the zero-paddings.
Revised: As discussed in Section 3.3, we hypothesize that StyleGAN2 highly relies on the positional information from the zero-paddings.

Original: In Table 1 and Figure 4, we perform an ablation by removing all paddings from StyleGAN2+NCI, yielding StyleGAN2+NCI+PFG that has no positional information in the generator.
Revised: In Table 1 and Figure 4, we perform an ablation by removing all paddings from StyleGAN2+NCI, yielding StyleGAN2+NCI+PFG that has no positional information in the generator.

Original: The results show that StyleGAN2+NCI+PFG fails to generate reasonable image structures, and signiﬁcantly degrades in all FID settings.
Revised: The results show that StyleGAN2+NCI+PFG fails to generate reasonable image structures, and signiﬁcantly degrades in all FID settings.

Original: Then, with the proposed G S , the positional information is properly provided from z S , and resumes the generator performance back to a reasonable state.
Revised: Then, with the proposed G S , the positional information is properly provided from z S , and resumes the generator performance back to a reasonable state.

Original: 4. A PPLICATIONS
Revised: 4. A PPLICATIONS

Original: Spatial style fusion.
Revised: Spatial style fusion.

Original: Given a single global latent variable z g , the corresponding inﬁnite-pixel image is tied to a single modal of global structures and styles.
Revised: Given a single global latent variable z g , the corresponding inﬁnite-pixel image is tied to a single modal of global structures and styles.

Original: To achieve greater image diversity and allow the user to interactively generate images, we propose a spatial fusion mechanism that can spatially combine two global latent variables with a smooth transition between them.
Revised: To achieve greater image diversity and allow the user to interactively generate images, we propose a spatial fusion mechanism that can spatially combine two global latent variables with a smooth transition between them.

Original: First, we manually deﬁne multiple style centers in the pixel space and then construct an initial fusion map by assigning pixels to the nearest style center.
Revised: First, we manually deﬁne multiple style centers in the pixel space and then construct an initial fusion map by assigning pixels to the nearest style center.

Original: The fusion map consists of one-hot vectors for each pixel, forming a style assignment map.
Revised: The fusion map consists of one-hot vectors for each pixel, forming a style assignment map.

Original: According to the style assignment map, we then propagate the styles in all intermediate layers.
Revised: According to the style assignment map, we then propagate the styles in all intermediate layers.

Original: Please refer to Appendix L for implementation details.
Revised: Please refer to Appendix L for implementation details.

Original: Finally, with the fusion maps annotated for every layer, we can apply the appropriate z g from each style center to each pixel using feature modulation.
Revised: Finally, with the fusion maps annotated for every layer, we can apply the appropriate z g from each style center to each pixel using feature modulation.

Original: Note that the whole procedure has a similar inference speed as the normal synthesis.
Revised: Note that the whole procedure has a similar inference speed as the normal synthesis.

Original: Figure 7 shows synthesized fusion samples.
Revised: Figure 7 shows synthesized fusion samples.

Original: Outpainting via GAN Inversion.
Revised: Outpainting via GAN Inversion.

Original: We leverage the pipeline proposed in In&Out (Cheng et al., 2021) to perform image outpainting with latent variable inversion.
Revised: We leverage the pipeline proposed in In&Out (Cheng et al., 2021) to perform image outpainting with latent variable inversion.

Original: All loss functions follow the ones proposed in In&Out.
Revised: All loss functions follow the ones proposed in In&Out.

Original: We ﬁrst obtain inverted latent variables that generates an image similar to the given real image via GAN inversion techniques, then outpaint the image by expanding z l and z n with their unit Gaussian prior.
Revised: We ﬁrst obtain inverted latent variables that generates an image similar to the given real image via GAN inversion techniques, then outpaint the image by expanding z l and z n with their unit Gaussian prior.

Original: See Appendix K for implementation details.
Revised: See Appendix K for implementation details.

Original: In Table 2, our model performs favorably against all baselines in image outpainting (Boundless (Teterwak et al., 2019),
Revised: In Table 2, our model performs favorably against all baselines in image outpainting (Boundless (Teterwak et al., 2019),

Original: NS-outpaint
Revised: NS-outpaint

Original: (Yang et al., 2019), and In&Out (Cheng et al., 2021)).
Revised: (Yang et al., 2019), and In&Out (Cheng et al., 2021)).

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: As shown in Figure 8, while dealing with a large outpainting area (e.g., panorama), all previous outpainting methods adopt a sequential process that generates a ﬁxed region at each step.
Revised: As shown in Figure 8, while dealing with a large outpainting area (e.g., panorama), all previous outpainting methods adopt a sequential process that generates a ﬁxed region at each step.

Original: This introduces obvious concatenation seams, and tends to produce repetitive contents and black regions after the multiple steps.
Revised: This introduces obvious concatenation seams, and tends to produce repetitive contents and black regions after the multiple steps.

Original: In contrast, with InﬁnityGAN as the image prior in the pipeline of (Cheng et al., 2021), we can directly outpaint arbitrary-size target region from inputs of arbitrary shape.
Revised: In contrast, with InﬁnityGAN as the image prior in the pipeline of (Cheng et al., 2021), we can directly outpaint arbitrary-size target region from inputs of arbitrary shape.

Original: Moreover, in Figure 9, we show that our outpainting pipeline natively supports multi-modal outpainting by sampling different local latent codes in the outpainting area.
Revised: Moreover, in Figure 9, we show that our outpainting pipeline natively supports multi-modal outpainting by sampling different local latent codes in the outpainting area.

Original: Image inbetweening with inverted latent variables.
Revised: Image inbetweening with inverted latent variables.

Original: We show another adaptation of outpainting with model inversion by setting two sets of inverted latent variables at two different spatial locations, then perform spatial style fusion between the variables.
Revised: We show another adaptation of outpainting with model inversion by setting two sets of inverted latent variables at two different spatial locations, then perform spatial style fusion between the variables.

Original: Please refer to Appendix K for implementation details.
Revised: Please refer to Appendix K for implementation details.

Original: As shown in Figure 10, we can naturally inbetween (Lu et al., 2021)
Revised: As shown in Figure 10, we can naturally inbetween (Lu et al., 2021)

Original: the area between two images with arbitrary distance.
Revised: the area between two images with arbitrary distance.

Original: A cyclic panorama of arbitrary width can also be naturally generated by setting the same image on two sides.
Revised: A cyclic panorama of arbitrary width can also be naturally generated by setting the same image on two sides.

Original: Parallel batching.
Revised: Parallel batching.

Original: The nature of spatial-independent generation enables parallel inference on a single image.
Revised: The nature of spatial-independent generation enables parallel inference on a single image.

Original: As shown in Table 3, by stacking a batch of patches together, InﬁnityGAN can signiﬁcantly speed up inference at testing up to 7 . 20 times.
Revised: As shown in Table 3, by stacking a batch of patches together, InﬁnityGAN can signiﬁcantly speed up inference at testing up to 7 . 20 times.

Original: Note that this speed-up is critical for high-resolution image synthesis with a large number of FLOPs.
Revised: Note that this speed-up is critical for high-resolution image synthesis with a large number of FLOPs.

Original:  C ONCLUSIONS
Revised:  C ONCLUSIONS

Original: In this work, we propose and tackle the problem of synthesizing inﬁnite-pixel images, and demonstrate several applications of InﬁnityGAN, including image outpainting and inbetweening.
Revised: In this work, we propose and tackle the problem of synthesizing inﬁnite-pixel images, and demonstrate several applications of InﬁnityGAN, including image outpainting and inbetweening.

Original: Our future work will focus on improving InﬁnityGAN in several aspects.
Revised: Our future work will focus on improving InﬁnityGAN in several aspects.

Original: First, our Flickr-Landscape dataset consists of images taken at different FoVs and distances to the scenes.
Revised: First, our Flickr-Landscape dataset consists of images taken at different FoVs and distances to the scenes.

Original: When InﬁnityGAN composes landscapes of different scales together, synthesized images may contain artifacts.
Revised: When InﬁnityGAN composes landscapes of different scales together, synthesized images may contain artifacts.

Original: Second, similar to the FoV problem, some images intentionally include tree leaves on top of the image as a part of the photography composition.
Revised: Second, similar to the FoV problem, some images intentionally include tree leaves on top of the image as a part of the photography composition.

Original: These greenish textures cause InﬁnityGAN sometimes synthesizing trees or related elements in the sky region.
Revised: These greenish textures cause InﬁnityGAN sometimes synthesizing trees or related elements in the sky region.

Original: Third, there is still a slight decrease in
Revised: Third, there is still a slight decrease in

Original: FID score in comparison to StyleGAN2.
Revised: FID score in comparison to StyleGAN2.

Original: This may be related to the convergence problem in video synthesis (Tian et al., 2021), in which the generator achieves inferior performance if a preceding network (e.g., the motion module in video synthesis) is jointly trained with the image module.
Revised: This may be related to the convergence problem in video synthesis (Tian et al., 2021), in which the generator achieves inferior performance if a preceding network (e.g., the motion module in video synthesis) is jointly trained with the image module.

Original:  E THICS S TATEMENT
Revised:  E THICS S TATEMENT

Original: Our work follows the General Ethical Principles listed at ICLR Code of Ethics ( https://iclr.
Revised: Our work follows the General Ethical Principles listed at ICLR Code of Ethics ( https://iclr.

Original: cc/public/CodeOfEthics )
Revised: cc/public/CodeOfEthics ).

Original: The research in generative modeling is frequently accompanied by concerns about the misuse of manipulating or hallucinating information for improper use.
Revised: The research in generative modeling is frequently accompanied by concerns about the misuse of manipulating or hallucinating information for improper use.

Original: Despite none of the proposed techniques aiming at improving manipulation of ﬁne-grained image detail or hallucinating human activities, we cannot rule out the potential of misusing the framework to recreate fake scenery images for any inappropriate application.
Revised: Despite none of the proposed techniques aiming at improving manipulation of ﬁne-grained image detail or hallucinating human activities, we cannot rule out the potential of misusing the framework to recreate fake scenery images for any inappropriate application.

Original: However, as we do not drastically alter the plausibility of synthesis results in the high-frequency domain, our research is still covered by continuing research in ethical generative modeling and image forensics.
Revised: However, as we do not drastically alter the plausibility of synthesis results in the high-frequency domain, our research is still covered by continuing research in ethical generative modeling and image forensics.

Original: Appendix Table of Contents
Revised: Appendix Table of Contents

Original: A
Revised: A

Original: Comparisons with Concurrent Work .................................................................................
Revised: Comparisons with Concurrent Work .................................................................................

Original: B
Revised: B

Original: Conceptual Comparisons Among Different Tasks .............................................................
Revised: Conceptual Comparisons Among Different Tasks .............................................................

Original: C
Revised: C

Original: Implementation Details of Coordinates ..............................................................................
Revised: Implementation Details of Coordinates ..............................................................................

Original: D
Revised: D

Original: InﬁnityGAN Architecture ....................................................................................................
Revised: InﬁnityGAN Architecture ....................................................................................................

Original: E
Revised: E

Original: More Comparison with baselines ........................................................................................
Revised: More Comparison with baselines ........................................................................................

Original: F
Revised: F

Original: More InﬁnityGAN Qualitative Results ...............................................................................
Revised: More InﬁnityGAN Qualitative Results ...............................................................................

Original: G
Revised: G

Original: InﬁnityGAN Results At A Higher Resolution ....................................................................
Revised: InﬁnityGAN Results At A Higher Resolution ....................................................................

Original: H
Revised: H

Original: More Results on Other Datasets ..........................................................................................
Revised: More Results on Other Datasets ..........................................................................................

Original: I
Revised: I

Original: More Diversity Visualization ...............................................................................................
Revised: More Diversity Visualization ...............................................................................................

Original: J
Revised: J

Original: Our Best Attempt in Including An Everlasting Image ......................................................
Revised: Our Best Attempt in Including An Everlasting Image ......................................................

Original: K
Revised: K

Original: Implementation Details of Image Outpainting and Inbetweening via Inversion ............
Revised: Implementation Details of Image Outpainting and Inbetweening via Inversion ............

Original: L
Revised: L

Original: Implementation Details of Spatial Style Fusion .................................................................
Revised: Implementation Details of Spatial Style Fusion .................................................................

Original: M More Qualitative Results of Outpainting via Inversion ....................................................
Revised: M More Qualitative Results of Outpainting via Inversion ....................................................

Original: N
Revised: N

Original: More Qualitative Results of Inbetweening via Inversion ..................................................
Revised: More Qualitative Results of Inbetweening via Inversion ..................................................

Original: O
Revised: O

Original: More Qualitative Results of  Panoramic Inbetweening via Inversion ...................
Revised: More Qualitative Results of Cyclic Panoramic Inbetweening via Inversion ...................

Original: P
Revised: P

Original: Experimental Details of The Speed Benchmark with Parallel Batching .........................
Revised: Experimental Details of The Speed Benchmark with Parallel Batching .........................

Original: Q
Revised: Q

Original: Ablation: Importance of Feature Unfolding ......................................................................
Revised: Ablation: Importance of Feature Unfolding ......................................................................

Original: R
Revised: R

Original: Limitations ............................................................................................................................. 
Revised: Limitations ............................................................................................................................. S

Revised: Implementation Illustration of ScaleInv FID .....................................................................

Revised: T

Revised: More Comparisons with Texture Synthesis Method .........................................................

Original: A C OMPARISONS WITH C ONCURRENT W ORK A. ALIS (S KOROKHODOV ET AL ., 2021)
Revised: A C OMPARISONS WITH C ONCURRENT W ORK A. ALIS (S KOROKHODOV ET AL ., 2021)

Original: ALIS is a concurrent work that achieves a similar application in inﬁnite-pixel generation by iteratively inbetween pairs of anchor patches.
Revised: ALIS is a concurrent work that achieves a similar application in inﬁnite-pixel generation by iteratively inbetween pairs of anchor patches.

Original: Here, we discuss some of the critical differences.
Revised: Here, we discuss some of the critical differences.

Original: ALIS has limited ability in extending vertically.
Revised: ALIS has limited ability in extending vertically.

Original: InﬁnityGAN achieves both vertical and horizontal extension while maintaining a plausible holistic appearance.
Revised: InﬁnityGAN achieves both vertical and horizontal extension while maintaining a plausible holistic appearance.

Original: ALIS only presents horizontal extension in the paper.
Revised: ALIS only presents horizontal extension in the paper.

Original: Vertically connecting anchors on any dataset used in their paper will produce invalid structures (e.g., layered landscapes periodically stacking in the sky).
Revised: Vertically connecting anchors on any dataset used in their paper will produce invalid structures (e.g., layered landscapes periodically stacking in the sky).

Original: Therefore, vertical anchor connection is limited to certain datasets, such as pattern-like textures or satellite images.
Revised: Therefore, vertical anchor connection is limited to certain datasets, such as pattern-like textures or satellite images.

Original: Differences in the problem formulation.
Revised: Differences in the problem formulation.

Original: InﬁnityGAN directly models each inﬁnite-pixel image with a shared global latent variable using an implicit function.
Revised: InﬁnityGAN directly models each inﬁnite-pixel image with a shared global latent variable using an implicit function.

Original: In contrast, ALIS learns to inbetween two independent global latent variables.
Revised: In contrast, ALIS learns to inbetween two independent global latent variables.

Original: Furthermore, InﬁnityGAN can still achieve the inbetweening setup similar to ALIS with spatial style fusion.
Revised: Furthermore, InﬁnityGAN can still achieve the inbetweening setup similar to ALIS with spatial style fusion.

Original: However, applying ALIS to synthesize images with a shared global context will lead to periodically repeating patches, as shown in Figure 13.
Revised: However, applying ALIS to synthesize images with a shared global context will lead to periodically repeating patches, as shown in Figure 13.

Original: InﬁnityGAN allows free-form anchor placements without training.
Revised: InﬁnityGAN allows free-form anchor placements without training.

Original: ALIS has to designate a constant relative position between the anchors before training starts.
Revised: ALIS has to designate a constant relative position between the anchors before training starts.

Original: It is also non-trivial for ALIS to inbetween multiple anchors.
Revised: It is also non-trivial for ALIS to inbetween multiple anchors.

Original: Meanwhile, as shown in Figure 12, our training-free spatial style fusion allows placing any number of anchors at any place.
Revised: Meanwhile, as shown in Figure 12, our training-free spatial style fusion allows placing any number of anchors at any place.

Original: However, the ﬂexibility also comes with a trade-off, our spatial style fusion is not trained with adversarial learning, causing the synthesis performance not compatible to the regularly synthesized images.
Revised: However, the ﬂexibility also comes with a trade-off, our spatial style fusion is not trained with adversarial learning, causing the synthesis performance not compatible to the regularly synthesized images.

Original: Following the ALIS evaluation protocol, we train our InﬁnityGAN on the ALIS LHQ dataset at 256 × 256 resolution, yielding an
Revised: Following the ALIS evaluation protocol, we train our InﬁnityGAN on the ALIS LHQ dataset at 256 × 256 resolution, yielding an

Original: FID 11 .
Revised: FID 11 .

Original: 82 with regular synthesis and an ∞ -FID 19 .
Revised: 82 with regular synthesis and an ∞ -FID 19 .

Original: 22 with spatial style fusion.
Revised: 22 with spatial style fusion.

Original: In contrast, ALIS shows negligible performance gap between FID ( 10 . 48 ) and ∞ -FID ( 10 . 64 ).
Revised: In contrast, ALIS shows negligible performance gap between FID ( 10 . 48 ) and ∞ -FID ( 10 . 64 ).

Original: Such a result suggest a ﬂexibility-accuracy trade-off between InﬁnityGAN and ALIS.
Revised: Such a result suggest a ﬂexibility-accuracy trade-off between InﬁnityGAN and ALIS.

Original: ALIS generates blocky and high-frequency lattice artifacts.
Revised: ALIS generates blocky and high-frequency lattice artifacts.

Original: ALIS adopts the generation-by-parts design from COCO-GAN.
Revised: ALIS adopts the generation-by-parts design from COCO-GAN.

Original: As shown in Figure 11, a critical consequence is creating the blocky and lattice artifacts between patches, since the inter-patch continuity is unstably maintained with adversarial learning.
Revised: As shown in Figure 11, a critical consequence is creating the blocky and lattice artifacts between patches, since the inter-patch continuity is unstably maintained with adversarial learning.

Original: In contrast, InﬁnityGAN is an improved version of COCO-GAN that the interpatch continuity is guaranteed with implicit-function and padding-free generator design.
Revised: In contrast, InﬁnityGAN is an improved version of COCO-GAN that the interpatch continuity is guaranteed with implicit-function and padding-free generator design.

Original: ALIS still suffers from content repetition.
Revised: ALIS still suffers from content repetition.

Original: The content repetition problem is discussed in the
Revised: The content repetition problem is discussed in the

Original: ALIS paper as a limitation.
Revised: ALIS paper as a limitation.

Original: InﬁnityGAN addresses the issue with a local latent space and enforces the contribution of the local variables with a diversity loss (2) between z l and z S .
Revised: InﬁnityGAN addresses the issue with a local latent space and enforces the contribution of the local variables with a diversity loss (2) between z l and z S .

Original: We demonstrate that InﬁnityGAN does not have the content repetition problem in Figure 30.
Revised: We demonstrate that InﬁnityGAN does not have the content repetition problem in Figure 31.

Original: [Figure]
Revised: [Figure]

Original: We train ALIS with the ofﬁcial implementation at 1024 × 1024 resolution.
Revised: We train ALIS with the ofﬁcial implementation at 1024 × 1024 resolution.

Original: We focus on the failure cases caused by COCO-GAN-based generation-by-parts framework, which artiﬁcially enforces the learned inter-patch continuity with adversarial learning. 
Revised: We focus on the failure cases caused by COCO-GAN-based generation-by-parts framework, which artiﬁcially enforces the learned inter-patch continuity with adversarial learning. [Figure]

Original: [Figure]
Revised: [Figure]

Original: MS-PIE found the positional information is crucial for GANs training.
Revised: MS-PIE found the positional information is crucial for GANs training.

Original: The paper explores different positional encoding schema, including sinusoidal coordinate encoding and padding removal.
Revised: The paper explores different positional encoding schema, including sinusoidal coordinate encoding and padding removal.

Original: We further discuss the relation and distinction between MS-PIE and InﬁnityGAN in these two modules.
Revised: We further discuss the relation and distinction between MS-PIE and InﬁnityGAN in these two modules.

Original: Extensible coordinate encoding.
Revised: Extensible coordinate encoding.

Original: The “expand” conﬁguration of MS-PIE allows coordinate value extrapolation in the spatial dimension.
Revised: The “expand” conﬁguration of MS-PIE allows coordinate value extrapolation in the spatial dimension.

Original: However, its training sticks to a ﬁxed coordinate matrix for each synthesis scale, and the framework aligns the real images to the ﬁxed coordinate matrix with bilinear interpolation 1 during training.
Revised: However, its training sticks to a ﬁxed coordinate matrix for each synthesis scale, and the framework aligns the real images to the ﬁxed coordinate matrix with bilinear interpolation 1 during training.

Original: Such a design causes the synthesized content to attach to the coordinate matrix, thus inevitably creates repetitive content when the coordinate value periodically repeats as the matrix expanding.
Revised: Such a design causes the synthesized content to attach to the coordinate matrix, thus inevitably creates repetitive content when the coordinate value periodically repeats as the matrix expanding.

Original: In Figure 14, we train a no-padding StyleGAN2 model with MSPIE-expand setting (i.e., conﬁg (k) in Table 5 of MS-PIE paper) on our Flickr-Landscape dataset.
Revised: In Figure 14, we train a no-padding StyleGAN2 model with MSPIE-expand setting (i.e., conﬁg (k) in Table 5 of MS-PIE paper) on our Flickr-Landscape dataset.

Original: The model is trained at scales 256, 384 and 512, then test at scale 1408.[Figure]
Revised: The model is trained at scales 256, 384 and 512, then test at scale 1408.

Original: Distinctions between no-padding and padding-free generator.
Revised: Distinctions between no-padding and padding-free generator.

Original: The no-padding generator (NPG) in MS-PIE is conceptually related to our padding-free generator (PFG), but fundamentally different while considering how the information of border pixels in the feature space is processed.
Revised: The no-padding generator (NPG) in MS-PIE is conceptually related to our padding-free generator (PFG), but fundamentally different while considering how the information of border pixels in the feature space is processed.

Original: NPG gradually loses the information from border pixels after each convolution, since the border pixels are less visited than the other pixels (consider a 3 × 3 convolutional kernel, most pixels will be scanned for 9 times, while edge pixels will be only scanned for 4 times, and corner pixels only 1 time).
Revised: NPG gradually loses the information from border pixels after each convolution, since the border pixels are less visited than the other pixels (consider a 3 × 3 convolutional kernel, most pixels will be scanned for 9 times, while edge pixels will be only scanned for 4 times, and corner pixels only 1 time).

Original: Further note that the information loss worsens exponentially as the network stacking more convolutional layers.
Revised: Further note that the information loss worsens exponentially as the network stacking more convolutional layers.

Original: In contrast, our PFG pads feature values from the neighbor context before the no-padding convolution is applied.
Revised: In contrast, our PFG pads feature values from the neighbor context before the no-padding convolution is applied.

Original: The information loss caused by the no-padding convolution is a natural way to discard the information that is spatially too far away from and less related to the current context.
Revised: The information loss caused by the no-padding convolution is a natural way to discard the information that is spatially too far away from and less related to the current context.

Original: B C ONCEPTUAL C OMPARISONS A MONG D IFFERENT T ASKS
Revised: B C ONCEPTUAL C OMPARISONS A MONG D IFFERENT T ASKS

Original: [Figure]
Revised: [Figure]

Revised: ) COCO-GAN : COCO-GAN can only synthesize samples slightly

Revised: larger than

Revised: its

Revised: training distribution

Revised: (e) InﬁnityGAN : Ours

Original:  without an input condition.
Revised: InﬁnityGAN can synthesize a more favorable global structure at arbitrary resolutions without an input condition

Original: C I MPLEMENTATION D ETAILS OF C OORDINATES
Revised: C I MPLEMENTATION D ETAILS OF C OORDINATES

Original: We ﬁrst derive the receptive ﬁeld size R of an L -layer G S after adding the 7 × 7 feature unfolding to all layers.
Revised: We ﬁrst derive the receptive ﬁeld size R of an L -layer G S after adding the 7 × 7 feature unfolding to all layers.

Original: Assume that the size of z S (i.e., output of G S , and input of G T ) is M .
Revised: Assume that the size of z S (i.e., output of G S , and input of G T ) is M .

Original: The value of R can be derived as M +(2 × 3) × L , where 3 is the half-size of the feature unfolding area.
Revised: The value of R can be derived as M +(2 × 3) × L , where 3 is the half-size of the feature unfolding area.

Original: In practice, with an architecture shown in Figure 17 with training patch size 101 × 101, we have M = 11 , L = 4 , and
Revised: In practice, with an architecture shown in Figure 18 with training patch size 101 × 101, we have M = 11 , L = 4 , and

Original: R = 35 .
Revised: R = 35 .

Original: Horizontal direction.
Revised: Horizontal direction.

Original: In order to avoid the generator discovering and exploiting any property of the periodic coordinate, we use a period T much larger than R .
Revised: In order to avoid the generator discovering and exploiting any property of the periodic coordinate, we use a period T much larger than R .

Original: Meanwhile, the period should be short enough to avoid the differences between consecutive coordinates vanish to almost zero.
Revised: Meanwhile, the period should be short enough to avoid the differences between consecutive coordinates vanish to almost zero.

Original: In practice, we use T = 4 × R for both of the cosine and sine coordinates.
Revised: In practice, we use T = 4 × R for both of the cosine and sine coordinates.

Original: Vertical direction.
Revised: Vertical direction.

Original: We utilize the property of the tanh function that its slope rapidly saturates to nearly zero and its value range is bounded by [ − 1 , 1] .
Revised: We utilize the property of the tanh function that its slope rapidly saturates to nearly zero and its value range is bounded by [ − 1 , 1] .

Original: Such a property is aligned to the property of real-world scenery—the structural interactions among landscape objects are mostly concentrated around the horizon and rapidly saturates to a single modal (i.e., sky, ground, or water) in the vertical direction.
Revised: Such a property is aligned to the property of real-world scenery—the structural interactions among landscape objects are mostly concentrated around the horizon and rapidly saturates to a single modal (i.e., sky, ground, or water) in the vertical direction.

Original: There are two hyperparameters while constructing the tanh coordinates: (i) a pair of cutoff points, and (ii) the sampling period of z l in the spatial dimension.
Revised: There are two hyperparameters while constructing the tanh coordinates: (i) a pair of cutoff points, and (ii) the sampling period of z l in the spatial dimension.

Original: • Cutoff points ( c cut ) are a pair of values that, during training, we only sample the coordinates between the value pair.
Revised: • Cutoff points ( c cut ) are a pair of values that, during training, we only sample the coordinates between the value pair.

Original: Such a hyperparameter is required since we are unable to sample all coordinates from an inﬁnitely large coordinate system within the ﬁnite training steps.
Revised: Such a hyperparameter is required since we are unable to sample all coordinates from an inﬁnitely large coordinate system within the ﬁnite training steps.

Original: In practice, we set the cutoff points at ± 0 .
Revised: In practice, we set the cutoff points at ± 0 .

Original: 995 of tanh-projected coordinate.
Revised: 995 of tanh-projected coordinate.

Original: Note that the underlying effect of using different cutoff point values is not well-investigated, we do not observe obvious changes with slightly different values.
Revised: Note that the underlying effect of using different cutoff point values is not well-investigated, we do not observe obvious changes with slightly different values.

Original: • Sampling period ( d l ) deﬁnes the distance between two spatially consecutive values of z l are sampled within a training sample.
Revised: • Sampling period ( d l ) deﬁnes the distance between two spatially consecutive values of z l are sampled within a training sample.

Original: Such a value relates to the grain of the representation that the generator overall models.
Revised: Such a value relates to the grain of the representation that the generator overall models.

Original: It is equivalent to the occupation ratio of z l between the cutoff points.
Revised: It is equivalent to the occupation ratio of z l between the cutoff points.

Original: Thus we can alternatively deﬁne a hyperparameter V that deﬁnes the occupation ratio of z l by RR + V .
Revised: Thus we can alternatively deﬁne a hyperparameter V that deﬁnes the occupation ratio of z l by RR + V .

Original: Then, the sampling period can be derived with d l = 2 × c cut × 1 R + V .
Revised: Then, the sampling period can be derived with d l = 2 × c cut × 1 R + V .

Original: In practice, we use V = 10 , resulting in d l = 0 .
Revised: In practice, we use V = 10 , resulting in d l = 0 .

Original: 1¯3 .
Revised: 1¯3 .

Original: C. M ORE D ISCUSSIONS ON T HE C HOICE OF C OORDINATE S YSTEM
Revised: C. M ORE D ISCUSSIONS ON T HE C HOICE OF C OORDINATE S YSTEM

Original: In this paper, we mainly tackle the scenery image datasets, such as landscape, LSUN bridge and
Revised: In this paper, we mainly tackle the scenery image datasets, such as landscape, LSUN bridge and

Original: LSUN tower.
Revised: LSUN tower.

Original: The coordinate prior we introduced in Section 3.2 is speciﬁcally designed for such types of data, which has self-similarity in the horizontal direction and rapid mode-saturation in the vertical direction.
Revised: The coordinate prior we introduced in Section 3.2 is speciﬁcally designed for such types of data, which has self-similarity in the horizontal direction and rapid mode-saturation in the vertical direction.

Original: However, we want to emphasize that the choice of coordinate system is a hyperparameter depending on the dataset.
Revised: However, we want to emphasize that the choice of coordinate system is a hyperparameter depending on the dataset.

Original: For instance, in Figure 16, we show that InﬁnityGAN can also work on satellite image dataset (Isola et al., 2017), which is more frequently used in texture synthesis task.
Revised: For instance, in Figure 16, we show that InﬁnityGAN can also work on satellite image dataset (Isola et al., 2017), which is more frequently used in texture synthesis task.

Original: But a more natural choice of coordinate system in such a setting is using periodic coordinates in both horizontal and vertical directions.
Revised: A more natural choice of coordinate system in such a setting is using periodic coordinates in both horizontal and vertical directions.

Revised: Nevertheless, in Figure 17 we observe that InﬁnityGAN can still produce visually plausible and globally sound appearances with using saturating (i.e., tanh) coordinates in the vertical direction and periodic coordinates in the horizontal direction.

Original: In Table 4, we identify three categories of coordinate systems, but these options may not have covered all possibilities.
Revised: In Table 4, we identify three categories of coordinate systems, but these options may not have covered all possibilities.

Original: One may need to be aware of the characteristics of the data and the intended outcome before selecting the appropriate coordinate system.
Revised: One may need to be aware of the characteristics of the data and the intended outcome before selecting the appropriate coordinate system.

Original: [Figure] [Table] 
Revised: [Table] [Figure]

Original: [Figure][Table] [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Figure]

Original: D I NFINITY GAN A RCHITECTURE
Revised: D I NFINITY GAN A RCHITECTURE

Original: [Table]
Revised: [Table]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Table]
Revised: [Table]

Original: (B × P ×
Revised: (B × P ×

Original: [Table]
Revised: [Table]

Original: ×
Revised: ×

Original: [Table]
Revised: [Table]

Original: ]) (B × Q × H/
Revised: ]) (B × Q × H/

Original: [Table]
Revised: [Table]

Original: ×
Revised: ×

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: ×
Revised: ×

Original: [Table]
Revised: [Table]

Original: ×
Revised: ×

Original: [Table]
Revised: [Table]

Original: ×
Revised: ×

Original: [Table]
Revised: [Table]

Original: Input
Revised: Input

Original: Output
Revised: Output

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: E M ORE C OMPARISON WITH BASELINES
Revised: E M ORE C OMPARISON WITH BASELINES

Original: [Figure]
Revised: [Figure]

Original: F M ORE I NFINITY GAN Q UALITATIVE R ESULTS
Revised: F M ORE I NFINITY GAN Q UALITATIVE R ESULTS

Original: G I NFINITY GAN R ESULTS A T A H IGHER R ESOLUTION
Revised: G I NFINITY GAN R ESULTS A T A H IGHER R ESOLUTION

Original: H M ORE R ESULTS ON O THER D ATASETS
Revised: H M ORE R ESULTS ON O THER D ATASETS

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: D i ff ere n t l o c a l l a t e n t va r i a b l e s
Revised: D i ff ere n t l o c a l l a t e n t va r i a b l e s

Original: J
Revised: J

Original: O UR B EST A TTEMPT IN I NCLUDING
Revised: O UR B EST A TTEMPT IN I NCLUDING

Original: A N E VERLASTING I MAGE
Revised: A N E VERLASTING I MAGE

Original: A
Revised: A

Original: B
Revised: B

Original: C
Revised: C

Original: D
Revised: D

Original: E
Revised: E

Original: F F
Revised: F F

Original: E
Revised: E

Original: D
Revised: D

Original: C
Revised: C

Original: B
Revised: B

Original: A
Revised: A

Original: K I MPLEMENTATION D ETAILS OF I MAGE O UTPAINTING AND I NBETWEENING VIA I NVERSION
Revised: K I MPLEMENTATION D ETAILS OF I MAGE O UTPAINTING AND I NBETWEENING VIA I NVERSION

Original: Our pipeline is similar to that of InOut (Cheng et al., 2021).
Revised: Our pipeline is similar to that of InOut (Cheng et al., 2021).

Original: With a given image x , the objective of
Revised: With a given image x , the objective of

Original: GAN-model inversion is to recover a set of generator-parameter-dependent input latent variables z ∗ that can synthesize a resulting image x ∗ that is similar to x .
Revised: GAN-model inversion is to recover a set of generator-parameter-dependent input latent variables z ∗ that can synthesize a resulting image x ∗ that is similar to x .

Original: There exist multiple different implementations to recover z ∗ , we adopt the gradient-descent-based method, which optimizes z ∗ as a set of learnable parameters with carefully designed objective functions.
Revised: There exist multiple different implementations to recover z ∗ , we adopt the gradient-descent-based method, which optimizes z ∗ as a set of learnable parameters with carefully designed objective functions.

Original: In the context of InﬁnityGAN, we optimize four groups of variables: z g , z l , style ( z T ) and z n .
Revised: In the context of InﬁnityGAN, we optimize four groups of variables: z g , z l , style ( z T ) and z n .

Original: In particular, we uses the W + space formulation (Abdal et al., 2020; Wulff & Torralba, 2020) for z T , named z +T , where each layer in G T has its own set of z T i and each of the z T i is optimized separately.
Revised: In particular, we uses the W + space formulation (Abdal et al., 2020; Wulff & Torralba, 2020) for z T , named z +T , where each layer in G T has its own set of z T i and each of the z T i is optimized separately.

Original: Objective functions.
Revised: Objective functions.

Original: We ﬁrst introduce two image-distance losses to the inversion objectives:
Revised: We ﬁrst introduce two image-distance losses to the inversion objectives:

Original: [Equation]
Revised: [Equation]

Original: which LPIPS is the learned perceptual distance proposed by Zhang et al.
Revised: which LPIPS is the learned perceptual distance proposed by Zhang et al.

Original: (Zhang et al
Revised: (Zhang et al

Original: [Equation]
Revised: [Equation]

Original: We utilize the Gaussianized latent space technique proposed by Wulff and Torralba (Wulff &
Revised: We utilize the Gaussianized latent space technique proposed by Wulff and Torralba (Wulff &

Original: Torralba, 2020), which uses a LeakyReLU of slope 5 to discount the last activation function (a
Revised: Torralba, 2020), which uses a LeakyReLU of slope 5 to discount the last activation function (a

Original: LeakyReLU with a slope 0 . 2 ) in the StyleGAN2 mapping layer, and recovers Gaussian-like marginal distribution of z +T .
Revised: LeakyReLU with a slope 0 . 2 ) in the StyleGAN2 mapping layer, and recovers Gaussian-like marginal distribution of z +T .

Original: With the Gaussian distribution prior, we can use the empirical mean µ + T and covariance matrix Σ + T (computed by sampling 

Revised: With the Gaussian distribution prior, we can use the empirical mean µ + T and covariance matrix Σ + T (computed by sampling

Original: , 000 z +T via z g ) to recover an estimated z +T distribution.
Revised: , 000 z +T via z g ) to recover an estimated z +T distribution.

Original: With the empirical statistics, (Wulff & Torralba, 2020) proposes to compute Mahalanobis distance:
Revised: With the empirical statistics, (Wulff & Torralba, 2020) proposes to compute Mahalanobis distance:

Original: [Equation]
Revised: [Equation]

Original: Then, we construct a prior loss L prior (Wulff & Torralba, 2020) that regularizes the z g , z l and z +T with Mahalanobis distance:
Revised: Then, we construct a prior loss L prior (Wulff & Torralba, 2020) that regularizes the z g , z l and z +T with Mahalanobis distance:

Original: [Equation]
Revised: [Equation]

Original: which λ α , λ β and λ γ are weight factors.
Revised: which λ α , λ β and λ γ are weight factors.

Original: For the cases of z g and z l with zero means and unit variances, the prior loss degenerates to an l 2 loss.
Revised: For the cases of z g and z l with zero means and unit variances, the prior loss degenerates to an l 2 loss.

Original: Following StyleGAN2, we adopt a noise regularization loss L nreg and noise renormalization.
Revised: Following StyleGAN2, we adopt a noise regularization loss L nreg and noise renormalization.

Original: The full objective function of the inversion is:
Revised: The full objective function of the inversion is:

Original: [Equation]
Revised: [Equation]

Original: where λ s are the weighting factor of each loss terms.
Revised: where λ s are the weighting factor of each loss terms.

Original: Hyperparameters.
Revised: Hyperparameters.

Original: For all tasks and datasets, we set λ pix = 10 , λ percept = 10 , λ nreg = 1 , 000 , λ α = 10 , λ β = 10 , and λ γ = 0 .
Revised: For all tasks and datasets, we set λ pix = 10 , λ percept = 10 , λ nreg = 1 , 000 , λ α = 10 , λ β = 10 , and λ γ = 0 .

Original: We use Adam (Kingma & Ba, 2015) optimizer with a learning annealing (Karras et al., 2020) from 0 .
Revised: We use Adam (Kingma & Ba, 2015) optimizer with a learning annealing (Karras et al., 2020) from 0 .

Original: 1 to 0 for 1000 iterations.
Revised: 1 to 0 for 1000 iterations.

Original: We use a batch size of 1 to avoid batch samples interfere with each other.
Revised: We use a batch size of 1 to avoid batch samples interfere with each other.

Original: Despite we observe batched inversion can sometimes yield superior results, it is not a conventional setting for real-world applications that batched inputs are mostly unavailable, it also signiﬁcantly increases the stochasticity while reproducing the results.
Revised: Despite we observe batched inversion can sometimes yield superior results, it is not a conventional setting for real-world applications that batched inputs are mostly unavailable, it also signiﬁcantly increases the stochasticity while reproducing the results.

Original: Outpainting and inbetweening with inverted latent variables.
Revised: Outpainting and inbetweening with inverted latent variables.

Original: With the inverted latent variables, we perform image outpainting by spatially extend the z ∗ l with its unit Gaussian prior, while using z ∗ g and z + T ∗ everywhere.
Revised: With the inverted latent variables, we perform image outpainting by spatially extend the z ∗ l with its unit Gaussian prior, while using z ∗ g and z + T ∗ everywhere.

Original: For image inbetweening, z ∗ l is also extended with its unit Gaussian prior, while z ∗ g and z +T ∗ are fused with spatial style fusion.
Revised: For image inbetweening, z ∗ l is also extended with its unit Gaussian prior, while z ∗ g and z +T ∗ are fused with spatial style fusion.

Original: Notice that we do not optimize c in our pipeline, since inverting c is non-trivial and requires additional regularization losses.
Revised: Notice that we do not optimize c in our pipeline, since inverting c is non-trivial and requires additional regularization losses.

Original: This introduces a limitation that the spatial position of the images is ﬁxed after the inversion, the users have to perform the inversion optimization again if they want to assign z ∗ l to a different location.
Revised: This introduces a limitation that the spatial position of the images is ﬁxed after the inversion, the users have to perform the inversion optimization again if they want to assign z ∗ l to a different location.

Original: Another limitation is that, despite the use of the prior loss, some dimensions of the inverted latents tend to drift far away from the normal distribution.
Revised: Another limitation is that, despite the use of the prior loss, some dimensions of the inverted latents tend to drift far away from the normal distribution.

Original: In combination with the use of the W + space of z T , the inverted latents are of high-instability, sometimes introduce checkerboard-like artifacts, and frequently mix multiple irrelevant contexts together.
Revised: In combination with the use of the W + space of z T , the inverted latents are of high-instability, sometimes introduce checkerboard-like artifacts, and frequently mix multiple irrelevant contexts together.

Original: We develop an interactive tool (released along with the code release) that allows users interactively and regionally resampling the undesired local latent variables z l in the outpainting area.
Revised: We develop an interactive tool (released along with the code release) that allows users interactively and regionally resampling the undesired local latent variables z l in the outpainting area.

Original: Such a limitation is highly related to the generalization of the inverted latents, we put it as an important future working direction.
Revised: Such a limitation is highly related to the generalization of the inverted latents, we put it as an important future working direction.

Original: L I MPLEMENTATION D ETAILS OF S PATIAL S TYLE F USION
Revised: L I MPLEMENTATION D ETAILS OF S PATIAL S TYLE F USION

Original: [Figure]
Revised: [Figure]

Original: To achieve spatial style fusion in the existing InﬁnityGAN pipeline, we introduce two additional procedures: “style fusion map creation” and “fused modulation-demodulation”.
Revised: To achieve spatial style fusion in the existing InﬁnityGAN pipeline, we introduce two additional procedures: “style fusion map creation” and “fused modulation-demodulation”.

Original: The former creates per-layer style fusion maps that specify the geometry of the style fusion area.
Revised: The former creates per-layer style fusion maps that specify the geometry of the style fusion area.

Original: The latter one is a modiﬁed version of feature modulation-demodulation that processes the volumetric styles created from the style fusion map.
Revised: The latter one is a modiﬁed version of feature modulation-demodulation that processes the volumetric styles created from the style fusion map.

Original: Style fusion map creation.
Revised: Style fusion map creation.

Original: Given N style centers designated by the user, in the pixel space, the target of style fusion map creation is to construct a set of style fusion maps for each layer of both G S and G T .
Revised: Given N style centers designated by the user, in the pixel space, the target of style fusion map creation is to construct a set of style fusion maps for each layer of both G S and G T .

Original: The fusion map is a spatially-shaped (i.e., batch × N × H × W ) tensor with N channels that speciﬁes the weight of the style for each spatial location, which the weights sums up to one across the N dimension for each spatial position.
Revised: The fusion map is a spatially-shaped (i.e., batch × N × H × W ) tensor with N channels that speciﬁes the weight of the style for each spatial location, which the weights sums up to one across the N dimension for each spatial position.

Original: We ﬁrst construct an initial fusion map in the pixel space by ﬁnding the spatially nearest style center, then assign a one-hot label for each spatial position in the initial fusion map.
Revised: We ﬁrst construct an initial fusion map in the pixel space by ﬁnding the spatially nearest style center, then assign a one-hot label for each spatial position in the initial fusion map.

Original: Since the spatial style fusion happens in all layers in the generator, we therefore reversely propagate the fusion map from the output of the generator to its input, we call such a procedure fusion map calibration .
Revised: Since the spatial style fusion happens in all layers in the generator, we therefore reversely propagate the fusion map from the output of the generator to its input, we call such a procedure fusion map calibration .

Original: We show an illustration of fusion map calibration in Figure 31(b).
Revised: We show an illustration of fusion map calibration in Figure 32(b).

Original: The fusion map calibration starts from the image space and sequentially backward-constructs the fusion maps for all generator layers.
Revised: The fusion map calibration starts from the image space and sequentially backward-constructs the fusion maps for all generator layers.

Original: For each pair (output-side and input-side) of the fusion maps in a network layer, we match the spatial dimension of the fusion map pair by padding or interpolating the output fusion map into a spatially aligned input fusion map.
Revised: For each pair (output-side and input-side) of the fusion maps in a network layer, we match the spatial dimension of the fusion map pair by padding or interpolating the output fusion map into a spatially aligned input fusion map.

Original: For different types of intermediate layers, the underlying implementation of the fusion map calibration can be slightly different, but a shared principle is to maintain a consistent geometrical position of the style fusion center throughout the generator.
Revised: For different types of intermediate layers, the underlying implementation of the fusion map calibration can be slightly different, but a shared principle is to maintain a consistent geometrical position of the style fusion center throughout the generator.

Original: In practice, such a binary map creates a sharp style transition that produces visible straight lines dividing the style regions.
Revised: In practice, such a binary map creates a sharp style transition that produces visible straight lines dividing the style regions.

Original: Accordingly, we apply a mean ﬁlter that smooths the style transition border.
Revised: Accordingly, we apply a mean ﬁlter that smooths the style transition border.

Original: While different kernel sizes for the mean ﬁlter only alter the range of style transition and the visual smoothness, we use a kernel size of 127 in our experiments as it empirically produces good visual results.
Revised: While different kernel sizes for the mean ﬁlter only alter the range of style transition and the visual smoothness, we use a kernel size of 127 in our experiments as it empirically produces good visual results.

Original: Fused modulation-demodulation.
Revised: Fused modulation-demodulation.

Original: After constructing the per-layer style fusion map, we can use the fusion maps to create volumetric styles (i.e., batch × D
Revised: After constructing the per-layer style fusion map, we can use the fusion maps to create volumetric styles (i.e., batch × D

Original: × H × W ) by weighted-sum the styles by the importance weights ( D -channel dimension) in each spatial position.
Revised: × H × W ) by weighted-sum the styles by the importance weights ( D -channel dimension) in each spatial position.

Original: The volumetric styles are applied to each layer of the generator.
Revised: The volumetric styles are applied to each layer of the generator.

Original: As the feature modulation-demodulation strategy used in both
Revised: As the feature modulation-demodulation strategy used in both

Original: StyleGAN2 and InﬁnityGAN is a pixel-wise operator, we can easily adapt it to volumetric styles.
Revised: StyleGAN2 and InﬁnityGAN is a pixel-wise operator, we can easily adapt it to volumetric styles.

Original: We demonstrate a possible implementation 2 of the fused modulation-demodulation in Figure 32.
Revised: We demonstrate a possible implementation 2 of the fused modulation-demodulation in Figure 33.

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: padding
Revised: padding

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: zero padding (ConvT special
Revised: zero padding (ConvT special

Original: [Figure]
Revised: [Figure]

Original: )out = out[:, :, rmpad:-rmpad
Revised: )out = out[:, :, rmpad:-rmpad

Original: [Figure]
Revised: [Figure]

Original: rmpad:-rmpad] out
Revised: rmpad:-rmpad] out

Original: [Figure]
Revised: [Figure]

Original: self
Revised: self

Original: [Figure]
Revised: [Figure]

Original: (out) #
Revised: (out) #

Original: StyleGAN2 Gaussian blurelse:
Revised: StyleGAN2 Gaussian blurelse:

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: _c
Revised: _c

Original: [Figure]
Revised: [Figure]

Original: in_c
Revised: in_c

Original: [Figure]
Revised: [Figure]

Original: k
Revised: k

Original: [Figure]
Revised: [Figure]

Original: k)
Revised: k)

Original: [Figure]
Revised: [Figure]

Original: = F
Revised: = F

Original: [Figure]
Revised: [Figure]

Original: conv2d(
Revised: conv2d(

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: =batch)
Revised: =batch)

Original: [Figure]
Revised: [Figure]

Original: batch
Revised: batch

Original: [Figure]
Revised: [Figure]

Original: channel shape due to grouping_, _, out_h
Revised: channel shape due to grouping_, _, out_h

Original: [Figure]
Revised: [Figure]

Original: out_w = out
Revised: out_w = out

Original: [Figure]
Revised: [Figure]

Original: out
Revised: out

Original: = out.view(batch, out_c, out_h, out_w) return out
Revised: = out.view(batch, out_c, out_h, out_w) return out

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original: The following two forms are equivalent:
Revised: The following two forms are equivalent:

Original: [Figure]
Revised: [Figure]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: M M ORE Q UALITATIVE R ESULTS OF O UTPAINTING VIA I NVERSION
Revised: M M ORE Q UALITATIVE R ESULTS OF O UTPAINTING VIA I NVERSION

Original: N M ORE Q UALITATIVE R ESULTS OF I NBETWEENING VIA I NVERSION
Revised: N M ORE Q UALITATIVE R ESULTS OF I NBETWEENING VIA I NVERSION

Original: O M ORE Q UALITATIVE R ESULTS OF C YCLIC P ANORAMIC I NBETWEENING VIA I NVERSION
Revised: O M ORE Q UALITATIVE R ESULTS OF C YCLIC P ANORAMIC I NBETWEENING VIA I NVERSION

Original: P E XPERIMENTAL D ETAILS OF T HE S PEED B ENCHMARK WITH P ARALLEL B ATCHING
Revised: P E XPERIMENTAL D ETAILS OF T HE S PEED B ENCHMARK WITH P ARALLEL B ATCHING

Original: We perform all the experiments on a workstation with Intel Xeon CPU (E5-2650 2.20GHz) and
Revised: We perform all the experiments on a workstation with Intel Xeon CPU (E5-2650 2.20GHz) and

Original: GTX 2080Ti GPUs.
Revised: GTX 2080Ti GPUs.

Original: We implement our framework with Pytorch 1.6, and execute in an environment with Nvidia driver version 440.44, cuDNN version 4.6.5, and Cuda version 10.2.89.
Revised: We implement our framework with Pytorch 1.6, and execute in an environment with Nvidia driver version 440.44, cuDNN version 4.6.5, and Cuda version 10.2.89.

Original: We report the summation of pure GPU execution time and data scatter-collection time introduced by data parallelism.
Revised: We report the summation of pure GPU execution time and data scatter-collection time introduced by data parallelism.

Original: The model synthesizes a single image for each trial.
Revised: The model synthesizes a single image for each trial.

Original: We ﬁrst warm up the GPUs with 10 proceeding trials without recording their statistics, then compute the mean and variance over 100 trials.
Revised: We ﬁrst warm up the GPUs with 10 proceeding trials without recording their statistics, then compute the mean and variance over 100 trials.

Original: The numbers are reported in Table
Revised: The numbers are reported in Table

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Q A BLATION : I MPORTANCE OF F EATURE U NFOLDING
Revised: Q A BLATION : I MPORTANCE OF F EATURE U NFOLDING

Original: R L IMITATIONS
Revised: R L IMITATIONS

Original: Here we discuss some empirical limitations of InﬁnityGAN.
Revised: Here we discuss some empirical limitations of InﬁnityGAN.

Original: Patch-training leads to performance degradation.
Revised: Patch-training leads to performance degradation.

Original: In order to construct a better conditional distribution in the vertical direction with an auxiliary loss L ar , InﬁnityGAN trains the generator with patches instead of full images.
Revised: In order to construct a better conditional distribution in the vertical direction with an auxiliary loss L ar , InﬁnityGAN trains the generator with patches instead of full images.

Original: However, training with patches reduces the training ﬁeld-of-view, thus leading to an inferior performance at the same ﬁeld-of-view compared to a model trained with full images.
Revised: However, training with patches reduces the training ﬁeld-of-view, thus leading to an inferior performance at the same ﬁeld-of-view compared to a model trained with full images.

Original: Designing unsupervised mechanisms in learning L ar without patch-cropping can improve InﬁnityGAN performance.
Revised: Designing unsupervised mechanisms in learning L ar without patch-cropping can improve InﬁnityGAN performance.

Original: Long-range coherence.
Revised: Long-range coherence.

Original: InﬁnityGAN assumes local coherence following a shared holistic appearance can achieve visually plausible synthesis.
Revised: InﬁnityGAN assumes local coherence following a shared holistic appearance can achieve visually plausible synthesis.

Original: However, certain physical relationships still require long-range dependency, such as we can observe two suns in the top-left image in Figure 22, and twilight should only happen near the horizon in Figure 24.
Revised: However, certain physical relationships still require long-range dependency, such as we can observe two suns in the top-left image in Figure 23, and twilight should only happen near the horizon in Figure 25.

Original: Despite InﬁnityGAN can independently sample pixels arbitrarily distant away, it remains unclear how to construct unsupervised losses for such conditions, as we only have ﬁnite-pixel images in training data.
Revised: Despite InﬁnityGAN can independently sample pixels arbitrarily distant away, it remains unclear how to construct unsupervised losses for such conditions, as we only have ﬁnite-pixel images in training data.

Original: Strip-shaped artifacts.
Revised: Strip-shaped artifacts.

Original: We observe InﬁnityGAN creates a unique type of artifact that forms a stripshaped structure sweeping through the sky or ground for a long distance, such as the clouds in the bottom-two image in Figure 22.
Revised: We observe InﬁnityGAN creates a unique type of artifact that forms a strip-shaped structure sweeping through the sky or ground for a long distance, such as the clouds in the bottom-two image in Figure 23.

Original: We hypothesize the root cause of such an artifact is that the model attends too much to the strong structural characteristics of the horizon line and accidentally shares the representation with other less related contexts.
Revised: We hypothesize the root cause of such an artifact is that the model attends too much to the strong structural characteristics of the horizon line and accidentally shares the representation with other less related contexts.

Original: Improving the modulation or coordinate encoding mechanisms may help suppress such behavior.
Revised: We anticipate improving the modulation or coordinate encoding mechanisms may help suppress such behavior.

Revised: S I MPLEMENTATION I LLUSTRATION OF S CALE I NV FID

Revised: import torch.nn.functional as F 2def eval_scaleinv_fid(real_imgs, fake_imgs, scale): # real_imgs: tensor of real images, shape (B, C, H, W).

Revised: # fake_imgs: tensor of fake images, shape (B, C, H, W).

Revised: # scale: the scale of the current ScaleInv FID.

Revised: T M ORE C OMPARISONS WITH T EXTURE S YNTHESIS M ETHOD

Revised: As we discussed in Section 1 that texture synthesis models are not directly applicable to real-world image synthesis.

Revised: In Figure 38, we demonstrate such a problem by running TileGAN (Fr¨uhst¨uck et al., 2019) on our Flickr-Landscape dataset.

Revised: The results show that the random texture synthesis cannot produce plausible global structures.

Revised: It is important to note that the images shown in TileGAN and other texture synthesis papers (Bergmann et al., 2017; Jetchev et al., 2018) with plausible global structure, such as the satellite map of Jurassic Park and the paintings in TileGAN, are all conditioned on an image that gives the blueprint of the global structure.

Revised: Implementation details of TileGAN experiment.

Revised: We use the ofﬁcially released TileGAN pipeline to synthesize the results with randomly sampled latent variables.

Revised: We follow the instructions and train a PGGAN model at 256 × 256 resolution, then test the model to synthesize at 512 × 512, 1024 × 1024, and 2048 × 2048 pixels.

Revised: Note that we discovered that TileGAN alters the PGGAN architecture from residual-based ToRGB branch 3 to single ToRGB projection 4 .

Revised: Such a modiﬁcation is not described in the TileGAN paper, but can be found by diagnosing the model checkpoints released by the authors.

Revised: The modiﬁcation leads to signiﬁcant visual-quality degradation.

Revised: However, even without the visualquality degradation, the lack of structural clues makes TileGAN impossible to infer a coherent global structure while synthesizing at larger image sizes.

Revised: [Figure]

Revised: U M ORE C OMPARISONS WITH S IN GAN- BASED M ODELS

Revised: We conduct additional experiments on ConSinGAN (Hinz et al., 2021), a concurrent work that proposes several improvements upon SinGAN.

Revised: We use the ofﬁcially released codes and hyperparameters to train the ConSinGAN models.

Revised: ConSinGAN has two types of training modes, generation and retargeting.

Revised: The author mentions the “retargeting” mode is more suitable for extending synthesis size in their GitHub release.

Revised: As shown in Figure 39, similar to SinGAN, ConSinGAN does not have specialized mechanisms to deal with different positional information while tested at a different synthesis size.

Revised: Therefore, neither of the training modes can produce a plausible global view while tested at extended synthesis sizes.

Revised: [Figure]