# Unlocking Efficiency in Fine-Grained Compositional Image Synthesis: A Single-Generator Approach
## 3 Methods
To offer a comprehensive understanding of the information processing flow in SSSGAN, we provide brief descriptions of the modules inherited from SemanticStyleGAN as needed. For more detailed information, please refer to the original paper [12]. **Noise Code Generation**. As SSSGAN uses a single local generator for the generation of all parts, we need to explicitly model their joint distribution \(P(\mathbf{q}_{1},\mathbf{q}_{2},\cdots,\mathbf{q}_{K})\), where \(\mathbf{q}_{k}\), \(k=1,\cdots,K\) is a random variable representing the \(k\)th part. Instead of directly modeling this joint distribution, inspired by the auto-regressive model [65], we factorize this joint distribution into the product of a sequence of conditional distributions, as defined below:  
\[P(\mathbf{z}_{1},\mathbf{z}_{2},\cdots,\mathbf{z}_{K})=\prod_{k=1}^{K}P( \mathbf{z}_{k}|\mathbf{z}_{1},\cdots,\mathbf{z}_{k-1}). \tag{1}\]  
We start the generation of the noise codes from a learnable constant vector \(V_{0}\), which is mapped to a tuple \((\mu_{0},\log var_{0})\) by a two-layer MLP; here, \(\mu_{0}\) and \(var_{0}\) represent the mean and variance of the Gaussian distribution, respectively. The logarithm operation is performed to ensure that the variance is a positive value. The initial input to the LSTM layer is sampled from the diagonal Gaussian \(\mathcal{N}(\mu_{0},\delta_{0})\) using the re-parameterization trick:  
\[z_{0}=\epsilon_{0}\odot\delta_{0}+\mu_{0}, \tag{2}\]  
where \(\odot\) denotes element-wise product, \(\epsilon_{0}\sim\mathcal{N}(\mathbf{0},\mathbf{I})\) and \(\delta_{0}=\sqrt{var_{0}}\). \(\mathcal{N}(\mathbf{0},\mathbf{I})\) represent the multivariate Gaussian distribution with zero mean \(\mathbf{0}\) and identity covariance matrix \(\mathbf{I}\). At step \(t\), the LSTM layer takes two vectors as input: one is the noise code \(z_{t-1}\) sampled at step \(t-1\) and the other is the hidden state of the LSTM layer \(h_{t-1}\), which aggregates the interaction of the parts up to step \(t-1\). The output layer of the LSTM is a single linear layer that maps the hidden state \(h_{t}\) to a tuple \((\mu_{t},\log var_{t})\). Similarly, the noise code \(z_{t}\) is  
Figure 1: Overview of the SSSGAN architecture: The LSTM module models noise code relationships, the shared generator \(g\) generates part features, the _Fusion_ module fuses part features, the render net \(R\) renders features to images and the discriminator \(D\) provides adversarial loss. sampled from the diagonal Gaussian \(\mathcal{N}(\mu_{t},\delta_{t})\) using the re-parameterization trick, defined as follows:  
\[z_{t}=\epsilon_{t}\odot\delta_{t}+\mu_{t}, \tag{3}\]  
where \(\epsilon_{t}\sim\mathcal{N}(\mathbf{0},\mathbf{I})\) and \(\delta_{t}=\sqrt{\sigma\pi t}\). In our experiment, we set the maximum step \(T\) to the part count \(K\), which means that we generate one part at each step. **Part Feature Generation and Composition. ** The noise codes \(\{z_{k}\}_{k=1}^{K}\) are mapped to a set of latent codes \(\{w_{k}\}_{k=1}^{K}\) by a group of equal linear layers [66]. As mentioned above, we do not factorize the latent code as in SemanticStyleGAN and use one latent code to modulate the generator; that is, SemanticStyleGAN uses a separate local generator for the generation of each part, while we use a shared local generator for the generation of all the parts. The architecture of the local generator is shown in Figure 2. The local generator is a stack of modulated convolutional layers [66] and the \(toDepth\) and \(toFeat\) modules are mainly single modulated convolutional layers with fixed style. The local generator \(g\) takes the Fourier feature \(p\) and the \(k\)th latent code \(w_{k}\) as input, then produces a pseudo depth map \(d_{k}\in\mathbb{R}^{H_{f}\times W_{f}\times 1}\) and a feature map \(f_{k}\in\mathbb{R}^{H_{f}\times W_{f}\times D_{f}}\), where \(H_{f}\), \(W_{f}\) and \(D_{f}\) are the spatial height, spatial width and channel dimension, respectively.