# Text-Guided Customizable Image Synthesis and Manipulation
## 3 Our Proposed Method
### Generator Implementation Details  
In the generator, we use conditional augmentation, instance normalization, and a word-level attention mechanism, and their specific implementation details are as follows:  
**Conditioning Augmentation. ** For text processing, we use the conditioning augmentation (\(CA\)) [11] technology to generate more conditional variables to smooth the text representation. The specific CA equation is as follows:  
\[D_{KL}(\mathcal{N}(\mu(\phi_{t}),\ \sum(\phi_{t}))\ ||\ \mathcal{N}(0,\ I)) \tag{7}\]  
Figure 2: The basic structure of synthesizing images based on text and contour is shown in the figure above. Through continuous processing and fusion of text and contour information, the entire structure finally synthesizes the image result that not only conforms to the text’s semantic information but also maintains the contour shape. The image manipulation based on the new text is shown on the far right. The image manipulation result shows that the new text’s semantic information is well reflected in the previously synthesized image. where \(KL\) represents the Kullback-Leibler divergence, \(\mathcal{N}\) denotes the Gaussian distribution, \(\phi_{t}\) represents the encoded text vector, and \(\mu\) and \(\sum\) denote the mean and diagonal covariance matrix operations, respectively. **Instance Normalization. ** Since instance normalization (\(IN\)) [27] is more suitable for image style conversion tasks than batch normalization (\(BN\)) [28], the \(IN\) operation is used in the generator. The relevant calculation of \(IN\) is as follows:  
\[\mu=\frac{1}{HW}\sum_{l=1}^{W}\sum_{m=1}^{H}x_{lm} \tag{8}\]  
\[\sigma^{2}=\frac{1}{HW}\sum_{l=1}^{W}\sum_{m=1}^{H}(x_{lm}-\mu)^{2} \tag{9}\]  
\[y=\frac{x-\mu}{\sqrt{\sigma^{2}+\varepsilon}} \tag{10}\]  
where \(\mu\) is the mean, \(\sigma\) is the covariance, \(x\) and \(y\) represent the input and output of \(IN\), respectively, and \(H\) and \(W\) represent the height and width of the input vector. **Word-Level Attention Mechanism. ** Since the attention mechanism has achieved great success in text-to-image synthesis and has been widely used, it is also employed in our work. We chose the attention mechanism in [29] as the baseline. It utilizes the spatial and channel attention mechanisms to combine word features and each stage's image features more closely to generate higher-quality results. The specific equation of the attention mechanism applied to this work is as follows:  
\[w_{i}=Bi\_LSTM(T),i=1,2,3,\ldots,N \tag{11}\]  
\[r_{i}=column(h_{j}),i=1,2,3,\ldots,N_{c} \tag{12}\]  
where \(Bi\_LSTM\) represents the text encoder used, \(w\_i\) represents the word features, \(N\) represents the number of words, \(h\_j\) represents the image features of the \(j\)th stage, \(r\_i\) represents the \(i\)th image regional features, which corresponds to the column in the \(h\_j\) vector matrix, \(N\_c\) represents the total number of columns in \(h\_j\), \(\forall j\in\{1,2,3\}\), and \(h_{1}\)-\(h_{3}\) represent the first stage's features up to the third stage's features. After the word and image region features are obtained, the similarity probability between them will be calculated:  
\[c_{i,j}=\frac{exp(conv(w_{i})^{T}*r_{j})}{\sum_{k=1}^{N}exp(conv(w_{k})^{T}*r _{j})} \tag{13}\]  
where \(c_{i,j}\) represents the semantic similarity probability between the \(i\)th word features and the \(j\)th regional features, \(conv\) represents a convolution layer, which is used to process word features so that it can perform matrix calculations with image region features, and \"\(*\)\" denotes the Hadamard product.