## 3 Materials and Methods
### Self-Attention DCGAN  
The traditional DCGAN generates high-quality and intricate features by utilizing fixed spatial local information within an image, exhibiting remarkable performance for texture features. Nevertheless, capturing specific geometric features remains challenging [37]. To address this issue, a self-attention mechanism is introduced into the DCGAN framework in this work. During the image generation process, the generator coordinates fine details from each position with distant parts of the image, allowing it to disregard irrelevant information and enhance the significance of key feature details. This combination of the attention mechanism is applied to the middle layer of the generator, enabling it to effectively extract local image features and enhance the generation diversity. The network structure of the self-attention module is illustrated in Figure 5.  
The image features from the previous hidden layer \(x\in\mathbb{R}^{C\times N}\) are first transformed into two feature spaces \(f\) and \(g\) to calculate the attention, where \(f(x)=W_{f}x\), and \(g(x)=W_{g}x\). Here, \(C\) indicates the number of channels, and \(N\) represents the number of feature locations. The transpose of \(f(x_{i})\) is multiplied by \(g(x_{j})\) to obtain the correlation \(S_{ij}\). Then, \(S_{ij}\) are normalized using Softmax to generate the attention feature, which is calculated as follows:  
\[\beta_{j,i}=\frac{\exp{(S_{ij})}}{\sum_{i=1}^{n}\exp{(S_{ij})}},\text{where }S_{ij}=f(x_{i})^{T}g(x_{j}). \tag{2}\]  
Here, \(\beta_{j,i}\) represent the extent to which the model focuses its attention on the \(i\)-th position when synthesizing the \(j\)-th region. It is important to note that the layer's output is denoted as \(o=(o_{1},o_{2},...,o_{j},...,o_{N})\)  
Figure 4: The general framework of our proposed SAITI-DCGAN approach. \[o_{j}=v(\sum_{i=1}^{N}\beta_{j,j}h(x_{i})),\text{where }h(x_{i})=W_{h}x_{i},v(x_{i})=W_{v}x_{i}. \tag{3}\]  
Here, the weight matrices \(W_{g}\in\mathbb{R}^{\overline{C}\times C}\), \(W_{f}\in\mathbb{R}^{\overline{C}\times C}\), \(W_{h}\in\mathbb{R}^{\overline{C}\times C}\), and \(W_{v}\in\mathbb{R}^{\overline{C}\times C}\) implement \(1\times 1\) convolutions. After multiple training iterations on ImageNet, no significant performance degradation was observed when reducing the number of channels in \(\overline{C}\) to \(C/k\), where \((k=1,2,4,8)\). To enhance the memory efficiency, we chose \(k=8\) (i.e., \(\overline{C}=C/8\)) for all experiments. Additionally, the output of the attention layer is scaled by a factor and then added to the input feature map. As a result, the final output is obtained as follows:  
\[y_{i}=\gamma o_{i}+x_{i}. \tag{4}\]  
Here, \(\gamma\) indicates a scalar value that can be learned and starts with an initial value of 0. The purpose of introducing the learnable \(\gamma\) is to prompt the network to initially depend on information from local neighborhoods, which is simpler, and then gradually acquire knowledge from non-local sources to assign greater importance. This strategy is adopted to facilitate the sequential learning of simpler tasks before gradually transitioning to complex tasks. In the proposed approach, an attention module is incorporated into both the generators and discriminators, which are trained alternately using a minimized form of the adversarial loss function [38; 39; 40]. \[L_{D} =-\mathbb{E}_{(x,y)\sim p_{data}}[\min(0,-1+D(x,y))]\] \[\quad-\mathbb{E}_{z\sim p_{z},y\sim p_{data}}[\min(0,-1-D(G(z),y))], \tag{5}\] \[L_{G} =-\mathbb{E}_{z\sim p_{z},y\sim p_{data}}D(G(z),y),\]  
where \(p_{data}\) represents the sample distribution, and \(\mathbb{E}\) denotes the expected distance. \(D(x,y)\) is the discriminator that takes \((x,y)\) as input and outputs a scalar, and \(G(z)\) is the generator in which a sample \(z\) can be drawn from a distribution \(p_{z}\) to the input space.