# High-Precision Image Editing via Dual Attention Control in Diffusion Models Without Fine-Tuning
## 4 Method Overview
### Model Architectures  
Our approach is based on the SD network structure, which consists of several sub-networks, as shown in Figure 2, including a text encoder, U-Net [42, 44], and Variational Autoencoder (VAE) [43]. The cross-attention [45, 46] and self-attention control modules are incorporated within the Transformer modules [47] of the U-Net architecture (explained in detail in Section 4.3). These components combine to form a model that takes text, noise level, and random Gaussian noise as inputs and outputs an image.  
The training process for the model is as follows:  
Figure 2: The model framework diagram of this paper. CAC Module is a cross-attention control module, and SAC Module is a self-attention control module. CAC Module is used throughout the denoising process, and SAC Module is used in layers 1–4 or 5–8, depending on the editing tasks.  
First, a dataset of text-image pairs \(\{\mathbf{X_{0}},\mathbf{p}\}\) is prepared, where \(\mathbf{X_{0}}\) is the original image, and \(\mathbf{p}\) is the textual description associated with image \(\mathbf{X_{0}}\). A noise level \(t\) is set, which represents the number of steps in the diffusion process where noise is added.  
Diffusion phase: The original image \(\mathbf{X}_{0}\) is passed through the encoder part of the VAE, converting it into a latent space representation \(\mathbf{Z}_{0}\). Starting from \(\mathbf{Z}_{0}\), Gaussian noise is incrementally added \(t\) times in the latent space, resulting in a highly noisy latent variable \(\mathbf{Z}_{t}\).  
Denoising phase: The text prompt \(\mathbf{p}\) and the noise level \(t\) are input into the text encoder and the time encoder, which convert them into text embeddings and time embeddings, respectively. At each time step \(t\), the text embedding, time embedding, and \(\mathbf{Z}_{t}\) are fed into the U-Net. The U-Net predicts the noise component in \(\mathbf{Z}_{t}\) and removes it. At this time, the generated result is not ideal, and a slight Gaussian noise will be added to the result to generate \(\mathbf{Z}_{t-1}\). This process is repeated, progressively reducing \(t\) until \(t=0\), at which point \(\mathbf{Z}_{0}\) should approximate the latent representation of the original image.  
In practice, image generation starts from random noise. Using a pre-trained model, noise is gradually removed through the denoising process until a high-quality image is generated. The sampling and training formulas are shown in Algorithm 1.  
```
repeat  Training:  Initialize \(\{\mathbf{x_{0}},\mathbf{p_{0}}\}\), sample \(t\leftarrow\text{Uniform}(\{1,\dots,T\})\), and noise \(\boldsymbol{\epsilon}\sim\mathcal{N}(\mathbf{0},\mathbf{I})\) \(\mathbf{z_{0}}\leftarrow\text{VAE-Encoder}(\mathbf{x_{0}})\)  Compute \(\mathbf{z}_{t}\leftarrow\sqrt{\delta_{t}}\mathbf{z_{0}}+\sqrt{1-\delta_{t}} \boldsymbol{\epsilon}\)  Perform gradient descent step on \(\
abla_{\boldsymbol{\theta}}\|\boldsymbol{\epsilon}-\boldsymbol{\epsilon_{ \theta}}(\mathbf{z}_{t},t)\|^{2}\) until convergence is achieved Sampling:  Sample \(Z_{T}\sim\mathcal{N}(\mathbf{0},\mathbf{I})\) for\(t=T\) down to \(1\)do  Sample noise \(\boldsymbol{\epsilon}\sim\mathcal{N}(\mathbf{0},\mathbf{I})\) if\(t>1\)then  Compute \(\mathbf{z}_{t-1}\) as follows: \[\mathbf{z}_{t-1}=\frac{1}{\sqrt{\delta_{t}}}\bigg{(}\mathbf{z}_{t}-\frac{1- \alpha_{t}}{1-\delta_{t}}\boldsymbol{\epsilon_{\varphi}}(\mathbf{z}_{t},t, \mathbf{y})\bigg{)}+\sigma_{t}\boldsymbol{\epsilon}\] else  Set \(\mathbf{z}\leftarrow\mathbf{0}\) endif endfor \(\mathbf{x_{0}}\leftarrow\text{VAE-Decoder}(\mathbf{z}_{0})\) return reconstructed \(\mathbf{x_{0}}\)
```  
**Algorithm 1** Training and sampling