## 3 Preliminary  
Figure 1a depicts the architecture of the Latent Diffusion Model (LDM) [8], which forms the foundation of the SD model. Initially, the input image \(x\) is transformed into a latent representation \(\mathcal{Z}\) via an encoder \(\varepsilon\). In the latent space, the diffusion process starts from the  
initial noise \(\mathcal{Z}_{\mathcal{T}}\) and progressively denoises it using a Denoising U-Net \(e_{\theta}\), resulting in the latent representation \(\mathcal{Z}_{0}\). The Denoising U-Net incorporates conditional information (such as semantic maps, text descriptions, representations, and images) during the denoising process, integrating external conditions at each step through a cross-attention mechanism. Finally, the latent representation \(\mathcal{Z}_{0}\) is converted back to pixel space by a decoder \(\mathcal{D}\), yielding the reconstructed image \(\widetilde{\mathcal{X}}\). This approach allows for efficient denoising and generation in the latent space, while leveraging conditional information to control the generation outcomes. The SD model can generate high-quality images based on given text descriptions. Training foundational models requires substantial computational resources, which small research teams cannot independently manage. Researchers have developed cost-effective fine-tuning methods [9; 21; 22], making it possible to deploy SD models on consumer-grade GPUs. From the perspectives of generation quality, time efficiency, and the size of the required training dataset, LoRA is a superior choice [23].  
LoRA (Low-Rank Adaptation) is a technique that reduces the complexity of large models by fitting their high-dimensional structures with low-dimensional ones. Instead of fine-tuning the entire model, LoRA focuses on training the \"residual\" part of the model (\(\Delta W\)). The fine-tuned weights (\(W^{\prime}\)) are given by:  
\[W^{\prime}=W+\Delta W\]  
The technical principle of LoRA is illustrated in Figure 1b. The input \(x\) has a dimension of \(d\), with the left side representing the frozen pretrained weights of the original model (\(W\)) and the right side representing the newly added branch under the LoRA strategy (\(\Delta W\)). First, through linear layer \(A\), the data dimension is reduced from \(d\) to \(r\), and after computation, it is restored from \(r\) to \(d\) through linear layer \(B\). By fine-tuning the smaller matrices \(A\) and \(B\) instead of \(W\), the hyperparameters that need to be fine-tuned are significantly reduced [9].  
As a powerful generative AI model, SD can be applied to various generation tasks, with its main input control methods including Text-to-Image [8], Image-to-Image [8], and ControlNet [24]:  
* Text-to-Image: Generates an image based on the input text.
* Image-to-Image: Adds a reference image as a condition to the Text-to-Image task.  
Figure 1: (**a**) the basic architecture of the SD model: Latent Diffusion Model [8]; (**b**) LoRA [9].  
* ControlNet: Adds structured condition signals (such as control_canny, control_depth, control_hed, control_mlsd, etc.) on top of the aforementioned tasks to finely control image content and layout [24]. The relationship among these three control conditions is as follows: a reference image can be added to the Text-to-Image task to transform it into an Image-to-Image task, and further adding ControlNet allows for more precise control of the image structure. Of course, ControlNet can also be directly added to the Text-to-Image task to perform image generation [25; 26; 27; 28; 29].