## 2 Backgrounds
### LDM Structure Overview  
The architecture of LDMs consists of several core components, each playing a unique role in the generative process. Figure 1 provides an overview of the LDM structure. Understanding these components is vital for identifying suitable points for watermark embedding:  
1. **Autoencoder (encoder and decoder):** The encoder is responsible for transforming the high-dimensional input image into a compact latent representation. This step significantly reduces the data's dimensionality, allowing for more efficient processing while retaining essential features of the original content [1]. The latent space created by the encoder is crucial as it serves as the stage for embedding watermarks without affecting pixel-level details [22]. The decoder reconstructs the image from the latent representation generated by the encoder. It is trained to reverse the transformation applied by the encoder, ensuring that the reconstructed image maintains high fidelity to the original input. Watermarking embedded in the latent space can thus be carried through the entire generation process and preserved in the output image [23].
2. **Latent diffusion process:** The diffusion process is the heart of LDMs, involving both a forward and a reverse process: (1) Forward diffusion: In this phase, Gaussian noise is progressively added to the latent representation, simulating a diffusion process that leads to increasingly noisy versions of the latent variables. The goal of this phase is to create a path through which the model learns to handle noisy data effectively [1]. (2) Reverse diffusion (Denoising): The model is trained to reverse the forward diffusion by gradually removing noise from the latent representation. The reverse process is a stepwise denoising operation that transforms the noisy latent into a clear, usable representation, which can then be decoded into an image. The iterative nature of this process provides multiple opportunities to embed watermarks within the noise removal steps, potentially making them more resilient [22].
3. **UNet-based architecture:** The denoising process in LDMs is typically implemented using a U-Net architecture, which is well suited for capturing both local and global features through its use of skip connections [1]. These skip connections allow the network to retain fine details from earlier layers, which is essential for high-quality image reconstruction. For watermarking purposes, the U-Net provides specific layers where watermark information could be injected during the denoising process, ensuring that it becomes an inherent part of the generative flow.
4. **Conditioning mechanisms:** (1) Cross-attention layers: Many LDMs, such as Stable Diffusion, incorporate conditioning inputs (e.g., text prompts) to guide the generative process [1]. This conditioning is facilitated by cross-attention layers that integrate the conditioning information directly into the diffusion process. These layers provide a strategic point where watermarking can be conditioned on specific inputs, adding flexibility to the watermarking strategy [24]. (2) Latent transformation blocks: Theseblocks are responsible for modifying latent variables at different stages of the generation process. They present another potential point for watermark embedding, where watermarks could be introduced in a manner that integrates seamlessly with the latent transformations, making them harder to remove [24].  
By understanding the detailed structure and functions of LDMs, it becomes clear that there are numerous opportunities for embedding watermarks that are robust, efficient, and imperceptible. The following sections will explore specific methodologies for implementing these watermarking strategies and evaluate their effectiveness in practical scenarios.