# Spatially-Precise Video Editing with Reference Imitation: A Region-Aware Cross-Modal Framework
## 3 Materials and Methods
### Local Editing Module
#### 3.4.2 Local Editing Processing  
The source image U-Net first acquires the canonical image and its mask. The source image U-Net is initialized based on the inpainting model of Stable Diffusion-1.5, and its input is a tensor containing 13 channels.  
Figure 3: Specify the editing area and provide a reference image.  
Figure 2: The local editing module structure.The figure illustrates the complete process of feature extraction, feature fusion, and image generation, demonstrating the core mechanism by which the local editing module achieves image content editing through reference image imitation.  
The 4-channel image latent representation: It governs the diffusion process that gradually evolves from initial random noise to the final output latent code. This process is the core link in generating the specific content of the edited area. The 1-channel binary mask: It serves as a clear marker, precisely indicating the specific area in the source image where the content needs to be regenerated. The 4-channel background latent representation: It contains information related to the background of the area in the source image that is not covered by the mask. Such information is crucial for maintaining the coherence and consistency of the background of the image after editing. Together with the 4-channel image latent representation and the 1-channel binary mask, these components work in concert to ensure the proper generation and refinement of the edited content while maintaining the overall integrity of the image.  
In addition, there is the depth map which is converted into a 4-channel depth latent representation after projection processing. Its primary function is to furnish the model with the requisite shape information, assisting the model in better performing reasonable and accurate content-filling operations based on the original shape of the objects in the source image during the editing process. The Correlation Auxiliary U-Net is another vital component within the local editing module. Its core task is to extract the multi-level features of the reference image. It is initialized with the Stable Diffusion-1.5 model and takes the 4-channel latent representation of the reference image as input, delving deep into the visual information within. It not only focuses on the surface information of the image but delves deep into different levels of the image, comprehensively analyzing the reference image from the macroscopic structure to the microscopic details.  
To effectively transfer these extracted reference features to the Source Image U-Net, a special attention mechanism is adopted. In the specific calculation process, the for-mula is as follows:  
\[\text{Attention}=\text{softmax}(\frac{\text{Q}_{i}\cdot\text{cat}(K_{i},K_{r} )^{T}}{\sqrt{d_{k}}})\cdot\text{cat}(V_{i},V_{r}) \tag{10}\]  
During the middle and upper layer stages of the Source Image U-Net, the keys and values of the Correlation Auxiliary U-Net are concatenated with the corresponding parts of the Source Image U-Net. \(K_{i}\) and \(V_{i}\) represent the keys and values of the Correlation Auxiliary U-Net respectively; \(K_{r}\) and \(V_{r}\) represent the keys and values of the Source Image U-Net.  
In this way, the Source Image U-Net can fully leverage the content in the reference image to complete the synthesis of the masked area in the source image. For example, when editing an object in the source image, if the reference image contains specific texture or shape features of a similar object, the Correlation Auxiliary U-Net extracts these features and transmits them to the Source Image U-Net through the attention mechanism. This enables the Source Image U-Net to generate an object part in the masked area of the source image that is similar to that in the reference image while also coordinating with the overall source image. This ensures that the edited image is more visually natural and realistic. As a result, the accuracy and quality of the editing are improved.