# Deep Watermarking Based on Swin Transformer for Deep Model Protection
## 3 Materials and Methods
### Embedding Network Based on Swin-Unet  
The improvement in the watermarking framework primarily focuses on embedding. While traditional CNN-based methods like [PERSON]'s [25] employ UNet for watermark embedding, they are limited by the small receptive field of convolutional operations. This restricts the network's ability to capture global context, potentially leading to information loss and reduced visual quality. These limitations motivate the integration of the Transformer, which has proven effective in various image processing tasks [28; 29], into the watermark embedding process. In the early stages of research, during the dimensionality reduction phase of the original UNet network, attempts were made to integrate lightweight attention modules such as ECA [30] and SENet [31]. These modules were chosen because they can enhance the CNN's sensitivity to channel information and improve feature reconstruction without significantly increasing computational overhead. SENet adaptively adjusts channel weights to emphasize important features, while ECA achieves efficient channel interaction. In the context of watermark embedding, these properties were expected to better extract and reinforce local features during the downsampling phase, thereby preserving critical information. However, experimental results revealed that these lightweight attention mechanisms did not significantly enhance the model's performance. This limitation arises because these modules are constrained by the local characteristics of convolutional neural networks (CNNs). The receptive field of convolutional kernels is limited, and the attention mechanisms primarily focus on local features within the image. Although these modules can strengthen feature correlations within local regions, they fall short in capturing global information. Furthermore, CNNs tend to lose global context and long-range dependenciesduring downsampling, and lightweight attention modules are not effective in addressing this issue. As a result, the quality of the generated images did not meet expectations, and, in some cases, was even inferior to that of the original UNet. Based on the aforementioned limitations, the research focus shifted to the transformer architecture, culminating in the adoption of Swin-UNet. This decision was motivated by the fact that the original watermark embedding network employed a UNet-based structure, which is recognized for its ability to maintain both overall architectural integrity and fine-grained details. Swin-UNet retains the inherent strengths of UNet while incorporating Swin Transformer modules to effectively address the challenge of global information capture. A distinguishing feature of the Swin Transformer is its shifted window mechanism. This mechanism partitions the image into local patches and applies self-attention within each window, while simultaneously facilitating inter-window interactions, thereby enabling the capture of long-range dependencies. Through this shifted window approach, Swin-UNet not only preserves local feature details but also integrates comprehensive global context information at each layer, thereby expanding the receptive field. In addition, this layered global information capture mechanism proves particularly advantageous for watermark embedding strategies, as it helps preserve subtle watermark details while maintaining overall image integrity. By balancing local precision with global context, the approach offers a robust solution for ensuring that both the embedded watermark and the overall visual quality are effectively maintained. In summary, the comparison between CNN-based UNet and the Swin Transformer is illustrated in Table 1, which highlights the architectural differences and further clarifies the motivation behind adopting the Transformer-based design in this study. Inspired by these studies, Swin-UNet is believed to be well suited for analyzing image details while ensuring global information capture through attention mechanisms.