# Deep Watermarking Based on Swin Transformer for Deep Model Protection
## 3 Materials and Methods
### Embedding Network Based on Swin-Unet
Additionally, Swin-UNet is designed for image-to-image tasks, where the input and output share structural similarities. This aligns with the objective of watermark embedding, ensuring that the watermarked image remains visually similar to the original. In the embedding network component illustrated in Figure 3, which is built upon the Swin-UNet architecture, the watermark embedding process is executed on the processed image O produced by the image processing model through the following detailed steps: \begin{table}
\begin{tabular}{c c c} \hline \hline
**Aspect** & **CNN** & **Swin Transformer** \\ \hline Receptive Field & Local; gradually expanded via layer stacking & Global modeling via self-attention with shifted windows \\ \hline Attention & Optional and limited in scope & Integrated multi-head self-attention with hierarchical design \\ \hline Feature Interaction & Primarily local due to convolutional operations & Enables long-range dependencies through window shifting \\ \hline Downsampling & May result in irreversible information loss & Preserves contextual information via attention mechanisms \\ \hline Computational Cost & Lower per layer; deep networks needed for context & Higher per layer; optimized by window-based computation \\ \hline \hline \end{tabular}
\end{table} 
Table 1: CNN-based vs. Swin Transformer-based models. Step 1. Input Image Preprocessing  
In this step, two inputs are prepared for feeding into the Swin-UNet embedding network. These inputs include the processed image O generated by the image processing model and the watermark image. To ensure consistency with common practices in image processing tasks and facilitate the comparison with existing models, both images are resized and cropped to a fixed resolution of 256 \(\times\) 256 pixels. This size is widely used in the field for balancing computational efficiency and preserving essential image features. Additionally, both images are normalized to the range [\(-\)1, 1], a standard practice that stabilizes gradient updates and accelerates convergence. This normalization also ensures the compatibility with the discriminator in a later module, contributing to stable training dynamics. Finally, the normalized O and watermark image are concatenated along the channel dimension, forming a six-channel input tensor. Concatenating these along the channel dimension yields a six-channel tensor represented as:  
\[\text{Merged Image}=\text{Concat(O,W)}=(\text{R,G,B,W_{R},W_{G},W_{B})}, \tag{1}\]  
\(O_{N}=(R,G,B)\) denotes the normalized processed image and \(W_{N}=(W_{R},W_{G},W_{B})\) denotes the normalized watermark image. The preprocessed tensor, the merged image with a size of 256 \(\times\) 256 \(\times\) 6 (width \(\times\) height \(\times\) channel), is then passed to the Swin-UNet for watermark embedding. Step 2. Patch Partition and Encoder Phase  
Figure 3: The embedding network pipeline of the proposed method. First, the merged image undergoes a patch partition operation, dividing it into non-overlapping patches of size 4 \(\times\) 4 \(\times\) 6, as shown in Figure 4. This process reduces computational complexity and prepares the data for Transformer architectures, which are designed to process sequential inputs. Each patch is then passed through the linear embedding layer, where it is flattened into a feature vector and meaningful features are extracted. This operation reshapes the image into \(\frac{W\times H}{16}\times 96\), achieving dimensionality compression while converting spatial information into a serialized representation suitable for Transformer processing. The subsequent encoder and decoder operations completely adhere to the original Swin-UNet structure, with no modifications made in this approach. The embedded feature sequence is then fed into the Swin Transformer module for feature extraction and attention computation.