# Deep Watermarking Based on Swin Transformer for Deep Model Protection
## 3 Materials and Methods
### Embedding Network Based on Swin-Unet
The skip connection features from the encoder are then element-wise added to the upsampled output features to achieve cross-layer feature fusion. The fused features are fed into the Swin Transformer module, whose core task is to effectively integrate watermark features with host image features. Through the shifted window attention mechanism, the Swin Transformer computes attention weights within local windows, thereby precisely controlling the embedding strength and spatial distribution of watermark features: enhancing the watermark feature weights in texture-rich regions to improve invisibility, while reducing the weights in smooth regions to preserve visual quality. Furthermore, the Swin Transformer establishes global dependencies through cross-window connections, ensuring that the watermark embedding strength is coordinated across different regions, thereby maintaining the continuity and integrity of the watermark pattern. The Swin Transformer also plays a crucial role in reconstructing image details, preserving the structural information and texture features of the host image while embedding the watermark, thereby balancing watermark invisibility and visual quality. These operations are performed across decoding layers, progressively reconstructing the feature maps. Through three feature upscaling module, the feature dimensions are transformed from \(\frac{W}{32}\times\frac{H}{32}\times 768\) to \(\frac{W}{4}\times\frac{H}{4}\times 96\). Finally, interpolation techniques are applied to restore the spatial dimensions to \(W\times H\times 96\), ensuring that the output image maintains the same resolution as the input while preserving rich feature information. Step 4. Output Processing  
After a series of operations performed by the decoder, the output image is adjusted to a size of \(W\times H\times 96\). To convert this output into a standard image, a linear projection is applied to reduce the number of channels from 96 to 3, representing the RGB channels of the image. Subsequently, the pixel values are mapped to the range of \([-1,\,1]\). This step ensures that the network operates within a consistent range, which is important for maintaining the stability of the network during training. Finally, a reverse normalization operation is applied to remap the pixel values back to the [0, 255] range, completing the watermark embedding process. In the design of the watermark embedding network, the primary objective is to ensure that the processed image and the watermarked image remain as similar as possible, thereby preserving the visual quality to the greatest extent. To achieve this, we refer to the approach proposed by [PERSON] et al. [25] and adopt a combination of commonly used image loss functions, primarily the mean squared error (\(Loss_{MSE}\)), visual geometry group (\(Loss_{VGG}\)), and generative adversarial network (\(Loss_{GAN}\)). The overall loss function is formulated as follows:  
\[Loss_{Embedded}=\lambda_{1}\cdot Loss_{MSE}+\lambda_{2}\cdot Loss_{VGG}+ \lambda_{3}\cdot Loss_{GAN}, \tag{4}\]  
where \(\lambda_{1}\), \(\lambda_{2}\), \(\lambda_{3}\) represent the weights of each loss term, and these three losses constrain visual quality from different perspectives. The MSE loss measures pixel-wise differences between the processed and watermarked images, ensuring that the embedded watermark does not significantly distort the original image's structure and details. The VGG loss compensates for the limitations of MSE loss by focusing on high-level image features, effectively preserving texture details and better aligning with human perceptual judgments of visual quality. The GAN loss incorporates an adversarial learning mechanism, where a discriminator detects the presence of a watermark in the image, encouraging the embedded watermark to be more imperceptible. By jointly considering pixel-level similarity (MSE), perceptual quality (VGG), and realism (GAN), our loss design effectively balances visual quality, detail preservation, and imperceptibility in watermark embedding.