CodeSwap: Symmetrically Face Swapping Based on Prior Codebook

Authors: xiangyang  luo
xin  zhang
yifan  xie
xinyi  tong
weijiang  yu
heng  chang
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-06 14:13:44

Abstract: Face swapping, the technique of transferring the identity from one face to another, merges as a field with significant practical applications. However, previous swapping methods often result in visible artifacts. To address this issue, in our paper, we propose $CodeSwap$, a symmetrical framework to achieve face swapping with high-fidelity and realism. Specifically, our method firstly utilizes a codebook that captures the knowledge of high quality facial features. Building on this foundation,  the face swapping is then converted into the code manipulation task in a code space. To achieve this, we design a Transformer-based architecture to update each code independently, which enable more precise manipulations. Furthermore, we incorporate a mask generator to achieve seamless blending of the generated face with the background of target image.
A distinctive characteristic of our method is its symmetrical approach to processing both target and source images, simultaneously extracting information from each to improve the quality of face swapping. 
This symmetry also simplifies the bidirectional exchange of faces in a singular operation.
Through extensive experiments on ClelebA-HQ and FF++, our method is proven to not only achieve efficient identity transfer but also substantially reduce the visible artifacts.

Main paper: 
 introduction: 
 With the rise of video media and the continuous development of image editing methods, face swapping techniques receive a lot of attention from both academic community and industry. With widely applications in various fields such as digital avatars [31] , the movie industry, privacy preservation [6] , face swapping aims to seamlessly transfer the identity of a source face to a target face while preserving the attributes of the target face, including expression, pose, and background.Initially, face swapping methodologies were elementary, relying on simple cropping and replacement of inner facial pixels [3, 7] . These methods are often sensitive to variations in lighting and angles, resulting in distinguishing forgeries easily. The rise of Generative Adversarial Networks (GANs) significantly revolutionize this field. A variety of GAN-based approaches have been introduced [5, 11, 22, 46] , proficient in transferring identity features of the source while retaining the attributes of the target face, but also struggle in generating high fidelity result. Additionally, there are also methods [23, 47, 52] utilizing pre-trained StyleGAN [19] to achieve a higher definition face swap results. And more recently, diffusion based methods for face swapping like those in [20, 50] have emerged. However, these methods still encounter challenges associated with visible artifacts.To tackle this problem, we propose a novel framework called CodeSwap, which manipulating codes symmetrically in a code space to achieve face swap. Specifically, our method leverage a discrete codebook rich in high quality facial features, which can effectively eliminate visible artifacts and improve the generation quality. After encoding the images into the code space, the swapping task is transferred to meticulously identifying and substituting specific segments of the target face from the codebook to achieve face swapping. To achieve this, we design a global fusion network based on Transformers [39] to capture global information from both images and manipulate the codes within the code space. It is worth noting that our approach does not require an additional face recognition model to extract the identity features of the source image for explicit control. Instead, we design a Unified Self-Attention mechanism, which treat source and target images equally, enabling a capture of more information and mutual face swapping in a single step. Different from [24] , which employs an asymmetric training method to stabilize the training process, our network is structurally symmetric and can not be decoupled.Furthermore, we propose a mask generator module that utilize middle-layer semantic features of our encoder and decoder to generate masks and successfully ensures high quality image generation. Extensive experiments demonstrate that our method can generate swapping results with high-fidelity and realism. In summery, our contributions are as follows:• We design a novel symmetrical face swapping structure by simultaneously and equally utilizing the information from two input facial images. • We propose an innovative approach that conceptualizes face swapping as a process of code manipulation in a code space. By leveraging priors inherent to high quality faces, our method greatly reduces the occurrence of visible artifacts. • Through extensive experiments, we demonstrate that our approach outperform the previous methods, especially in the term of high-fidelity and realism. 
related work 2.1 face swapping: 
 Research in face swapping attracts significant interest because of its potential practical applications. Early efforts in addressing this challenge primarily utilized classical image processing techniques and three-dimensional morphable models (3DMMs) [3, 29] , which frequently resulted in face swaps that looked obviously artificial. However, the emergence of generative networks marked a significant step forward. FSGAN [29] , an early pioneer, utilizes GANs for face reenactment and blending back to target image, but struggle with preserving the target's authentic attributes. This problem facilitates the development of AdaIN-based [15] methods [5, 11, 22] , which focus on identity transfer and attribute preservation. Sim-Swap [5] , for instance, adopts a pretrained face recognition model to extract the identity features of the source image and fuses them with the target features in the latent space, while FaceShifter [22] emphasizes attribute retention by incorporating target features during upsampling. InfoSwap [11] introduces mutual information to disentangle non-identity information, aiming for extracting the most valuable information from the identity representation. Blend-Face [35] proposes a new face encoder to mitigate attribute leakage.To further enhance image quality, several studies [25, 44, 45, 52] have explored StyleGAN's [19] potential for refined face swapping.MegaFS [52] creates face swaps by substituting the target images' high-level semantic features with those from the source images. FSLSD [45] conveys attributes across multiple levels by utilizing the side-outputs from StyleGAN. E4S [25] adopts regional GAN inversion to separating facial shape and texture. Despite such advancements, the quest for artifact-free results continues. Recent diffusion-based methods [20, 50] introduce innovative approaches.For example, DiffFace [20] utilizes addtional facial guidance driven by pretrained models to guide the denoising process, and DiffSwap employs a 3DMM to blend faces and extract landmarks for explicit facial shape control. Although these diffusion-based methods adopt more complex training procedures and extended inference durations, the problem of visible artifacts remain. However, our method suppresses visible artifacts to a large extent, resulting in a more natural-looking face swap.Traditional methods that adopt sparse representation through the use of learned dictionaries have significantly advanced image processing tasks, particularly in the areas of restoration [12, 36, 37, 48] and denoising [9] . Building on these works, innovations like Vector Quantized Variational AutoEncoders (VQVAE) [32, 38] and Vector Quantized Generative Adversarial Networks (VQGAN) [10] take a further step by introducing the idea of compact codebooks. These codebooks efficiently compress image data into a set of discrete vectors, resulting in a compact and rich image representation. This development has significantly raised the bar for image quality and computational efficiency, offering improvements over traditional sparse representation methods by facilitating more detailed image reconstructions with less computational overhead. Such progress finds extensive application across various domains, including face restoration [13, 42, 51] , motion generation [27, 43] , gesture synthesis [2] , and so on. Drawing inspiration from the concept of codebook learning, this work utilizes a prior codebook of faces to achieve face swapping. By manipulating codes with high quality facial priors, we can achieve more natural face-swapping results. 
method: 
 The overarching structure of our face swapping framework is illustrated in Figure 2 . This section intoduces our innovative approach to face swapping. We begin by describing the training of prior codebook in Section 3.1, which serves as the foundation of our framework. Leveraging this facial prior, we create a globally aware and symmetric code swapping framework. This framework effectively update the original facial code with entries from the codebook in the code space, facilitating precise face swapping. For further details, please refer to Section 3.2. To preserve the complex background details, we introduce a mask generator in Section 3.3. This module utilizes middle layer features of encoder and decoder to generate masks that seamlessly integrate with our face swapping architecture.The face prior guarantees the generation of highly natural faces with diversity. To capture the facial prior knowledge, we leverage the task of face reconstruction through large number of natural faces as depicted in Figure 3 , utilizing the VQGAN [10] architecture. The facial image 𝐼 𝑓 ∈ R 𝐻 ×𝑊 ×3 undergoes a transformation into a code space via an encoder E, yielding 𝑍 𝑓 ∈ R 𝑚×𝑛×𝑑 . Subsequently, within this code space, we substitute each item by identifying the nearest term from a learnable codebook 𝑄 = {𝑞 𝑘 ∈ R 𝑑 } 𝑁 𝑘=1 , thereby acquiring the quantized facial features 𝑍 𝑞 ∈ R 𝑚×𝑛×𝑑 :EQUATIONFinally, 𝑍 𝑞 is sent to decoder D to obtain the restructed face image Î𝑓 . As shown in the lower path in Figure 3 , given a low quality face image, the prior codebook learned from natural faces can reduce the visible artifacts and produce a high quality recovered one.Losses. Similar to [10, 38] , we adopt gradient replication to address the issue of gradient truncation that arises during the quantization process. This strategy allows us to train the encoder, decoder and prior codebook in an end-to-end manner utilizing the following loss function:EQUATIONwhere the first three of terms aims to enhance the quality of the reconstruction. Here, 𝐿 1 represents the L1 loss, 𝐿 𝑝𝑒𝑟 indicates perceptual loss [17] , and 𝐿 𝑎𝑑𝑣 denotes adversarial loss with 𝜆 serving as a adaptive weight. These components are expressed as follows:EQUATIONThe last term, 𝐿 𝑐𝑜𝑑𝑒 , is meticulously crafted to minimize the discrepancy between the code 𝑄 and the features 𝑍 𝑓 , defined as:EQUATIONwhere 𝑠𝑔[•] stands for gradient-stop operation and 𝛽 = 0.25 is a weighting factor.After training the a prior codebook, the next step is to manipulate the code of the image to achieve face swapping. As depicted in Figure 2 , face images 𝐼 𝑎 and 𝐼 𝑏 are fed into a shared encoder to the code space and then flattened to derive the feature embeddings 𝑍 𝑎 , 𝑍 𝑏 ∈ R 𝑚𝑛×𝑑 . These embeddings are then concatenated to form 𝑍 :EQUATION)Since the encoder is trained on the reconstruction task in Stage 1, the concatenated feature 𝑍 contains all the information of the two images, which we called global features. The global features 𝑍 will be fed into a global fusion module and a code selector to choose the best codes from prior codebook to achieve face swap. Global Fusion Module is consist of several unified self-attention mechanism, which can fuse the features of two images with a global attention and output new embeddings symmetrically. The fusion mechanism is shown in Figure 4 (b) and expressed as follows: where 𝑍 𝑙 denotes the output of the 𝑙 𝑡ℎ layer, 𝑄 𝑙 , 𝐾 𝑙 , 𝑉 𝑙 are obtained from 𝑍 𝑙 through linear transformations fromEQUATIONCode Space Soft Quantization Prior Codebook𝑊 𝑄 ∈ R 𝑑 ×𝑑 𝑘 , 𝑊 𝐾 ∈ R 𝑑 ×𝑑 𝑘 , 𝑊 𝑉 ∈ R 𝑑 ×𝑑 𝑣 , respectively.And 𝐴 is the calculated attention map of all codes from both images. After the self-attention mechanism, a linear layer is utilized to project the dimension into 2𝑚𝑛×𝑑, and a Feed-Forward Network (FFN) to generate the updated features 𝑍 𝑙+1 . As shown in Figure 4 , compare to traditional AdaIN-based methods, our global fusion module has several advantages. i) Previous AdaIN-based methods [5, 11, 22] use additional pretrained face recognition models to extract identity features from the source image. While some methods employ more powerful encoders to gather extensive information [49, 52] , they may overlook the potential contributions of the target image. In contrast, our model uses an encoder from a reconstruction task to extract global features from both images and applies unified self-attention to leverage information from both, thus enhancing the swap process.ii) Compared to AdaIN's impact on the overall style of the image, our approach manipulates each code individually based on the calculated attention map, which ensures more precise updating of features.iii) Our symmetric process manipulates the features of both images at the same time, thus enabling swapping two faces at once, extending a functionality absent in previous methods.Code Selector. After the global fusion module, we adopt a code selector to select the suitable items from the prior codebook to update the original one, which is a MLP module and can be expressed as:EQUATIONwhere 𝑍 𝑙𝑎𝑠𝑡 ∈ R 2𝑚𝑛×𝑑 is the output of global fusion module and 𝑁 denotes the number of items in codebook. Instead of employing 𝑎𝑟𝑔𝑚𝑎𝑥 function for 𝑃 to get the code with the highest probability, we adopt soft quantization to refer to all the codes in the codebook, which is expressed as:EQUATIONThe obtained 𝑍 𝑞 selectively refers to the individual codes and contribute to the generation of more natural details, which is then passed to the decoder D to generates high-fidelity swapped face images.Losses. To train our symmetrical swap module effectively, we apply following losses:EQUATIONwhere the weights 𝜆 𝑖𝑑 , 𝜆 𝑝𝑒𝑟 , 𝜆 𝑒𝑥𝑝 are set to 1.5, 2 × 10 -3 , and 0.5, respectively. The perceptual loss, 𝐿 𝑝𝑒𝑟 , is same to Equation (3). The identity loss, 𝐿 𝑖𝑑 , ensures the transfer of identity features between face images and is expressed as: [5, 11, 22, 35] fuse the whole features through AdaIN. (b) Our self-attention based approach has two advances: 1) we are able to accept more feature information from both images, 2) we update each code individually based on calculated attention map for more precise control.EQUATION)where 𝐸 𝑖𝑑 represents a pre-trained face recognition model [8] and 𝐶𝑜𝑠𝑆𝑖𝑚(•, •) denotes the cosine similarity between two vectors. The reconstruction loss, 𝐿 𝑟𝑒𝑐𝑜𝑛 , is applied to ensure fidelity when the input faces 𝐼 𝑎 and 𝐼 𝑏 in the training sample are identical. It is formulated as:EQUATIONWhat's more, 𝐿 𝑒𝑥𝑝 aims control ensure the resulting expression is consistent with the face being swapped [26] , expressed as:EQUATION) Our symmetrical architecture also effectively utilizes code cycle loss 𝐿 𝑐𝑦𝑐 , which enhances the stability of training. Due to the frozen encoder and decoder in this stage, we do not need to rerun the full network, instead we only focus on achieveing this loss in the code space. 𝐿 𝑐𝑦𝑐 can be formulated as:EQUATIONwhere embeddings 𝑍 𝑏𝑎 𝑞 and 𝑍 𝑎𝑏 𝑞 are the result codes in the first swap, and they are processed again through the modules in Stage 2 to derive 𝑍 𝑎𝑏𝑎 𝑞 , 𝑍 𝑏𝑎𝑏 𝑞 . Comparing 𝑍 𝑎𝑏𝑎 𝑞 , 𝑍 𝑏𝑎𝑏 𝑞 against original embeddings 𝑍 𝑎 , 𝑍 𝑏 for reconstruction loss, we can stable the training process to produce high-fidelity results. This comprehensive loss 𝐿 𝑠𝑤𝑎𝑝 ensures the successful transfer of identity features, while maintaining the fidelity and expression consistency of the swapped faces.In order to keep all the background information of the target image, we design a mask generator that can distinguish between identityspecific and unrelated regions. Inspired by previous methods [1, 44] , our approach utilize the middle layer features from both encoder and decoder of our framework to produce detailed masks. This involves concatenating features from encoder and decoder, refining through 1×1 convolutions, activations, and upsampling to generate the final mask, which is detailed in the appendix. The blending operation yields the final images as follows:EQUATIONwhere ⊙ denotes Hadamard product.For optimal mask generator training, we employ a dual-strategy approach: initially adopting a supervised method by pretrained parsing model for coarse mask generation, followed by an unsupervised technique to finetune the mask genertor for fine-grained mask creation. During the supervised phase, We utilize ground truth obtained by a pretrained parsing model from input images to guide mask production, with the training loss formulated as:EQUATIONSubsequently, we integrate the mask generator into the face swapping framework in Stage 2, leveraging losses in Stage 2 for unsupervised training. This enables the mask generator to automatically distinguish between areas that are relevant and irrelevant to the identity feature. 
experiment 4.1 datasets: 
 We train our models using two high-definition facial datasets: FFHQ [19] , which comprises 70,000 facial images, and VGGFace2-HQ [30] , which is a high resolution version of VGGFace2 [4] and contains millions of images with thousands of identities. Both datasets are widely recognized and utilized for training generative models due to their diverse and high quality image content. To train our model, we resize the images into 512 × 512. To better validate the effect of high resolution face swapping, we process the CelebA-HQ [18] and FaceForensics++ (FF++) [33] as our test set, which represent high quality images and low quality images, respectively.Our model processes 512 × 512 images within a 16 × 16 × 512 latent space and utilizes a codebook of 1,024 codes, following the autoencoder structure from [10] . Our model features a 6-layer Transformer and is trained across three stages: 1,000K iterations for Stage 1, 600K for Stage 2, and 10K for Stage 3, using a global batch size of 32 on two NVIDIA Tesla A100 GPUs. We employ the Adam optimizer [21] with a 10 -4 learning rate.To demonstrate the effectiveness of our method, we conduct quantitative experiments, qualitative experiments and user study, comparing our method with FSGAN [28] , Simswap [5] , InfoSwap [11] , MegaFS [52] , DiffSwap [50] , BlendFace [35] , and E4S [25] as baselines.In quantitative study, we utilize two distinct test datasets: CelebA-HQ [18] and FF++ [33] . For the CelebA-HQ dataset, we select a subset of 1,000 images featuring unique identities and craft 1,000 testing pairs through random selection. In the case of FF++, we extract 10 random frames from each video, resulting in a dataset of 10,000 images. These images are then combined randomly to generate an additional 10,000 images for testing. We benchmark our method against prior works, evaluating performance across several metrics including ID retrieval, ID similarity, pose and expression accuracy, and the Frechet Inception Distance (FID) [14] . ID retrieval is performed using a specialized face recognition model [41] to extract identity features, employing cosine similarity for matching. For CelebA-HQ, we match images within the selected subset, while for FF++, to heighten the challenge, we adopt the approach of [23] by matching against a single frame from each video instead of all frames. Pose accuracy is assessed through the Euclidean distance between estimated and ground truth poses [34] . Meanwhile, expression accuracy is evaluated by comparing the L2 distance between expression embeddings from the swapped and target faces using a model distinct from the one used during the training phase [40] .As shown in Table 1 , our method obtains the highest ID Retrieval and ID Similarity metrics on both datasets, indicating that our method works well to transfer the facial identity from source to target. The expression and pose accuracy metrics show that we are also able to preserve the attribute of the target face. In addition to this, our method achieves the lowest FID on the CelebA-HQ dataset , indicating the capability to generate high-fidelity images of the proposed methods. Our FID scores are slightly higher on the FF++ dataset, which is likely due to visible artifact reduction in our generated images compared to the original FF++ images. This leads to differences in data distribution and the increase of FID.Comparisons. We perform qualitative comparisons on CelebA-HQ [18] and FFHQ [19] . As shown in Figure 5 , existing methods often have visible artifacts, especially at the corners of the mouth, nose, and forehead. But our method greatly suppresses visible artifacts, producing a more natural image with a higher clarity. It is noteworthy to mention the observation made in the last line of Figure 5 . When there is a significant difference in the mouth shapes between the source and the target, existing approaches either failing to alter the mouth shape properly or resulting in obvious visible artifacts. In contrast, our method correctly changes the shape of the mouth, renders the skin in the constricted area of the mouth, and blends the skin color of the target face with the texture of the source face for a natural look.To explore human perception of face swapping results, we perform a user study encompassing three distinct evaluations with ratings on a scale from 1 to 5: 1) identity similarity to the source image, 2) consistency of expression, lighting, and pose with the target image, and 3) the degree of realism. As illustrated in Table 2 , our method outperforms existing methods in all metrics, especially in realism. These findings suggest that our method effectively preserves the attributes of the target image and ensures a high-fidelity generation of realistic images, all while successfully transfer the identity from the perspective of human. In this section, we prove the validity of some of the modules in our CodeSwap.4.4.1 Effectiveness of Prior Knowledge via the Codebook. To assess the critical function of the codebook in embedding prior knowledge, we skip the code selector and prior codebook, allowing the Transformer's feature outputs to directly proceed to the decoder and retrain the model. The qualitative and quantitative results are shown as w/o 𝐶 in Figure 6 and Table 3 , respectively. The qualitative results, illustrated in Figure 6 , demonstrate that the lack of codebook results in more artifacts in the generated image and a blurrier face. The FID metrics in Table 3 also demonstrates the bad quality of generated images. These results prove the important role of the prior codebook in generating high-fidelity images.4.4.2 Significance of Symmetrical Swap. In our approach, we employ symmetrical swap, which utilize self-attention on concatenated features to obtain the global information of both images and achieve face swapping with each other at once. To prove the significance of symmetrical swap, we adopt the cross-attention mechanism to fuse the features from the source face, thus breaking the symmetrical process. The qualitative and quantitative results are shown as w/o 𝑆 in Figure 6 and Table 3 , respectively. Qualitative results show that the generated images are not precise enough in expression accuracy especially the shape of the mouth. Quantitative experiments also shows the degradation of its generation quality. Both results demonstrate the validity of our symmetrical framework.Quantization. In our approach, we utilize the output probabilities from the code selector to achieve soft quantization through a weighted averaging process. To evaluate the efficacy of this technique, we contrasted two quantization methods: 1) hard quantization which select the code with the highest probability instead of our weighted average, and 2) Gumbel-Softmax trick [16] , which correspond to w/o 𝑆 1 and w/o 𝑆 2 both in Figure 6 and Table 3 , respectively. The quantitative results of Table 3 show that both methods have varying degrees of degradation in their effectiveness especially the gumbel-softmax trick. The qualitative results demonstrate that the use of soft quantization is able to synthesize the information from the individual codes, leading to more natural details.4.4.4 Impact of the Mask Generator. Our CodeSwap focuses on facial priors to generate high quality faces. In addition, we design a mask generator to preserve the background information of the target image. To validate the impact of our mask generator, we conduct the experiment under the setting without mask generator (w/o 𝑀) . The result is shown in Figure 7 , which demonstrates that our generated mask can distinguish the identity related area and help to achieve better background preservation. In this section, we extend our application to regional control face swap and analyze some of the additional results.4.5.1 Regional Control Swap. Although our method manipulates on all codes of the input image, we demonstrate the ability of regional control swap of our method. This can be achieved through selective manipulation of codes in the code space. As shown in Figure 8 , our method allows for the high quality regional face swap, which is not possible with purely pixel-level operations. This characteristic ensures that even if part of the codes are manipulated, the resulting image blends perfectly with other features of the face, achieving highly natural results. To gain a deeper insight into the feature fusion process of our global fusion module, we visualize the attention maps generated during self-attention, as depicted in Figure 9 . Since we perform the self-attention mechanism after concatenating the features of the two images, each code obtains two attention maps from the image itself and the other input image. The visualization results show that the global fusion module not only successfully focuses on correct region of the other image but also on other regions of its own image. This proves that our symmetrical framework successfully obtains information on both images and automatically discovers an optimal feature fusion strategy that is particularly beneficial for face swap tasks. 
conclusion: 
 In this paper, we propose the concept of code manipulation as a novel approach to achieve face swapping. By leveraging a prior codebook learned from natural face images, we manipulate the codes to achieve high-fidelity results. Unlike traditional techniques for extracting identity features, we propose a symmetrical framework that concentrates on the global information of the both images and manipulates each code individually to achieve precise face swap of two images at once. Our method is validated across a broad range of experiments, proving its superior ability to produce natural, high-fidelity facial images. We believe that this approach will significantly contribute to advancing the field of face swapping research. 
 
 Appendix:  


Decision: The paper received mixed scores (BR,WR,BA,BA). One reviewer decreased the final rating (BA->WR) yet without deepened reasoning, while another increased their score (BR->BA) after rebuttal clarified some mistakes in the initial review.

The scores do not provide a very clear and unique indication. The major positive aspects of the paper are the extensive experiments which show the method outperforms previous ones, and the conceptual idea of casting the problem of face swapping as manipulation of pieces of latent codes in a disentangled latent space. Whereas this idea is not innovative as pointed out by gjBy, it seems to be addressed in a way such that disentanglement is achieved locally, so that swapping results are improved quantitatively and qualitatively. 

Most limitations found by reviewers were clarified in the rebuttal. Some still remain, in particular the lack of comparison with some recent methods and explanations as pointed out by gjBy, and lack of technical novelty.  

Even though some comparisons are neglected and some deepened analyses missing, the overall contribution is still considered good.

Considering the reviewers' scores, comments, paper and rebuttal, the AC preliminarily still recommends acceptance. On the one hand reviewers found some technical similarities with previous works ([1] in gjBy review), yet the rebuttal provided quite detailed description of the conceptual differences and advantages. On the other hand, the positive comments from reviewers suggest such changes led to an improvement in the performance. Overall, whereas the paper does not introduce breakthrough discoveries it looks a good contribution.

Reviews: review: paper_summary: paper_summary: This paper presents a symmetrical framework to achieve face swapping with high-fidelity and realism. A Transformer-based architecture is proposed to update face codes, which enables precise face swapping manipulations. Meanwhile, a mask generator is incorporated to achieve seamless blending of the generated face with the background of target image. Extensive experiments on face datasets verify the ability of the proposed method to not only achieve efficient identity transfer but also substantially reduce the visible artifacts.
strength_weakness: strength_weakness: 1. The architecture details of each proposed module are described in detail and the figures illustrates each module well, which are easy to follow. 
2. The authors provide rich experiments on different datasets, e.g., CelebA-HQ and FF++. Experimental results show the priority of the proposed method over other face swapping baselines from both quantitative and qualitative aspects.
limitations: limitations: 1. The novelty or the first contribution of this paper that states about the symmetrical design is quite limited. The previous paper [1] has already present this design and this paper was posted to arXiv in 2022. And the description in this paper does not adequately explain the motivation for symmetrical design and its advantages.
2. The analysis of the limitations of the diffusion methods is insufficient. The authors should provide a more in-depth explanation or experimental verification of why limited representations or unidirectional face swapping design can lead to synthesis artifacts. 
3. What’s the main differences between Stage 1 and the training process of VQGAN? Why not adopt the VQGAN pre-trained on the FFHQ dataset?
4. For the Regional Control Swap in section 4.5.1, how the selective manipulation of codes in the code space can be achieved to enable such fine-grained facial feature control?
5. Lack of comparative analysis of the related face swapping work that is also based on attention mechanism for code swapping [2].

[1] Q. Li, W. Wang, C. Xu, Z. Sun and M. -H. Yang, "Learning Disentangled Representation for One-Shot Progressive Face Swapping," in IEEE Transactions on Pattern Analysis and Machine Intelligence, doi: 10.1109/TPAMI.2024.3404334.
[2] Zeng H, Zhang W, Fan C, et al. Flowface: Semantic flow-guided shape-aware face swapping[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2023, 37(3): 3367-3375.
paper_summary: paper_summary: This paper presents a Transformer-based architecture designed to independently update each code, enabling more precise manipulations. Extensive experiments on CelebA-HQ and FF++ demonstrate that the method not only achieves efficient identity transfer but also significantly diminishes visible artifacts.
strength_weakness: strength_weakness: 1. This paper introduces an intriguing approach that conceptualizes face swapping as a manipulation of codes within a designated code space.
2. In addition to metric tests, this paper also conducted a user study.
3. The authors performed extensive ablation experiments.
limitations: limitations: 1. Why are the representations learned in the GAN's latent space limited, while those in the code space are not? It is suggested that the authors provide further explanation.
2. How are the test pairs on FF++ constructed, and why are they chosen randomly instead of using the original dataset's test pairs?
paper_summary: paper_summary: This paper leverages a codebook to capture high-quality facial features. The method converts face swapping into a code manipulation task using a Transformer-based architecture. Additionally, a mask generator is incorporated to seamlessly blend the swapped face with the background of the target image.
strength_weakness: strength_weakness: 1. This paper proposes an innovative approach that conceptualizes face swapping as a process of code manipulation in a code space.
2. The method has the ability of regional control swap through selective manipulation of codes in the code space.
limitations: limitations: 1. The paper does not compare this method with face-swapping methods based on StyleGAN. StyleGAN-based face-swapping methods can generate highly realistic facial images.
2. The paper mentions that a mask generator is used to achieve seamless blending of the generated face with the background of the target image, but it does not provide specific challenging examples of face occlusion to illustrate the advantages of the algorithm. Provide examples where faces are partially occluded (e.g., by hair, hands, glasses, etc.), showing how your algorithm successfully achieves seamless blending.
paper_summary: paper_summary: This paper introduces a symmetric face-swapping framework that utilizes vector quantization techniques. The framework concentrates on the global information of the two images and manipulates each code separately, allowing for precise face swapping. The effectiveness of this approach is validated through qualitative and quantitative experiments.
strength_weakness: strength_weakness: 1. The paper is well-written and easy to understand.
2. The paper visualizes results well and has other applications.
limitations: limitations: 1. Why did unidirectional face swapping previously lead to the creation of artifacts? What experiments support this conclusion?
2. How are fine-grained masks produced? If a pre-trained model was used to generate the Ground Truth, why not use this model directly instead of adding an extra step to auto-generate the mask?
3. Why need to use vector quantization techniques?
4. The current problem with face-swapping is its poor performance when the gap between face shapes is large. The authors' experiments do not appear to highlight the advantages in this respect.