# Face Swapping Consistency Transfer with Neural Identity Carrier
## 3 Methods
### Training Losses
#### 3.4.1 Static Detail Extractor Training  
In Section 3.3, we introduce the pipeline of detail reconstruction. Given a set of images from one individual, the detail reconstruction is trained by minimizing \(\mathcal{L}_{recon}\), formally as  
\[\mathcal{L}_{recon}=\mathcal{L}_{pho}+\mathcal{L}_{mrf}+\mathcal{L}_{sym}+ \mathcal{L}_{chr}+\mathcal{L}_{reg}, \tag{5}\]  
with photometric loss \(\mathcal{L}_{pho}\), ID-MRF loss \(\mathcal{L}_{mrf}\), soft symmetry loss \(\mathcal{L}_{sym}\), coherence loss \(\mathcal{L}_{chr}\) and regularization loss \(\mathcal{L}_{reg}\).  
The photometric loss \(L_{pho}\) computes the distance of the input image \(I\) and the rendering \(I_{r}\) as \(L_{pho}=\|V_{I}\odot(I-I_{r})\|\). Here, \(V_{I}\) is a binary mask generated by a face segmentation method [5] which represents the facial region, and \(\odot\) denotes the Hadamard product. The photometric loss \(L_{pho}\) can enforce more attention focused on the facial region and awareness of occlusions with the help of mask \(V_{I}\).  
Besides, we adopt the Implicit Diversified Markov Random Fields (ID-MRF) loss for geometric details reconstruction [26]. Given two images of the same subject, the ID-MRF loss minimizes the distance between these two images on VGG19's feature level. As the same setting as previous work [26], we compute the ID-MRF loss on layers _conv3\(2\) and _conv4\(2\) of VGG19 as  
\[\mathcal{L}_{mrf}=2\mathcal{L}_{M}(conv4\_2)+\mathcal{L}_{M}(conv3\_2), \tag{6}\]  
where \(\mathcal{L}_{M}(layer)\) denotes the VGG19's feature-level distance between \(I^{\prime}_{r}\) and \(I\) on layer _layer_ of VGG19.  
In consideration of occlusions, we also add a soft symmetry loss to regularize the non-visible face parts. The soft symmetry loss can be formulated as  
\[\mathcal{L}_{sym}=V_{uv}\odot(D-Flip(D)), \tag{7}\]  
where \(V_{uv}\) denotes the facial mask in UV space, and \(Flip\) denotes the horizontal flip operation.  
Figure 3: Illustration of our 3D detail extractor’s training process. \(E_{c}\) is the state-of-the-art 3D reconstruction model which disentangles the input face. The disentangled face parameters are then recombined into coarse feature and detail feature respectively.  
As mentioned in Section 3.2, detail information is divided into two parts, dynamic and static. We believe that replacing the static detail codes of another image of the same subject should have no effect on the final rendered image, which conforms to the logical evidence that one specific person should have his own consistent static detail code. Formally, given two images \(l_{i}\) and \(l_{j}\) of the same subject, the loss is defined as  
\[\begin{split}\mathcal{L}_{cltr}=\|I_{i}-\mathcal{R}(M(\beta_{i},\theta_{i},\psi_{i}),A(\alpha_{i}),\\ F_{d}(\delta_{j},\psi_{i},\theta_{i}),l_{i},c_{i}))\|^{2}\end{split} \tag{8}\]  
where \(\beta_{i}\), \(\theta_{i}\), \(\psi_{i}\), \(\alpha_{i}\), \(l_{i}\), and \(c_{i}\) are the parameters of \(l_{i}\), while \(\delta_{j}\) is the detail code of \(l_{j}\).  
Finally, the detail displacements \(D\) are regularized by \(\mathcal{L}_{reg}=\|D\|_{1,1}\) to reduce noise.