# SupCon-MPL-DP: Supervised Contrastive Learning with Meta Pseudo Labels for Deepfake Image Detection
## 3 Proposed SupCon-MPL-Based Deepfake Detection
### SupCon-MPL Loss Function  
SupCon-MPL, as shown in Figure 3, is composed of a teacher model (\(T\)) and a student model (\(S\)), each of which consists of an encoder and a linear classifier. SupCon-MPL has two loss functions in order to sequentially train each model. One involves the teacher model \(T\) distilling knowledge to the student model \(S\), while the other entails the teacher model \(T\) training from the feedback factor provided by \(S\) on the labeled data. The knowledge distilled by \(T\) includes previously learned content about deepfakes. In SupCon-MPL, let the parameters of \(T\) classifier and \(S\) classifier be \(\theta_{T}\), \(\theta_{S}\), respectively, and denote the batch of images and labels on the labeled data as \((x_{t},y_{t})\in D_{K}\), and the  
Figure 3: Modified Meta Pseudo Labels and loss functions for deepfake detection. batch of images on the unlabeled data as \(x_{u}\in D_{U}\). The goal of SupCon-MPL is to minimize the parameters \(\theta_{S}^{PL}\) of the generalized deepfake detection model \(S\):  
\[\theta_{S}^{PL}=\underset{\theta_{S}}{argmin}\underbrace{E_{x_{u}}[CE(T(x_{u}; \theta_{T}),S(x_{u};\theta_{S}))]}_{L_{u}:=(\theta_{T},\theta_{S})} \tag{1}\]  
Hence, the objective function of SupCon-MPL is defined as follows. \[ \begin{split} L_{l}\text{ with respective to }\theta_{T}:\\ \min_{\theta_{T}}L_{l}\big{(}\theta_{S}^{PL}(\theta_{T})\big{)},\\ \text{ where }\theta_{S}^{PL}(\theta_{T})=\underset{\theta_{S}}{ \operatorname{argmin}}\ L_{u}(\theta_{T},\theta_{S}).\end{split}  \tag{2}\]  
For optimization, SupCon-MPL approximates \(\theta_{S}^{PL}(\theta_{T})\) by the learning rate \(\eta\)s, and then,  
\[\theta_{S}^{PL}(\theta_{T})\approx\theta_{S}-\eta s\cdot\
abla_{\theta_{S}}L _{u}(\theta_{T},\theta_{S}) \tag{3}\]  
defines the final objective function as follows:  
\[ \begin{split} L_{l}\text{ with respective to }\theta_{T}:\\ \min_{\theta_{T}}L_{l}\big{(}\theta_{S}-\eta s\cdot\
abla_{ \theta_{S}}L_{u}(\theta_{T},\theta_{S})\big{)},\\ \text{ where }\theta_{S}^{PL}(\theta_{T})=\underset{\theta_{S}}{ \operatorname{argmin}}\ L_{u}(\theta_{T},\theta_{S}).\end{split}  \tag{4}\]  
Both \(T\) and \(S\) consist o-f an encoder and a classifier and are trained according to their respective loss functions. The loss function of \(ENC_{T}\), the encoder of \(T\), is composed of SupConLoss [38], and the loss function of CL\(F_{T}\), the classifier, is composed of labeled loss for the labeled data \(x_{l}\) and MPL loss, reflecting the feedback from \(S\). First and foremost, \(g_{T\_contrast}^{(t)}\) the loss function of \(ENC_{T}\), receives image pairs \((RandAugment_{a}(x_{l}),RandAugment_{b}(x_{l}),y_{l})\) as inputs that reflect different random augmentations on the same image \(x_{l}\) and the label \(y_{l}\). Subsequently, the loss value is obtained by passing image pairs through SupConLoss [38]. At this juncture, given the similarity between the current training process and that of the original MPL's UDA loss, the utilization of the UDA loss is no longer used:  
\[ \begin{split} g_{T\_contrast}^{(t)}=\
abla_{\theta_{T}}SupConLoss (RandAugment_{a}(x_{l}),RandAugment_{b}(x_{l}),y_{l})\big{|}_{\theta_{T}= \theta_{T}^{(t)}}\end{split}  \tag{5}\]  
\(ENC_{T}\) is promptly updated following the computation of the \(g_{T\_contrast}^{(t)}\):  
\[\theta_{T\_ENC}^{(t+1)}=\theta_{T\_ENC}^{(t)}-\eta_{S}\cdot g_{T\_contrast}^{ (t)} \tag{6}\]  
The labeled loss of \(ClF_{T}\), \(g_{T\_supervised}^{(t)}\) measures the difference between \(y_{l}\) and the label predicted by \(T\) through cross-entropy loss (CE Loss). Here, \(emb_{l}^{T}\) denotes the embedding value derived by passing the labeled data \(x_{l}\) through \(ENC_{T}\):  
\[ \begin{split} g_{T\_supervised}^{(t)}=\
abla_{\theta_{T}}CE\Big{(} y_{l},CLF_{T}\Big{(}emb_{l}^{T};\theta_{T}\Big{)}\Big{)}\big{|}_{\theta_{T}= \theta_{T}^{(t)}}\end{split}