# Deep Fake Image Detection Based on Pairwise Learning
## 2 Fake Face Image Detection
### Discriminative Feature Learning  
The main drawback of supervised learning is that it is hard to identify the subject that is excluded from the training phase. To enhance the performance of the proposed method, we introduce contrastive loss to learn the CFFs by pairwise learning. Therefore, the Siamese network structure [22] is used for allowing the pairwise inputs, as illustrated in Figure 3.  
With the aim to make the proposed CFFN learn the discriminative features during the training process, the contrastive loss term is incorporated into the energy function of the traditional loss function for supervised learning (i.e., the cross-entropy loss). Afterward, given the face image pair \((\mathbf{x}_{1},\mathbf{x}_{2})\) and the pairwise label \(\mathbf{y}\), where \(y=0\) indicates an impostor pair, and \(y=1\) indicates a genuine pair, the energy function between two images is defined as:  
\[E_{W}(\mathbf{x}_{1},\mathbf{x}_{2})=||f_{CFFN}(\mathbf{x}_{1})-f_{CFFN}( \mathbf{x}_{2})||_{2}^{2}. \tag{1}\]  
The most intuitive way to learn the discriminative features is to minimize the energy function \(E_{W}\) given by (1). Specifically, direct computation of \(E_{W}(\mathbf{x}_{1},\mathbf{x}_{2})\) by calculating \(l_{2}\) norm distance in the feature domain leads to a constant mapping, and the constant mapping makes any input to a constant vector such that the energy function \(E_{W}\) can be minimized. For instance, the learned mapping function can be \(f_{CFFN}(\mathbf{x}_{1})=f_{CFFN}(\mathbf{x}_{2})=[1,1,\dots,1]\). Thus, the constant mapping leads to useless feature representation. Therefore, to overcome this problem, the contrastive loss is introduced to learn the discriminative feature representation as well as to avoid constant mapping, which can be expressed as:  
\[L(W,(P,\mathbf{x}_{1},\mathbf{x}_{2}))=0.5\times(y_{ij}E_{w}^{2})+(1-y_{ij}) \times max(0,(m-E_{w})_{2}^{2}), \tag{2}\]  
\begin{table}
\begin{tabular}{c l l} \hline \hline
**Layer Number** & **Feature Learning** & **Classification** \\ \hline
1 & conv. layer, kernel = 7 \(\times\) 7, stride = 4, \#channels = 48 & Conv. layer, kernel = 3 \(\times\) 3, \#channels = 2 \\
2 & Dense block \(\times\) 2, \#channels = 48 & Global average pooling \\
3 & Dense block \(\times\) 3, \#channels = 60 & Fully connected layer, neurons = 2 \\
4 & Dense block \(\times\) 4, \#channels = 78 & \\
5 & Dense block \(\times\) 2, \#channels = 126 & \\ \hline
6 & Fully connected layer, neurons = 128 & SoftMax layer \\ \hline \hline \end{tabular}
\end{table}
Table 1: Structure Parameters of the Proposed Common Fake Feature Network for Fake Face Image Detection.  
Figure 3: The proposed pairwise learning based on the Siamese network and contrastive loss.  
where \(m\) denotes the predefined threshold value. When the input is the genuine pair \(y_{ij}=1\), the cost function tends to minimize the energy (defined by the feature distance) \(E_{W}\) between two images. When the input is an impostor pair, the contrastive loss will minimize the function \(max(0,(m-E_{w})\). In other words, the energy \(E_{W}\) will be maximized if the feature distance between the impostor pair is smaller than the predefined threshold value \(m\). In this way, it is possible to learn the common characteristics of the fake images generated by different GANs. When the contrastive loss is used, the feature representation \(f_{\text{CFFN}}(\mathbf{x}_{i})\) will tend to become similar to \(f_{\text{CFFN}}(\mathbf{x}_{j})\) at \(y_{ij}=1\) (i.e., for a fake-fake or real-real pair). By iteratively train the network \(f_{\text{CFFN}}\) using the contrastive loss, the CFFs of the collected GANs can be learned well.