  We propose to detect Deepfake generated by face manipulation based on one of
their fundamental features: images are blended by patches from multiple
sources, carrying distinct and persistent source features. In particular, we
propose a novel representation learning approach for this task, called
patch-wise consistency learning (PCL). It learns by measuring the consistency
of image source features, resulting to representation with good
interpretability and robustness to multiple forgery methods. We develop an
inconsistency image generator (I2G) to generate training data for PCL and boost
its robustness. We evaluate our approach on seven popular Deepfake detection
datasets. Our model achieves superior detection accuracy and generalizes well
to unseen generation methods. On average, our model outperforms the
state-of-the-art in terms of AUC by 2% and 8% in the in- and cross-dataset
evaluation, respectively.
