# Modal-Guided Multi-Domain Inconsistency Learning for Face Forgery Detection
## 2 Related Work
### Unimodal Deepfake Detection Approaches  
**Image-based face forgery detection:** The traditional methods [14, 15, 16] have limited practical ability and dissemination due to the highly professional knowledge they require and their unsatisfactory generalization. Moreover, most image-based deepfake detection methods focus on the spatial inconsistencies within a single frame. For example, [PERSON] et al. [17] adopt a two-branch network to exploit the relationships between local regions, which takes into account both the RGB and frequency information. [PERSON] et al. [18] discern the blending boundary by extracting more general face representations called face X-rays, which contributes to the robustness of their deepfake detection model. [PERSON] et al. [19] devised a localized artifact attention network (LAA-Net), achieving high-quality face forgery detection by rationally combining a feature pyramid mechanism and an attention mechanism within a multi-task learning framework. [PERSON] et al. [20] employ disentanglement learning to capture demographic and domain-agnostic forgery clues, forcing the models to simultaneously consider the features, loss, and optimization aspects. [PERSON] et al. [21] introduce the mechanism of neighboring pixel relationships to extract and characterize common representations of synthetic artifacts derived from up-sampling operations for generalized deepfake detection. [PERSON] et al. [22] propose a multi-attention detection framework for exploring discriminative local regions and enhancing the texture features, which captures more subtle artifacts. [PERSON] et al. [23] leverage central difference convolution (CDC) and atrous spatial pyramid pooling (ASPP) to exploit multi-scale texture difference information for robust deepfake detection. [PERSON] et al. [24] designed Specific and Common Forgery Feature Extractors to explore the blending boundaries caused by different forgery methods, which improved the generalization of deepfake detection models. Most of these works have tried to find spatial inconsistencies in the image modality while ignoring temporal clues. **Video-based face forgery detection:** While some image-based approaches obtain significant improvements for image forgery detection, some works have attempted to extend these into video-tampered detection and have achieved unsatisfactory results due to the stronger timing and compression considerations than those that apply to images. [PERSON] et al. [25] utilize a two-stream network based on one-class classification loss to propagate the original RGB information and amplify the multi-band frequencies via a Laplacian of Gaussian (LoG), which pulls the real video closer and pushes the doctored video away simultaneously. [PERSON] et al. [21] design a dynamic fine-grained module that captures distances and a multi-scale spatiotemporal aggregation module for modeling precise spatiotemporal inconsistencies by considering both long- and short-range inter-frame inconsistencies in facial action. For the visual modal, our method takes both temporal inconsistency features in the video modal and finer spatial subtle artifact signals from key frames into account, which collaboratively allows it to learn the inconsistent visual features for a higher performance. **Audio-based human forgery detection:** [PERSON] et al. [26] use various spectrograms to replace hand-crafted features as the input for an end-to-end-network DNN which directly detects the unrevealed characteristics that reside in modified speech. [PERSON] et al. [27] leverage a RIDNet denoiser, as well as a multi-head relative interactive method, to extract the underlying forensic noise traces left behind in deepfake videos. [PERSON] et al. [28] apply the \"CNN+LSTM\" network to extracting key representations from conventional audio features, such as the MFCC [29] and its spectrograms, which is beneficial for detecting spoofed audio. There are relatively few methods for audio-based human forgery detection because it is often detected as a whole alongside video.