Orthogonal Subspace Decomposition for Generalizable AI-Generated Image Detection
Authors: Zhiyuan Yan
Jiangming Wang
Peng Jin
Ke-Yue Zhang
Chengchun Liu
Shen Chen
Taiping Yao
Shouhong Ding
Baoyuan Wu
Li Yuan
Venue: ICML 2025 oral (2025)
Keywords: AI-Generated Image Detection
Face Forgery Detection
Deepfake Detection
Media Forensics
Primary area: applications->computer_vision
TL;DR: We introduce a novel approach via orthogonal subspace decomposition for generalizing AI-generated images detection.

Abstract: Detecting AI-generated images (AIGIs), such as natural images or face images, has become increasingly important yet challenging. In this paper, we start from a new perspective to excavate the reason behind the failure generalization in AIGI detection, named the asymmetry phenomenon, where a naively trained detector tends to favor overfitting to the limited and monotonous fake patterns, causing the feature space to become highly constrained and low-ranked, which is proved seriously limiting the expressivity and generalization. One potential remedy is incorporating the pre-trained knowledge within the vision foundation models (higher-ranked) to expand the feature space, alleviating the model's overfitting to fake. To this end, we employ Singular Value Decomposition (SVD) to decompose the original feature space into two orthogonal subspaces. By freezing the principal components and adapting only the remained components, we preserve the pre-trained knowledge while learning fake patterns. Compared to existing full-parameters and LoRA-based tuning methods, we explicitly ensure orthogonality, enabling the higher rank of the whole feature space, effectively minimizing overfitting and enhancing generalization. We finally identify a crucial insight: our method implicitly learns a vital prior that fakes are actually derived from the real, indicating a hierarchical relationship rather than independence. Modeling this prior, we believe, is essential for achieving superior generalization. Our codes are publicly available at https://github.com/YZY-stack/Effort-AIGI-Detection.

Orthogonal Subspace Decomposition for Generalizable AI-Generated Image Detection
Zhiyuan Yan 12 Jiangming Wang Peng Jin
Ku-Yue Zhang? Chengehun Liu
Shen Chen
Taiping Yao
Shouhong Ding²
Baoyuan Wu
Li Yuan
Abstract
Quick Shortcut to Fake
Fake Loss
Real Loss
Test AUC
Detecting AI Al-genrrated images (AIGIs), such as natural images or face images, has become in- creasingly important yet challenging. In this pa- per, we start from a new perspective to exca- vate the reason behind the failure generalization in AIGI detection, named the asymmetrr phe- nomenon, where a naively trained detector tends to favor overfitting to the limited and monotonous fake patterns, causing the feature space be- come highly constrained and low-ranked, which is proved seriously limiting the expressivity and generalization. One potential remedy is incorpo- rating the pre-trained knowledge within the vision foundation models (higher-ranked) to expand the feature space, alleviating the model's overfitting to fake. To this end, we employ Singular Value Decomposition (SVD) to decompose the original feature space into two orthogonal subspaces. By freezing the principal components and adapting only the remained components, we preserve the pre-trained knowledge while learning fake pat- terns. Compared to existing full-parameters and LoRA-based tuning methods, explicitly en- sure orthogonality, enabling the higher rank of the whole feature space, effectively minimizing overfitting and enhancing generalization. We fi- nally identify a crucial insight: our method im- plicitly learns a vital prior that fakes are actually derived from the real, indicating a hierarchical relationship rather than independence. Modeling this prior, we believe, is essential for achieving superior generalization. Our codes are publicly available at GitHub.
1.00
1.00
Loss (Cross-entropyy
Best AUC
0.80
0.80
0.60
0.60
AUC
0.40
0.40
0.20
100x Larger
0.00
0.00
1500
2500
Iterations
Figure I. Illustration of the asymmetry phenomenon in AI- generated image detection. We show that the baseline detector (i.e, Xception) tends to quickly overfit to the fake patterns in the training (Rossler al., 2019), causing the limited general- ization when facing previously unseen fakes (Li et al., 2020b).
1. Introduction
The rapid development of Al generative technologies has significantly lowered the barrier for creating highly realis- tic fake images. As deep generative models advance and mature (Goodfellow al., 2020; Ho et al., 2020; Rom- bach et al., 2022b; Yan et al., 2025), the proliferation of AI- generated images (AIGIs!) ) has drawn considerable attention, driven by their ability to produce high-quality content with relative ease. However, these advancements also introduce significant risks, if misused for malicious purposes such as deepfakes (mainly including swappinn (Korshunov Marcel, 2018) and face-reenactment (Thies al., 2016)), which may violate personal privacy, spread misinformation. and erode trust in digital media. Consequently, there is an urgent need to develop reliable and robust framework for detecting AIGIs.
Eqqul contribution 'Peking University Shenzhen Gradu- ate School "Tencent Youtu Lab "The Chinese University of Hong Kong, Shenzhen. Correspondence to: Taiping Yao <taip- mggao@ttencent.ns. Li Yuan <yuanli-ece@@ pku.edu.cn
Most existing studies AIGI detection (Wang et al., 2020b; Rossler al., 2019) typically approach the real/fake classi- fication problem symmetric binary classification task, akin to the "cat versus dog" problem. A standard binary
Proceedings of the ndnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
II the context of this research, AIGI primarily refers deep- fakes (face-swapping) and synthetic images (e.g., nature or arts).

---

Orthogonal Subspace Decompasition for Generalizable AI-Gencrated Image Detection
Seen Fake Pattern
Sean Rea
Seen Fake Pattern
Sean Rad Sean Fabe Unoeen thad Unownn Tatte
Vanilla Delector
Ours
Seen Faoe
Explotred Venteroo Principll Companents
Explaimed Verience fietle Principal Components
Ureeet that
Urewen Taaa
prino pal component
Second prinippal component
(a) Xception
(b) CLIP (full-finetune)
30
Principal Component Indes"
1s prindipal Component Indec.
Figure 2. t-SNE visualizations between Xception and CLIP ftull-fnetuune. We show that hoth models only learn the specific fake patterns within the training set, treating samples with seen fake patterns as fake while other samples are all considered real, thereby limiting their generalization in detecting unseen fakes.
principal companents capture Information
Many principal components meaningfu
Figure 3. Analysis for the effective information contained in the model's feature space. We apply PCA for dimension reduction and visualize the explained variance ratio of principal components with high contribution. We show that the baseline model trained on the AIGI dataset can be highly constrained and low-ranked.
classifier, often based on deep neural networks, is trained to distinguish between real and fake images by predicting the likelihood of a given test image being fake during inference. Although this paradigm yields promising results when the training and testing distributions (in terms of fake generation methods) are similar, its performance tends to degrade sig- nificantly when applied to previously unseen fake methods, indicating the generalization issue (Ojha et al., 2023).
nent Analysis (PCA). Specifically, we visualize the ex- plained variance ratio of different principal components for the model's feature space in Fig. 3. The results show that the feature space of the naive detector can be highly constrained and low-ranked, with only two principal com- ponents to capture all information, resulting in limited generalization. This aligns with the previous theoretical analysis (Gunasekar et al., 2017) that low-ranked feature spaces hinder generalization by memorizing trivial patterns.
To understand the underlying reasons for the failure in gen- eralization, we have conducted extensive preliminary in- vestigations and identified an asymmetry phenomenon in AIGI detection: naively trained detectors tend take the shortcut and very quickly overfit the limited fake patterns presented in the training set. Visualization in Fig. cor- roborates this claim. Specifically, the vanilla detector (i.e., Xception (Rossler et al., 2019)) quickly fits the fake pat- terns at the very early training stage (only a few iterations), resulting in a very low loss of fake, while the real loss is significantly higher than the fake loss (~ 100x larger). This is likely because existing AIGI detection datasets (Rossler et al., 2019; Wang et al., 2020b) typically contain limited and homogeneous fake types, while real samples exhibit significantly greater diversity and variance between each other such as different categories and scenarios.
One potential remedy to address the low-ranked problem is incorporating pre-trained knowledge within vision founda- tion models (VFMs), which provide higher-ranked feature representations, to expand the low-ranked feature space, thus alleviating the overfitting. However, naively fine-tuning VFM (even CLIP) risks distorting the original rich repre- sentation feature space (Fig. 2), pushing the feature space to become low-ranked again (verified in Fig. 6).
To address this, we design a novel approach called Effort: Efficient orthogonal modeling for generalizable AIGI detec- tion. Specifically, we employ Singular Value Decomposition (SVD) to construct two orthogonal subspaces. By freez- ing the principal components and adapting the remained components, we preserve the pre-trained knowledge while learning forgery-related patterns.
Consequently, the learned feature spaces become fake- dominated and thus highly constrained. As evidenced by the t-SNE visualization in Fig. 2, we see that the whole feature space is indeed dominated by the forgery patterns, where both the Xception (Vanilla CNN) and CLIP (Radford et al., 2021) detector group only the specific fake patterns within the training set into a single cluster, while all other data, including real samples and fake samples from unseen forgeries, are mapped into separate cluster.
We have conducted extensive experiments on both deepfake detection and synthetic image detection benchmarks and find that our approach achieves significant superiority over other SOTAs with very little training cost. Compared to ex- isting full-parameters and LoRA-based tuning methods, we explicitly ensure orthogonality, enabling the higher rank of the whole feature space, effectively minimizing overfitting to fake and enhancing generalization.
To quantify this, we analyze the effective information con- tained in the model's 's feature space via Principal Compo-
Finally, we arrive a key insight for generalizable AIGI detection: there exists an important prior of the detection
In PCA, it refers to variance captured by principal components. Larger eigenvalues indicate components explaining more variance and contributing more to data representation.
"The "rank" here means the number of significant principal components.
2

---

Orthogonal Subspace Decompasition for Generalizable AI-Gencrated Image Detection
task, where fake images are generated from real ones, es- tablishing hierarchical relationship rather than an in- dependent or symmetric one. When aligning semantic information-suue as distinguishing a fake dog from real dog-this prior allows the model to focus discrimination within a smaller, semantically consistent subspace, e.g., only among dogs (see Fig. 4 for illustration). This focused dis- crimination simplifies the task and aligns with theoretical re- sults from Rademacher complexity (Mohri & Rostamizadeh, 2008), which states that reducing model complexity leads to tighter generalization bounds. In contrast, naively trained detectors that treat real and fake data as independent fail to capture this structure, resulting in limited generalization performance. Therefore, modeling this hierarchical prior, we believe, is vitally crucial for AIGI detection.
Unseen Fake (CycleGAN)
Semanric
Forgery
Forgery
Vanilla CNN (Ouly Forgery Specd)
Ours (Semantic sad Forgery Space)
Acc:
Real:
Apple
Hone
Omags
Summer
Winker
7chra
Fake:
Apple
Sunmer
Winter
Zebra
Figure 4. t-SNE visualizations of the latent feature spaces be- tween vanilla CNN (Wang et al., 2020b) and ours. Our method achieves orthogonal learning between the dimensions of semantic and forgery, allowing the model capture fake patterns the semantically-alignn subspace, simplifying the discrimination and thereby improving the generalization.
Our work makes the following key contributions:
Asymmetry phenomenon in AIGI detection: We in- troduce the concept of asymmetry phenomenon, where a naively trained detector tends to quickly fit the seen fake methods well but, in doing so, it often overfits to specific fake patterns in the training set, limiting its generalization ability to detect unseen fake methods.
2. Related Work
Our work focuses on detecting Al-generated images (AIGIs), especially deepfake images (e.g., face-swapping) and synthetic images (e.g. nature or art), following Yan et al. (2024a). As the majority of recent works specifi- cally focus on dealing with the generalization issue, where the training and testing distribution differ (in terms of fake methods), we will briefly introduce the classical and recent detection methods toward generalization in deepfake and synthetic images, respectively.
New perspective to explain the failure reasons be- hind generalization: We use PCA to quantitatively as- sess the effective information within the learned feature space, and we find that the overfitting to fake, results in a highly low-ranked and constrained feature space, thus leading to the limited generalization capability.
Novel method via orthogonal subspace decomposi- tion: To address the overfitting, we propose novel ap- proach, Effort, with two careful designs: (1) incorporat- ing the pre-trained knowledge (proving higher-ranker feature space) within the vision foundation models to expand the previous feature space, improving the model's expressivity and alleviating the overfitting; and (2) employing SVD to explicitly construct two orthog- onal subspaces, where the principal one for preserving pre-trained knowledge and the remained one for leam- ing new forgeries, avoiding the distortion of original rich feature space during learning fakes.
Generalizable Deepfake Image Detection. The task of deepfake detection grapples profoundly with the issue of generalization. To tackle the generalization issue, one main- stream approach is fake pattern learning. Most existing works are within this line. These methods generally de- sign a "transformation function"' e.g., frequency transfor- mation (Li et al., 2021; Luo et al., 2021; Liu et al., 2021a), blending operations (Li et al., 2020a; Zhao et al., 2021; Shiohara & Yamasaki, 2022; Chen et al., 2022a), recon- struction (Zhu et al., 2021; Cao al., 2022), content/ID disentanglement (Yan et al., 2023a; Fu et al., 2025; Huang et al., 2023; Dong et al., 2023), to transform the original input x into x', where they believe that the more general fake patterns can be captured within the feature space of x' compared to x. However, given the ever-inereasing diver- sity of forgery methods in the real world, it is unrealistic to elaborate all possible fake patterns and "expect" good gener- alization on unseen fake methods. Another notable direction is to real distribution learning. with a specific methodol- ogy involved: one-class anomaly detection (Khalid & Woo, 2020; Larue et al., 2023). Specifically, Khalid & Woo (2020) introduced a one-class- based anomaly detection, where "ab-
Key insight toward generalizable detection: We re- veal that fake data is actually generated from real data, forming a hierarchical relationship rather than being independent. Our method effectively models this prior by maintaining the pre-trained semantic components while adapting to fake detection effectively, enabling the detector to make discrimination the semantic- aligned subspaces, reducing model complexity and thus improving generalization.
3

---

Orthogonal Subspace Decompasition for Generalizable AI-Gencrated Image Detection
normal" data is detected by the proposed reconstruction error as the anomaly score. Larue al. (2023) proposed similar approach to create pseudo-fake "anomaly" samples by using image-level blending on different facial regions. However, it is challenging to ensure that the detector can learn a robust representation of real images by using the very limited real data in existing deepfake datasets (e.g., the FF++ dataset (Rossler et al., 2019) contains only 1,000 real videos with imbalanced facial attribute distributions (Trinh & Liu, 2021)).
Formally, given a pre-trained weight matrix W E Rdixda for certain linear layer, we perform SVD decompose W:
xda
W = UEVT
where U E [dd xdd and V E Rdxxds are orthogonal ma- trices containing the left and right singular vectors, respec- tively, and Σ E IR xd2 is a diagonal matrix with singular values on the diagonal. Since the linear layer of VFM gener- ally has the same input and output dimensions, we consider the case of SVD with d1 d2 = n in the following discus- sion.
Generalizable Synthetic Image Detection. With the rapid advancement of existing Al generative technologies, the scope of forged content has expanded beyond facial forgeries to encompass a wide range of scenes. In this con- text, similar to the deepfake detection field, most existing works typically focus on fake pattern learning that mines the low-level forgery clues from different aspects. Specifically, several approaches have been proposed to capture low-level artifacts, including RGB data augmentations (Wang et al., 2020b), frequency-base -based features (Jeong et al., 2022), gradi- ents (Tan et al., 2023), reconstruction artifacts (Wang et al., 2023a; Chen et al., 2024; Luo et al., 2024), and neighboring pixel relationships (Tan et al., 2024c), random-mapping fea- ture (Tan et al., 2024a). To illustrate, BiHPF (Jeong et al., 2022) amplifies artifact magnitudes through the application of dual high-pass filters, while LGrad (Tan et al., 2023) uses gradient information from pre-trained models as arti- fact representations. NPR (Tan et al., 2024c) introduces a straightforward yet effective artifact representation by re- thinking up-sampling operations. In addition to learning from scratch, there are also several research works (Ojha et al., 2023; Wu et al., 2023; Liu et al., 2024) that perform fake pattern learning by leveraging the existing vision foun- dation models. For instance, UniFD (Ojha et al., 2023) directly freezes the visual encoder of the pre-trained CLIP model and tunes only a linear layer for binary classification, demonstrating effective deepfake detection even with previ- ously unseen sources. LASTED (Wu et al., 2023) proposes designing textual labels to supervise the CLIP vision model through image-text contrastive learning, advancing the field of synthetic image detection. These arts have shown notable improvement in generalization performance when facing previously unseen fake methods.
To obtain rank-r approximation of the pre-trained weight matrix, we retain only the top singular values and corre- sponding singular vectors:
= U,E,V
(2)
where U, E R"xr, E, E R and V, E R We keep W, frozen during training to preserve dominant pre-trained knowledge learned from large-scale scale data.
The residual component, defined as the difference between the pre-trained weights and the SVD approximation, is used to learn representations specific to fake images:
AW = W - W,, = Un Un-
(3)
where U, n-r E (nx(n-r), E,n E R((-r)x((-r) Vn-r R=x(n-1), It is important to note that AW represents a learnable form associated with the remaining singular value decomposition, reflecting slight modifications or perturba- tions to the original weight matrix.
During training, we only optimize AW while keeping U,. Σ,, and V, fixed. This implementation ensures that the model retains its capability to process real images via the SVD approximation and adapts to detect deepfakes through the trivial residual components of the weight matrix.
To encourage the AW to capture both useful and meaning- ful discrepancy between the real and fake, it's significant to guarantee that optimizing AW does not change the prop- erties of the overall weight matrix W (i.e., Minimize the impact on the real information of the pre-trained weight as much as possible). Thus, we proposed two constraints to realize this goal, as follows.
3. Methodology
Orthogonal Constraint. We maintain the orthogonality among each singular vector to keep orthogonal subspace for learning real/fake:
The overall pipeline of the proposed Effort approach is illus- trated in Fig. 5, aiming to address the asymmetry phenom- ena in AIGI detection. Our approach involves the SVD to construct explicit orthogonality for preserving pre-trained knowledge and learning forgery-relatee patterns, avoiding the distortion of well-learned pre-trained knowledge during learning forgeries.
Lorth = -10 - 11 -1
+ - - -11
(4)
where U E Raxx denote the concatenation of U, and Un-r along the row dimension, V E Rn'x= denote the concatena- tion of V, and Va-r along the row dimension, ||-I|r denotes
4

---

Orthogonal Subspace Decompasition for Generalizable AI-Gencrated Image Detection
Output
Orthogonal learning residual components;
Self-Attention Layer
Ur
North = -10" - /||i
Un-r
Q
K
Pre-trained Weights
SVD
Principle Components
Residual Components
E
A
=|1:01
||Wl|
Proj
W
mg
W, E R
I2 m
W
X
Weights to be tuned
VG Vi
forth
-1PTTV -IBB
Input
RR"
Figure 5. The proposed approach for AIGI detection. The left branch is the decomposition matrix of the principle components approximation using SVD, while the right residual branch enables the orthogonal learning of real/fake discriminative features.
the Frobenius norm, and / the identity matrix of appropri- ate dimensions.
Finally, we provide an algorithm illustration of the proposed approach in Alg. for overall understanding.
Singular Value Constraint. The singular values can be interpreted as a type of scaling that affects the magnitude of the corresponding singular vectors. There a relationship between singular values and the Frobenius norm the weight matrix being decomposed:
Algorithm 1 Effort Approach Algorithm
Input: Pre-trained weight matrix W E R=**; Rank r: Training
data D = [(1a, )=)] Hyperparameters X1. A2
Output: Updated weight matrix W
Step Singular Value Decomposition
2: Decompose W via SVD: W =UVV
3: Retain top singular values and vectors:
W Eo
(5)
4: U, C R Σ, E R V, E B
5: Compute W. V.
Keep W, fixed during training
where o; denotes the i-th singular value of the corresponding weight matrix.
7: Compute residual component: AW = W W,
8: Decompose AW via SVD: AW = U. Un-r-m--VV
9: Initialize AW
To maximize the reduction of the impact of real knowledge,
10: Define concatenated matrices:
11: Ú = [U,, Un=] E R
we constrain the singular values of the optimized weight matrix W to remain consistent with those the original weight matrix W:
12: İ -VV,, Vn-r] R
13: D. Step Training Loop
14: for each epoch do
15: for each batch in D do
L
of
W 111 (6)
16:
Forward Pass
17:
Compute model output using W W, AW
18:
Compute classification loss Lets
19:
Compute Constraints
where W. represents the weights after the optimization of W. and |-| represents the absolute value. Note that this regularization will control the importance the AW during optimization to prevent overfitting of learning real/fake.
20:
Compute orthogonality loss:
Corth 0 - || | VTV 1PVV-10 1|
21:
22:
Compute singular value constraint loss:
| I
23:
Lkov
Loss Function. The overall loss function for training the model combines the classification loss Lels (e.g., cross- entropy loss for binary classification) and the orthogonality regularization loss:
Total Loss
25:
C Lets + Aorth + AgC
26:
BBacwarrd Pass and Optimization
27:
Update AW using gradient descent to minimize f.
28:
end for
L = Lels + t1 Σ Carch + A2
1
29:
end for
30: return Updated weight matrix W <- W. + AW
where Åι, A2 are hyperparameters that balance the impor- tance of the corresponding regularization term, and m rep- resents the number of pre-trained weight matrices on which our approach is applied. In practice, we adapt our approach to the linear layers within the self-attention module across all transformer layers of the VFM to leverage their rich, well-learned real distributions.
4. Experiment
4.1. Deepfake Image Detection Implementation Details. We utilize CLIP ViT-L/14 (Rad- ford et al., 2021) as the default vision foundation model (VFM). We also investigate other VFMs in Tab. 5, We fol- low the pre-processing and training pipeline and use the
5

---

Orthogonal Subspace Decompasition for Generalizable AI-Gencrated Image Detection
Table Benchmarking Results of Cross-dataset Evaluations (Protocol-I) and Cross-method Evaluations (Protocol-2). All detectors are trained on FF++_e23 (Rossler et al.. 2019) and evaluated on other fake data. i indicates the results are obtained by using the model's s checkpoint provided by the authors, otherwise, the results are cited from (Yan et al., b023b; 2024a; Cheng et al., 2024).
Madadds
Trinable
Cross diteet E Evaluzioo
Cross nethed Evalusiion
Prrm
CDI-42
DHD
DWUC
DFFXP
DHo
WIN
FFW
Arg.
[ Lmiface
Hleeace
MonSSwap
ols FoodDoen
PSGAN
InSwap
StmsSoap
Arg
FiNer ( al., 2020)
12M
0.789
0.644
0.715
0.240
0.730
0.728
0.649
0.743
0.809
0.908
0.357
0.494
0.717
0.545
0.757
0.674
0.746
SPSL (Liu et el.. 2021al
21M
0.799
0,67]
0.724
0.770
0.723
0.702
0.794
0.769
0.247
0.745
0.355
0.514
0.666
0.812
0.643
0.665
0.710
SBM (1as al. 1021)
SSM
0.600
01885
0.695
0.238
0.712
0.700
0.794
0.367
0.349
0.204
0.770
0.204
0.659
0.775
0.703
0,994
0.732
CORE (Ni c022) 2022)
22M
0.809
0.822
0.721
0.220
0.765
0.724
0.710
0.762
0.371
0.443
0.959
0.679
0.734
0.988
0.855
0.724
0.83
RBOCK (Cacce/Cao-etal.... 2022)
48M
0823
(1891
0,996
0.734
01786
03898
11812
0.925
0A83
01568
0.985
0.588
01708
0S64
SLADD (Chen il., 2022a)
0.37
0.772
0.756
0.882
0.765
0.825
0.943
0.379
0.794
SBI (Stichara Yanasaki, 20223
0.885
0717
0.948
0.891
0.250
0.594
0.712
0.701
UCF
20036)
0.542
0.770
0.827
0.862
0.800
0.67
llluung 2023)
0.63
0.700
0.60
0.759
0.844
0.769
0.64
USDA (Yon 20244)
131M
0.775
0.701
0312
01875
0.09
0.211
0.855
11733
PoDDett (Cheng et al., 2024)
9694
0.525
0.707
0.328
0.879
0.828
0.929
0.771
0.747
0.92
0.837
0.814
CDFM (Lin 2034)
0.938
0.954
0.530
0.851
0.973
0.777
0.578
0.762
0.756
0.323
0.611
0.803
0.942
0.772
0.757
0.781
(Drrs)
0.965
0,909
0873
0.885
0.766
Table 2. Benchmarking Results of Cross-method Evaluations in terms of AP Performance on the aaieersllFaeDDeted Dataset. indicates that the results are obtained by using the official pre- pre-traine model reproduction.
Methods
Pro- GAN
Cycle- GAN
Big- GAN
GAN
Style- GAN
Gan- GAN
Star- GAN
Deep Sakes
SITD
Low level
SAN
Parceptual looss
CRN
IMLE
Grided
200
1.DM
widlg
230
steps
100
100
27
Cliile
50
27
100
10
Dalle
mAP
CNN Spot (Wang et al., 2020h)
100,0
93:47
84.50
99.54
$9.49
98.11 15
$9.00
73.75
59.
47
98.24
98:40
73.72
70.62
71.00
70.54
80.65
$4.91
$2.07
70.59
83.58
Patchfor (Chii al., 2000)
80.63
72.84
71.65
83.75
65.99
69.33 25
76.55
76.19
76. 34
74.52
68.52
75.03
67.00 10.10
86.
86.40
85.37
83.73
78.38
75.67
77:73
Checcmmnce (Natsraj ctal. 2019)
90.74
$0.95
50.61
98.63
53.11
67.99
99.14
68.95
60.42
73.06
87:31
20.20
91.31
89.00
92.39
89.32
98.35
$2.79
50.96
75.11
Heg-sppe (Zhrag
55.39
75.05
100.
47.46
57.12
53.61
50.98
57.72
77.72
77.25
68.58
66.58
66.21
FiNer eQ 2020)
99.96
84.32
69.90
99.73
56.71
100.0
78.82
$2.89
46 70
63.
64.37
20.53
73.76
81.66
74.62
89.81
91.04
90.86
71.84
76
UniD (Ojhs at 2023)
100.0
98.13
94.45
86.66
09.25
09.53
91.67
78.54
67:54
83.12
91.06
70.2
95.81
79.77
95.93
93.93
95.12
94.59
$8.45
90.14
LGiad] (Tan et ol., 2023)
100.0
93.98
90.69
99.86
79.36
99.58
67.91
90.42
51.42
63.52
69.61
87.06
59.003
99.16
99.18
95.23
95.10
94.93
97.23
86.35
HeggNett ('la
2014bb
99.92
95.11
74.59
10.10
75.70
96.27
98.00
99:20
99.78
91.95
NPR) (Tan et ol., 2024c)
100.0
99.53
94.53
99.94
$8.92
100.0
84.41
97.95
99.99
50.16
50.10
98. 36
99.92
99.01
99.92
99.87
99
09.92
99.36
92.76
Rutoomerr (Lau et al., 20144)
100.0
100.0
99.95
99.75
100.0
10000
97.99
97.94
80.21
99.84
99.93
93.99
99.81
09.09
99.87
99.13
99.41
99.30
99.82
96.16
Effort (Ours)
100,0
100.0
999.99
99.71
100.0
100.0
98.99
97.53
97.53
100,0
100.0
95.59 3.39
99.
99.88 89
100.0
99.87
99.92
99,69
99.96
99.41
Table 3. Benchmarking Results of Cross-method Evaluations in terms of Ace Performance on the UniversalFakeDetec Dataset. f indicates that the results are obtained by using the official pre-trained model reproduction.
GAN
Low level
Percrptual loos
EDMM
Gliie
Methods
Pro- GAN
Cycle- GAN
Big- GAN
Style- GAN
Grza- GINN
Star- GAN
Deep Tales
SITD
SAN
CRN
IMLE
Guiddd
steps
200
willg
200
steps
100)
1000
27
27
50
100
10
Dallee
mAce
CNN-Sppo (Wang et 20700)
99.09
85.20
70:20
$5.70
78.95
91.20
$3.47
66.67
48.60
$6.
96.36
60.07
54.03
54.96
60.78
63.90
65.65
69.58
Pachffr (Chai et 2020;
75:00
68.97
65.47
79.16
64.23
63:94
75.54
75.04
75.28
72.33
55.30
67:41
76.50
76.
75.77
74.81
73.28
68. 53
67.91
71.24
Coeccunenee (Natraj etal. 2019)
97.70
63:15
53:75
92.50
51.10
54.20
57.10
63.06
55.85
65.55 15.65
65.80
60.50
70.70
70.55
71.00
70.25
69.50
69.90
67.55
66.86
Regg-ppe (Zhang
99.90
$0.50
$0.30
99.20
50.10
50.90
90.40
50
40
55.45
F3Nast (Qian et 20250)
99.38
76.38
05:33
92.56
55.10
100.0
63.48
54.17
47.36
51.47
51.47
69.21
08.15
75.35
68.80
$1.65
53.25
63.05
66.30
71.33
UniFD (Oih et 2033)
100.0
98.50
94.50
$0.00
90:50
97.00
66.60
63.00
57.50
59. 00
72.00
30.03
94.19
73.76
94.36
79.07
79.85
74.14
96.78
$1.38
LGrra]] (Tan et ol., 2023)
99.54
85.39
82.88
94.83
72:45
99.62
58.00
62.50
50.00
50.74
50.78
77.50
94.20
95.85
94.80
$7.40
90.70
89.55
63.35
$1128
RegNari (Tla al., 20144)
95.354
90.45
93.41
71.92
711
13.58
97.40
88.15
35.09
NPR) (lan et all, 2024c)
99.54
95.00
87.55
96.23
86.57
99.75
76.39
66.94
98.63
50.00
50.00
94.35
97.65
98.00
98.20
96.25
97.35
97.35
57.15
57.56
FutFomeri (Lin et al., 2004)
99.80
99.32
09.50
97.15
99.41
99.75
93.23
31.11
68.04
69.45
60.45
76.00
98.60
94.90
98.65
94.35
94.65
94.20
98.75
90.86
Effort (Ours)
100.0
99.85
99:60
98.05
99.60
100.0
87.60
92.50
$1.50
98.90
98.90
69.15
99.30
96.80
99.45
97.45
97.80
97.15
98.05
95.19
codebases of DeepfakeBench (Yan et al., 2023b). Addition- ally, we sample 8 frames from each video for training and 32 frames for inference, following (Shiohara & Yamasaki, 2022). We use the fixed learning rate of 2e-4 for training our approach and employ the Adam (Kingma & Ba, 2014) for optimization. We set the batch size to 32 for both training and testing. We also employ several widely used data aug- mentations, such as Gaussian Blur and Image Compression, following other existing works ( (Yan et al., 2024a; Shiohara & Yamasaki, 2022; Cheng et al., 2024). For the evaluation metric. we report the widely-used video-level Area Under the Curve (AUC) to compare our approach with other works, following (Lin et al., 2024; Shiohara & Yamasaki, 2022). We compute the average model's output probabilities of each video to obtain the video-level AUC.
Evaluation Protocols and Dataset. We adopt two widely used and standard protocols for evaluation: Protocol- 1: cross-dataset evaluation and Protocol-2: cross- manipulation evaluation within the FF++ data domain. For Protocol-1, we conduct evaluations by training the mod- els on FaceForensicst+ (FF++) (Rossler et al., 2019) and testing them on other seven deepfake detection datasets: Celeb- (CDF-v2) (Li et al., 2020b), DeepfakeDetec- tion (DFD) (DFD., 2020), Deepfake Detection Challenge (DFDC) (detection challenge.. 2020), the preview version of DFDC (DFDCP) (Dolhansky et al., 2019), DeeperForensics (DFo) (Jiang et al., 2020), WildDeepfake (WDF) (Zi et al., 2020), and FFIW (Zhou et al., 2021). Note that FF++ has three different compression versions and we adopt the c23 version for training all methods in our experiments, follow-
6

---

Orthogonal Subspace Decompasition for Generalizable AI-Generated Image Detection
ing most existing works (Yan et al., 2024a). For Protocol- 2, we evaluate the models on the latest deepfake dataset DF40 (Yan et al., 2024b), which contains the forgery data generated within the FF++ domain, ensuring the fake meth- ods different while the data domains remain unchanged.
Table 4. Ablation studies regarding the proposed SVD, singular value constraint (Cx.v), ), and orthogonal constraint (Corth). All models are trained on FF++ (c23) and tested on other datasets.
Ours
CDF-v2
SimSwap
Avg.
SVD
Clar
Carta
Evaluation Benchmarking, To provide a comprehensive benchmark for comparison, we introduce 13 competitive de- fectors, including several classical detection methods such as F3Net (Qian et al., 2020) 20), SPSI. (Liu et al., 2021a) (CVPR'20) 20), SRM (Liu et al., 2021a) (CVPR'21), CORE (Ni et al., 2022) (CVPRW"222 22), RECCE (Cao al.,
x
x
x
0.857
0.860
0.859
x
x
0.940
0.910
0.925
x 0.944
0.936
0.945
0,914
0.930
0.956
0.926
0.941
Table 5. Ablation studies regarding different vision foundation models (VFMs) were used. All models are trained on FF++ (c23) and tested on CDF-v2 and SimSwap.
2022) (CVPR' 22), and SBI (Shiohara & Yamasaki, 2022)
(CVPR'22), and also several latest SOTA methods (af- ter 2023), such as UCF (Yan et al., 2023a) (IC IID (Huang et al., 2023) (CVPR'23) 23)). TALL (Xu et al., 2023) (ICCV'23), LSDA (Yan et al., 2024a) (CVPR'24), ProDet (Cheng et al., 2024) (NeurIPS'24) 24), and CFDA (Lin et al., 2024) (ECCV'24). All detectors are trained on FF++ (c23) and tested on other fake data. Results in Tab. demon- strate two notable advantages of our approach. (1) gen- eralizability: we see that our approach consistently and largely outperforms other models across basically all tested scenarios, validating that our method is generalizable for detecting unseen fake data, even for the latest swappingg techniques such as BleFace (Shiohara et al., 2023). (2) effi- ciency: it is worth noting that our method only needs 0.19M parameters for training to achieve superior generalization. As we can see most latest SOTA detectors such as LSDA and ProDet all use about 100M parameters for training, while we are about 1,000x smaller.
VIMs
#Parass
#lmgSize
CDF-v2
SimSwap
Avg.
BEIT=2 (Peng et al., 2022)
303M
224
0.855
0.821
0.838
Ouns
0.14M
224
0.894
0.850
0.872
SigLIP (Zhai et al., 2023)
316M
256
0.877
0.713
0.795
4 Qurs
0.19M
256
0.895
0.778
0.867
CLIP (Radford et al., 2021)
307M
224
0.857
0.860
0.859
+ Ours
0.19M
224
0,956
0,926
0,941
Ba, 2014) with fixed learning rate of 2e-4. The batch size is set to 48. Other settings and details are the same with (Ojha et al., 2023).
Evaluation Analysis. The AP and Acc results are pre- sented in Tab. 2 and Tab. 3, respectively. Our method attains impressive detection results, achieving 99% mAce and 99.41 % mAP across the 19 test subsets. One similar ap- proach to ours is UniFD, which also preserves the original pre-trained knowledge of CLIP and fine-tunes only the FC layer for discrimination. In contrast to UniFD which di- rectly performs discrimination in the pre-trained knowledge space, our approach utilizes SVD to create an orthogonal low-ranked subspace for learning forgeries while preserving the essential high-ranked representational space, achieving the discrimination by leveraging both representational and forgery subspaces, achieving a significant improvement of 9.27% in mAcc and 13.81% in mAp over UniFD. Besides, when compared to the SOTA method, FatFormer, we achieve 4.33% mAcc improvement without relying on the extra text encoder of CLIP. This further demonstrates the superiority of our approach.
4.2. Synthetic Image Detection
Evaluation Metrics. We follow existing works (Wang et al., 2020a; Ojha et al., 2023; Liu et al., 2024) for bench- marking and report both average precision (AP) and classi- fication accuracy (Acc). For Acc, we set the classification threshold for each dataset to 0.5 to ensure fair comparison. UniversalFakeDDteer Dateset. We adhere to the protocol outlined in (Wang et al., 2020a; Ojha et al., 2023) and uti- lize ProGAN's real and fake images as our training dataset, which includes 20 subsets of generated images. The evalua- tion set contains 19 subsets derived from different kinds of generative models, including ProGAN (Karras et al., 2018), 8), CycleGAN (Zhu et al., 2017), BigGAN (Brock et al., 2018a), StyleGAN (Karras et al., 2019), GauGAN (Park et al., 2019).
4.3. Ablation Study and Analysis
Incremental improvement of the proposed designs. Ab- lation studies in Tab. 4 demonstrate incremental gains from the proposed SVD method and loss constraints: the base- line (no modules) achieves an average AUC of 0.859, while adding SVD alone boosts performance to 0.925 (+6.6%), underscoring its efficacy in isolating forgery artifacts via orthogonal feature decomposition. Further integrating the singular value constraint (Lxsv) and orthogonal constraint (Corth) refines performance, yielding 0.936 and 0.930 AUC, respectively, with their combined synergy (SVD) + Lasv +
StarGAN (Choi et al., 2018), DeepFakes (Rössler et al., 2019), SITD (Chen et al., 2018), SAN (Dai et al., 2019), CRN (Chen& & Koltun, 2017), IMLE (Li et al., 2019), Guided (guided diffusion mohrri & Nichol, 2021), LDM (latent diffusion model) (Rombach et al., 2022a), Glide (Nichol et al., 2022), and DALLE (Ramesh et al., 2021). Implementation Details. Similar to the setting of deep- fake image detection, we adopt pre- pre-trdined CLIP ViT-L/14 as the backbone and use the Adam optimizer (Kingma &
7

---

Orthogonal Subspace Decompasition for Generalizable AI-Gencrated Image Detection
Number of main principal components (by PCA)
Table Ablation studies on synthetic image defection regard- ing the tunable r values in SVD (Ours) and r values in LORA. All models are trained on ProGAN's images and tested on 19 different generative models' images. "FFT" indicates the full fine-tuning. "LLnear-Prob" indicates fine-tuning FC layer only, where we reproducc the results from UniFD (Ojha et al., 2023).
-33
1/0
-103
160
137
Archs.
mAce
UniFD (Lincar-Prob)
81.02
57
40 CLP
Baseline (FFT)
86.22
(DVB)
CLI (L:1AA)
CLP-7TT)
Variant-1
256
91.42
Figure 6. PCA quantifying feature space information retention, measured by the minimum number of components required to explain 2 90% variance. Our SVD-based fine-tuning preserves 159/160 principal components (retaining 99.3% original variance), while LoRA and FFT exhibit notable degradation, with 14.4% and 64.4% variance loss, respectively. See Tab. 8 for more results.
Variant-2
91,06
LoRA
Variant-3
16
91.89
Variant-4
93.53
Variant-5
93.03
Variant-1
256
92.13
Variant-2
64
93.68
Ours
Variant-3
16
94.45
Variant-4
94.37
Variant-5
95.19
Table 8. Comparison of different fine-tuning methods. We com- pute both the effective rank and mean accuracy for evaluation. Our SVD retains most principal components (notably higher effective rank), thus achieving the best detection results.
Table? 7. Cross-dataset generalization evaluations with existing adapter-based deepfake detectors. The results are cited from their original papers. The metric is frame-level AUC.
Face Domain
General Domain
tine-tuning Mechods
Effective Rank
Accuracy
Effective Kank
Accuracy
Methods
CDF-v2
DFD
DFDC
Avg.
Baseline (No
160
479
LoRA (Kong et al.. 2023)
0,838
0.834
0.717
0.796
SVD3 (ours)
159
95.60
316
95.19
LoRA FFT
137
93.03
MoE- LoRA (Kong et al., 2024)
0.867
0.904
57
83.70
238
86.22
Dual-Adapter (Shao et al., 2023)
0.717
0.727
Ours
0.901
0.923
0.798
0.874
method explicitly constructs two orthognoal subspaces for pre-trained knowledge and forgery using SVD, ensuring the pre-existing pre-trained knowledge will not be distorted, thereby achieving better generalization performance. To verify this, we provide several empirical results in Tab, 7 and Tab. 6. From these results, we see that our pro- posed SVD-based method achieves clearly higher general- ization results than adapter- methods in both deepfake detection and synthetic image detection fields, as our ap- proach explicitly preserves the pre-trained knowledge while learning the forgery patterns. We also use PCA to compute the rank of the feature space, similar Fig. 3. Results in Fig. 6 highlight the superiority of our method, where full- hallmearseetess fine-tuning and LoRA can lead to notable reduction (-103 and -23, respectively) while our method best retains the pre-trained knowledge, maintaining a higher-ranked feature space for better generalization.
1orth) achieving peak performance (0.941 AUC). These results highlight the significant contribution of SVD as the primary driver of performance gains while illustrating the complementary benefits of Leev and Lorth in further refin- ing the generalization performance.
Compatibility with other vision foundation models. By default, we choose CLIP as the vision foundation model (VFM) in our experiments. To validate the generality and versatility of our approach, we conduct an ablation study to apply our method to other VFMs, including BEIT-v2 (Peng et al., 2022) and SigLIP (Zhai et al., 2023). Results in Tab. 5 show that our approach can be seamlessly applied other VFMs to improve the model's generalization performance.
Comparison with Existing Adapter-Based Detectors. Parameter-efffcieet fine-tuning (PEFT) has become popu- lar technique for adapting pre-trained large models to down- stream tasks (Ding et al., 2023). Low-ranked adaptation (LoRA) (Hu et al., 2021) is a widely used approach for PEFT. Previous works (Kong et al., 2023; Liu et al., 2024) employing LoRA in the VFMs have achieved good empiri- cal generalization results for detection. Additionally, (Kong et al., 2024) and (Shao et al., 2023) introduce MoE-LoRA techniques and dual adapters into the deepfake detection fields. However, the existing based methods do not explicitly ensure this orthogonality, still having the po- tential to distort the pre-existing pre-trained knowledge and result in unexpected generalization results. In contrast, our
The naively trained models leverage very limited forgery patterns for discrimination. In our motivation, we ar- gue that conventional training paradigms cause models to over-rely on very limited forgery patterns for discrimina- tion, thereby causing the highly low-ranked and constrained feature space, limiting their expressivity and generalization (Fig. 2 and Fig. 3). To further validate this argument, analyze the discrimination behavior of baseline models (e.g., Xception and CLIP) through decision boundary visu- alization using the model's output logits of real and fake classes (Fig. 7). The linear alignment of predictions along y - + b reveals that Xception collapses real/fake
8

---

Orthogonal Subspace Decompasition for Generalizable AI-Gencrated Image Detection
the optimal choice. Second, the inherent characteristics of binary classification further justify selecting a smaller rank. Binary classification tasks typically do not require the model to learn extensive and nuanced patterns, but rather to identify just enough distinctive features to separate the two classes, making the learned feature space inherently con- strained. Thus, binary classification inherently simplifies the complexity of the learning problem. meaning that employ- ing higher rank would not provide significant additional benefit.
In-Domain Ev Evalatioon
Cross-Forgers Evaluation
Cross Datasct Evaluation
ALIC: 0.999
AECC: 0.708
19pa www.
Aggs for SAL
Vanilia CNN
leght Eo slos
leghn and alow
Lighns
AUC: 0,997
AUC: 0L8K3
AUC
WWW. CA -2
CLIP(FFT)
hight fo LAk
5. Conclusion
In this paper, we start our research from new perspec- tive to excavate the failure reason of the generalization in AIGI detection, namely the asymmetry phenomena, where a naively trained detector very quickly shortcuts to the seen fake patterns, collapsing the feature space into a low-ranked structure that limits expressivity and generalization. To ad- dress this, we propose integrating higher-ranked pre-trained knowledge from vision foundation models to expand the fea- ture space. Simultancously, we decompose the feature space into two orthogonal subspaces, for preserving pre-trained knowledge while learning forgery. Beyond LoRA and full- parameters tuning, we explicitly ensure the orthogonality, maintaining the higher rank of the whole feature space for better generalization. Furthermore, reveal very impor- tant prior for generalizable AIGI detection that fake data actually originates from real data in a hierarchical structure, not independently. Our method leverages this prior by pre- serving pre-trained semantic components while adapting to fake detection, enabling discrimination in comantic-alignen subspaces with reduced model complexity and improved generalization. Extensive experiments with deep analysis on both deepfake and synthetic image detection benchmarks have demonstrated the superior advantages of the proposed method in AIGI detection.
lajia Ex
Segln roll
light
AEC:
AIC:: 0.962
AUC:
Hobu Pe tae dar
Iedil for Like dass
SVD (ours)
Tegls for
kghh bar Cass
Sught for rull Clai
Ingit fer not dass
Figure 7. Evidence for validating the discrimination dimension from the logits space. The first row shows that the vanilla CNN (i.e., Xception) overfits to the seen fakes and relies only on the forgery pattern for discrimination. The second row shows that fully fine-tuning the VFM (i.e.. CLIP) involves pre-trained knowl- edge but also distorting part of during learning forgery patterns. The third row shows that our method uses SVD to achieve the orthogonality, thereby best retaining the pre-trained knowledge.
discrimination into single discriminative dimension, confirming its exclusive dependence on forgery cues. No- tably, while CLIP's original higher-ranked representations initially preserve informative structures, full fine-tuning catastrophically degrades this structure, forcing decisions into a similarly collapsed subspace (y- = -I + b). Our method addresses this concern by learning forgery-related features in a novel orthogonal complement subspace rela- tive CLIP's original representational embedding space, thereby achieving optimal generalization performance.
Impact Statement
This research advances application-driven machine learning by proposing novel method for detecting Al-generated images. By effectively identifying deepfakes and curbing malicious uses of generative models, it holds significant potential for positive social impact. However, a possible negative outcome is the misuse of our method to enhance the realism of deepfake generators. To address this concern, we plan to implement measures like access control. Overall, we urge the research community to minimize negative impacts while leveraging the positive contributions of this work.
How determine the value of rank in tuning SVD? Our choice of using lower rank (specifically, "n-r"=1) for fine-tuning DFD is primarily motivated by two critical factors: First, the nature of the real-fake classification task itself makes it relatively straightforward. Specifi- cally, fake samples in existing training sets tend to exhibit a limited number of distinctive forgery patterns (FF++ con- tains only four forgery types). each with relatively simple and consistent characteristics. Due to this simplicity and limited diversity, low-rank adaptation with small rank (e.g., "n-r"=1, 4, or 16) is sufficient for the model to ef- fectively learn these forgery patterns. As demonstrated by Table 5 in our paper, choosing ranks of 1, 4, or 16 yields very similar performance results. Given this observation, we prioritize efficiency and parameter economy, making rank
Acknowledgment
This work was supported in part by the Natural Science Foundation of China (No. 62202014, 62332002, 62425101).

---

Orthogonal Subspace Decompasition for Generalizable AI-Generated Image Detection
References
DFD., 2020.
https://ai,
googleblog.ccm//22
Wukong, 2022. 5, Inhttps://xibe wiw.hsore.. cn/modelzoo wtw.e///oo//////g.ooo/oo 2022. 5.
contributing waaiing---t----a data-to deepfake html Accessed 2021-04-24.
Brock, A., Donahue, J., and Simonyan, K. Large scale gan training for high fidelity natural image synthesis. arXiv preprint arXiv:1800 1809. 11096, 2018a.
Dhariwal, and Nichol, A. Diffusion models beat gans on image synthesis. In NeurIPS, 2021.
Dhariwal, P. al. Diffusion models beat gans on image synthesis. NeurIPS, 34:8780-8794, 2021.
Brock, A. et al. Large scale gan training for high fidelity natural image synthesis. In ICLR, 2018b.
Ding, N., Qin, Y., Yang, G., Wei, F., Yang. Z., Su, Y., Hu, S., Chen, Y., Chan, C.-M., Chen, W., et al. Parameter- efficient fine-tuning of large-scale pre-trained language models. Nature Machine Intelligence, 5(3):220-235, 2023.
Cao, J., Ma, C., Yao, I., Chen, S., Ding, S., and Yang, X. End-to-end ceattruucttio-ccassicftttte learning for face forgery detection. In CVPR, pp. 41 4113-4122, 2022.
Chai, L., Bau, D., Lim, S.-N., and Isola, What makes fake images detectable? understanding properties that generalize. In ECCV, pp. 103-120. Springer, 2020.
Dolhansky, B., Howes, R., Pflaum, B., Baram, N., and Ferrer, C. C. The deepfake detection challenge (dfdc) preview dataset. arXiv preprint 0ttt:::99/00085...0 2019.
Chen, B., Zeng, J., Yang, J., and Yang, R. Dret: Diffu- sion reconstruction contrastive training towards universal detection of diffusion generated images. In ICML, 2024.
Dong. S., Wang. J., Ji, R., Liang, J., Fan, H., and Ge, Implicit identity leakage: The stumbling block to im- proving deepfake detection generalization. In CVPR, pp. 3994-4004, 2023.
Chen, C., Chen, Q., Xu, and Koltun, V. Learning see in the dark. In CVPR, 2018.
Fu, X., Yan, Z.,, Yao, T., Chen, S., and Li, X. Exploring unbiased deepfake detection via token-level shuffling and mixing. In AAAI, 2025.
Chen, L., Zhang, Y., Song, Y., Liu, L.,, and Wang, J. Self- supervised learning adversarial example: Towards good generalizations for deepfake detection. In CVPR, pp. 18710-18719, 2022a.
Goodfellow, I., Pouget-Abaas J., Mirza, M., Xu, B., Farde- D., Ozair, S., Courville, A., and Bengio, Generative adversarial networks. Communications of the ACM, 63(1 13:9)119914441 2020.
Chen, L.. Zhang, Y., Song, Y.. Wang, J., and Liu, L. Ost: Improving generalization of deepfake detection via one- shot test-time training. In NeurIPS, 2022b.
Gu, S., Chen, D., Bao, J., Wen, F., Zhang, B., Chen, D., Yuan, L., and Guo, B. Vector quantized diffusion model for text-to-image synthesis. In CVPR, pp. 10696-10706. 2022a.
Chen, Q. and Koltun, V. Photographic image synthesis with cascaded refinement networks. In ICCV, 2017.
Cheng, J., Yan, Z., Zhang, Y., Luo, Y., Wang, Z., and Li, C. Can we leave deepfake data behind training deepfake detector? NeurIPS, 2024.
Gu, Z., Yao, T., Chen, Y., Ding, S., and Ma. L. Hierarchi- cal contrastive inconsistency learning for deepfake video detection. In ECCV, pp. 996-613 Springer, 2022b.
Choi, J., Kim, T., Jeong, Y., Baek, S., and Choi, J. Ex- ploiting style latent flows for generalizing deepfake video detection. In CVPR, pp. 1133-1143, 2024.
Guan, J., Zhou, H., Hong. Z., Ding, E., Wang. J., Quan, C., and Zhao, Delving into sequential patches for deepfake detection. NeurlPS, 35:4517-4530. 2022.
Choi, Y., Choi, M., Kim, M., Ha, J.W.., Kim, S., and Choo, J. Stargan: Unified generative adversarial net- works for multi-domain image-to to- to-image translation. In CVPR, 2018.
Gunasekar, S., Woodworth, B. E., Bhojanapalli, S., Neyshabur, B., and Srebro, N. Implicit regularization in matrix factorization. NeurIPS, 30, 2017.
Dai. T., Cai. J.. Zhang, Y.. Xia, S.-T., and Lei, Z. Second- order attention network for single image super-resolution. In CVPR, 2019.
Haliassos, A., Vougioukas, K., Petridis. S., and Pantic, M. Lips don't lie: generalisable and robust approach to face forgery detection. CVPR, 2021.
detection challenge., D., 2020. httts:://www.kagge... com/c/ c/ cete///eppfaa fake detection- chwllonnnnccaeeeeeeeeeeo Ac- cessed 2021-04-24.
Haliassos, A., Mira, R., Petridis, S., and Pantic, M. Lever- aging real talking faces via self-supervision for robust forgery detection. In CVPR, pp. 14950-14962. 2022.
10

---

Orthogonal Subspace Decompasition for Generalizable AI-Generated Image Detection
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In CVPR, pp. 770-778, 2016.
Li, Xie, H., Li, J., Wang. Z., and Zhang, Y. Frequency- aware discriminative feature learning supervised by single-center loss for face forgery detection. In CVPR, 2021.
Ho, J., Jain, A., and Abbeel, P. Denoising diffusion proba- bilistic models. NeurIPS, 53:6840-6853 2020.
Li K., Zhang, T., and Malik, J. Diverse image synthesis from semantic layouts via conditional imle. In ICCV, 2019.
Hu, E. J., Shen, Y. Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106. (06,65,,,,, 2021.
Li, I.., Bao, J., Zhang, T., Yang, H., Chen, D., Wen, F, and Guo, B. Face x-ray for more general face forgery detection. In CVPR, 2020a.
Huang, B., Wang, Z., Yang, J., J., Zou, Q. Wang. Q. and Ye, D. Implicit identity driven deepfake face swapping detection. In CVPR, pp. 4490-4499, 2023.
Li, Y., Yang, X., Sun, P., Qi, H., and Lyu, S. Celeb-df: A new dataset for deepfake forensics. In CVPR, 2020b.
Jeong, Y. et al. Bihpf: Bilateral high-pass filters for robust deepfake detection. In WACV, pp. 48-57, 2022.
Lin, Y., Song, W. Li, B., Li, Y., Ni, J., Chen, H., and Li, Q. Fake it till you make it: Curricular dynamic forgery augmentations towards general deepfake detection. arXiv preprint arXiv:2409. 14444, 2024.
Jiang, L., Li, R., Wu, W., Qian, C., and Loy, C. C. Deeperforensics-1.. A large-scale dataset for real-world face forgery detection. In CVPR, 2020.
Liu, H., Li, X., Zhou, W., Chen, Y., He, Y., Xue, H., Zhang, W., and Yu, N. Spatial-phase shallow learning: rethinking face forgery detection in frequency domain. CVPR, 2021a.
Karras, T., Aila, T., Laine, S., and Lehtinen, Progres- sive growing of gans for improved quality, stability, and variation. In ICLR, 2018.
Karras, T., Laine, S., and Aila, T. A style-bused genera- tor architecture for generative adversarial networks. In CVPR, 2019.
Liu, H., Tan, Z., Tan, C., Wei, Y., Wang. J., and Zhao, Y. Forgery- adaptive transformer for generalizable synthetic image detection. In CVPR, pp. 10770-10780. 2024.
Khalid, H. and Woo, S. S. Oc-fakedect: Classifying deep- fakes using one-class variational autoencoder. In CVPRW, pp. 656-657, 2020.
Liu. Z., Lin, Y., Cao. Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. Swin transformer: Hierarchical vision transformer using shifted windows. In ICCV, pp. 10012- 10022, 2021b
Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1xXi::442.66980 2014.
Kong, C., Li, H., and Wang, S. Enhancing general face forgery detection via vision transformer with low-rank adaptation. In ICCV, pp. 102-107. IEEE, 2023.
Liu, Z. et al. Global texture enhancement for fake face detection in the wild. In CVPR, pp. 8060-8069, 2020.
Luo, A., Cai, R., Kong, C., Kang, X., Huang, J., and Kot, A. C. Forgery-aware adaptive vision transformer for face forgery detection. arXiv preprint 0xXiv22309.1199. 2023a.
Kong. C., Luo, A., Bao, P., Yu, Y., Li, H., Zheng, Z., Wang, Moe-ffd: Mixture of experts for gen-
S., and Kot, eralized and parameter-efffcient face forgery detection. arXiv preprint arXiv:2404.084 2024.
Luo, A., Kong, C., Huang, J., Hu, Y., Kang, X., and Kot, A. C. Beyond the prior forgery knowledge: Mining criti- cal clues for general face forgery detection. IEEE TIFS, 19: 168-118., 2023b.
Korshunov, P. and Marcel, S. Deepfakes: a new threat to face recognition? assessment and detection. arXiv preprint arXiv: 180.00885. 2018.
Larue, N., Vu, N.-S., Struc, V., Peer, P., and Christophides, V. Secable: Soft discrepancies and bounded contrastive learning for exposing deepfakes. In ICCV, pp. 21011- 21021, 2023.
Luo, Y., Zhang, Y., Yan, J., and Liu, W. Generalizing face forgery detection with high high-frequency features. In CVPR, 2021.
Li, D., Yang, Y., Song. Y.-Z., and Hospedales, T. Learning to generalize: Meta-learning for domain generalization. In AAAI, volume 32, 2018.
Luo, Y., Du, J., Yan, K., and Ding, Lare2: Latent re- construction error based method for diffusion diffusion-generated image detection. In CVPR, pp. 17006-17015 2024.

---

Orthogonal Subspace Decompasition for Generalizable AI-Gencrated Image Detection
Miao, C., Tan, Z., Chu, Q., Liu, H., Hu, H., and Yu, N. F2 trans: High-frequency fine-grained transformer for face forgery detection. IEEE TIFS, 18: 1039-10555,, 2023.
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffusion models. In CVPR, 2022a.
MidJoumny. home.
http:///wwwwiidfoooeee com/
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffusion models. In CVPR, pp. 10684-10695, 2022b.
Mohri, M. and Rostamizadch, A. Rademacher complex- ity bounds for non-iid processes. Advances in neural information processing systems, 21, 2008.
Rössler, A.., Cozzolino, D., Verdoliva, L., Riess, C., Thics, J., and Nießner, M. FaceForensicst+: Learning to detect manipulated facial images. In ICCV, 2019.
Nataraj, L., Mohammed, T. M., Chandrasekaran, S., Flen- ner, A., Bappy, J. H., Koy-Chowdhury A. K., and Manju- nath, B. Detecting gan generated fake images using co- occurrence matrices. arXiv preprint arXiv:/903.066: 06.66, 2019.
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., and Nießner, M. Faceforensics*#: Learning detect manipulated facial images. In ICCV, pp. 1,,1,9009
Shao, R., Wu, T., Nie, L., and Liu, Z. Deepfake-adapter: Dual-level adapter for deepfake detection. arXiv preprint arXiv:2306.0053 2023.
Ni, Y., Meng, D., Yu, C., Quan, C., Ren, D., and Zhao, Y. Core: Consistent representation learning for face forgery detection. In CVPRW, pp. 12-21, 2022.
Shiohara, K. and Yamasaki, T. Detecting deepfakes with self-blended images. In CVPR, pp. 18720-18729, 2022.
Nichol, A., Dhariwal, P., Ramesh, A., Shyam, P., Mishkin, P. McGrew, B., Sutskever, I., and Chen, M. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2112.107411 2021.
Shiohara, K., Yang, X., and Taketomi, T. Blendface: Re- designing identity encoders for face-swapping. In ICCV, pp. 7634-7644, 2023.
Sun, K., Liu, H., Ye, Q. Gao. Y., Liu, J., Shao, L., and Ji, R. Domain general face forgery detection by learning to weight. In AAAI, volume 35, pp. 2638-2646, 2021.
Nichol, A., Dhariwal, P., Ramesh, A., Shyam, P., Mishkin, P., McGrew, B., Sutskever, I., and Chen, M. Glide: To- wards photorealistic image generation and editing with text-guided diffusion models. In ICML, 2022.
Sun, K., Yao, T., Chen, S., Ding, S., Li, J., and Ji, Dual contrastive learning for general face forgery detection. In AAAI, volume 36, pp. 2316-2324, 2022.
Ojha, U. et al. Towards universal fake image detectors that generalize across generative models. In CVPR, pp. 24480-24489, 2023.
Tan, C., Zhao, Y., Wei, S., Gu, G., and Wei, Y. Learning on gradients: Generalized artifacts representation for gan- generated images detection. In CVPR, pp. 12105-12114, June 2023.
Park, T., Liu, M.-Y., Wang, T.-C., and Zhu, J.-Y. Semantic image synthesis with spatially-adaptive normalization. In CVPR, 2019.
Tan, C., Liu, P., Tao, R., Liu, H., Zhao, Y., Wu, B., and Wei, Y. Data-independent operator: A training feieinn-ffree free arti- fact representation extractor for generalizable deepfake detection. arXiv preprint arXiv:2403.06880 2024a.
Peng, Z., Dong, L., Bao, H., Ye, Q., and Wei, F. Beit v2: Masked image modeling with vector-quantize visual tokenizers. arXiv preprint 2rXiv:2208.06638 2022.
Qian, Y., Yin, G., Sheng, L., Chen, Z., and Shao, J. Thinking in frequency: Face forgery detection by mining frequency- aware clues. In ECCV, pp. 86-103. Springer, 2020.
Tan, C., Zhao, Y., Wei, S., Gu, G., Liu, P., and Wei, Y. Frequency-awaid deepfake detection: Improving gener- alizability through frequency space domain learning. In AAAI, volume 38, pp. 5052-5060, 2024b.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. Learning transferable visual models from natural language supervision. In ICML., pp. 5748-8763. PMLR, 2021.
Tan, C., Zhao, Y., Wei, S., Gu, G., Liu, P., and Wei, Rethinking the up- up-sampling operations in enn-based gen- erative network for generalizable deepfake detection. In CVPR, pp. 281 100-28139, 2024c.
Ramesh, A., Pavloy, M., Goh, G., Gray, S., Voss, C., Rad- ford, A., Chen, M., and Sutskever, I. Zero-shot text-to- image generation. In /CML., 2021.
Tan, M. and Le, Q. Efficientnet: Rethinking model scaling for convolutional neural networks. In ICML, pp. 6105- 61 14. PMLR, 2019.
12

---

Orthogonal Subspace Decompasition for Generalizable AI-Generated Image Detection
Thies, J., Zollhofer, M., Stamminger, M., Theobalt, C., and Nießner, M. Face2face: Real-time face capture and reenactment of rgb videos. In CVPR, 2016.
Yan, Z., Ye, Li, W., Huang, Z.. Yuan, S., He, X., Lin, K., He, J., He, C., and Yuan, L.. Gpt-imgeval: A comprehen- sive benchmark for diagnosing gpt4o in image generation. arXiv preprint arXiv:2504.02780 2025.
Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., and Jégou, H. Training data-efficient image trans- formers & distillation through attention. In ICML, pp. 10347-10357. PMLR, 2021.
Zhai, X.,
Mustafa, B., Kolesnikov, A., and Beyer, L. Sig- moid loss for language image pre-training. In ICCV, pp. 11975-11986, 2023.
Trinh, L. and Liu, Y. An examination of fairness of ai models for deepfake detection. arXiv, 2021.
Zhang, D., Xiao, Z.., Li, S., Lin, F., Li, J., and Ge, S. Learn- ing natural consistency representation for face forgery video detection. arXiv preprint arXiv:2407.10550 2024.
Wang, J., Wu, Z., Ouyang, W., Han, X., Chen, J., Jiang, Y.-G., and Li, S.-N. M2tr: Multi-modal multi-scale trans- formers for deepfake detection. In ICMR, pp. 615-623, 2022.
Zhang, X., Karaman, S., and Chang, S.-F. Detecting and simulating artifacts gan fake images. arXiv preprint vrXiv:19070066 2019.
Wang, S.-Y., Wang, O., Zhang, R., Owens, A., and Efros, A. A. Cnn-generated images surprisingly easy to spot...for now. In CVPR, 2020a.
Zhao, T., Xu, X., Xu, M.. Ding. H., Xiong, Y., and Xia, W. Learning self- ceff-consistency for deepfake detection. In ICCV, 2021.
Wang, S.-Y., Wang, O., Zhang, R., Owens, A., and Efros,
Zheng, Y., Bao, J., Chen, D., Zeng, M., and Wen, F. Ex- ploring temporal coherence for more general video face forgery detection. In ICCV, pp. 15044-15054, 2021.
A. A. Cnn-generates images are surprisingly easy to spot... for now. In CVPR, pp. 8695-8704, 2020b.
Wang. Z., Bao, J., Zhou, W., Wang, W., Hu, H., Chen, H., and Li, H. Dire for diffusion--gneratee image detection. In ICCV, pp.
Zhou, T., Wang, W., Liang, Z., and Shen, J. Face forensics in the wild. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5778-5788, 2021.
2023a.
Wang, Z., Bao, J., Zhou, W., Wang, W., and Li. H. Alt- freezing for more general video face forgery detection. In CVPR, pp. 4 4129-4138, 2023b.
Zhu, J.-Y., Park, T., Isola, P., and Efros, A. A. Unpaired image-to-image translation using cycle-consistent adver- sarial networks. In ICCV, 2017.
Wu, H., Zhou, J., and Zhang, S. Generalizable synthetic image detection via language-guider contrastive learning. arXiv preprint ar 2r 2023.
Zhu, M., Chen, H., Yan, Q., Huang, X., Lin, G., Li, W., Tu, Z., Hu, H., Hu, J., and Wang, Y. Genimage: A million-scale benchmark for detecting ai-generated image. NeurIPS, 36, 2024.
Xu, Y., Liang, J., Jia, G., Yang. Z., Zhang, Y., and He, R. Tall: Thumbnail layout for deepfake video detection. In ICCV, pp. 0668-22668,, 2023.
Zhu, X., Wang, H., Fei, H., Lei. Z., and Li, S. Z. Face forgery detection by 3d decomposition. In CVPR, 2021.
Yan, Z., Zhang, Y., Fan, Y., and Wu, B. Ucf: Uncovering common features for generalizable deepfake detection. arXiv preprint arXiv:2304. 13949, 2023a.
Zi, B., Chang, M., Chen, J., Ma, X., and Jiang, Y.GG. Wild- deepfake: A challenging real-world dataset for deepfake detection. In ACM MM, pp. 2382-33222222222 2890, 2020.
Yan, Z., Zhang, Y., Yuan, X., Lyu, S., and Wu, B. Deep- fakebench: A comprehensive benchmark of deepfake de- tection. In Oh, A., Neumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), NewrIPS, volume 36, pp. 4534 4565. Curran Associates, Inc., 2023b.
Yan, Z., Luo, Y., Lyu, S., Liu, Q., and Wu, B. Transcending forgery specificity with latent space augmentation for generalizable deepfake detection. In CVPR, pp. 8984- 8994, 2024a.
Yan, Z., Yao, T., Chen, S., Zhao, Y., Fu, X., Zhu, J., Luo, D., Yuan, L., Wang, C., Ding, S., et al. Df40: To- ward gextreenerttion deepfake detection. arXiv preprint arXiv:2406. 13495, 2024b.
13

---

Orthogonal Subspace Decompasition for Generalizable AI-Gencrated Image Detection
A. Additional Results and Ablations
the superior generalization performance of our approach.
In this section, we provide additional experimental results and ablation studies of our proposed approach.
A.3. Robustness Evaluation
To evaluate our model's robustness to random perturbations, we adopt the methodology used in previous studies (Halias- sos et al., 2021; Zheng et al., 2021), which involves ex- amining three distinct types of degradation: Biock-wise distortion, Change contrast, and JPEG compression. We apply each of these perturbations at five different levels to assess the model's robustness under varying degrees of dis- tortion, following (Chen et al., 2022b; Yan et al., 2024a). The video-level AUC results for these unseen perturbations, using the model trained on FF++ (c23), are depicted in Fig. 8. Generally, our approach shows higher results than other methods, demonstrating the better robustness of our approach than other models.
A.I. Results on GenImage Benchmark
In our manuscript, we present the benchmarking outcomes of the UniversalFakeDettet Dataset. Additionally, we report the results obtained from another widely utilized bench- mark known GenImage (Zhu al., 2024). This Gen- Image dataset predominantly utilizes the Diffusion model for image generation, incorporating models such as Mid- journey (MidJourney.), SDv1.4 (Rombach et al., 2022b), SDv1.5 (Rombach et al., 2022b), ADM (Dhariwal et al., 2021), GLIDE (Nichol et al., 2021). Wukong (Wuk, 2022. 5), VQDM (Gu et al., 2022a), and BigGAN (Brock al., 2018b). Following the settings defined for Genlmage, we
designate SDv1.4 as the training set and the remaining mod- els as the test set. Given the diverse image sizes within the
B. Additional Cross-Method Evaluation
GenImage dataset, images with a size smaller than 224 pix- els are duplicated and subsequently cropped to 224 pixels, following (Tan et al., 2024c). We employ the same setting to re-implement FreqNet, FatFormer, and NPR, and also report the results of UnivFD and DRCT from (Chen al., 2024).
The capability of face forgery detectors to generalize new manipulation methods is crucial in practical, real-world ap- plications. In our manuscript, we present cross-method eval- uations using the DF40 dataset (Yan et al., 2024b). Specifi- cally, we train the models with four manipulation methods from FF++ (c23) and then test them on the other eight ma- nipulation techniques provided in DF40. Furthermore, we conduct an additional cross-method evaluation following the protocol introduced in (Sun et al., 2022; Miao et al., 2023; Luo et al., 2023a). This protocol involves training the model on diverse manipulation types of samples and subsequently testing it on unknown manipulation methods. The results of this evaluation are reported in Tab. 11. is evident that our proposed method attains remarkable performance in cross- pross-manipulation evaluation. In terms of accuracy (ACC), it outperforms the latest SOTA detector FA-ViT by 2.85% GID-DF and 3.14% on GID-F2F, respectively.
The results on the GenImage dataset are presented in Table 9. When SDv 1.4 is employed as the training set, our method attains an overall accuracy rate of 91.1% % across the entire test set. Compared to similar methods that utilize CLIP as the backbone, such as UnivFD and FatFormer, our approach improves accuracy by 1 11.6% and 2.2%, respectively. More- over, when contrasted with the latest state-of-the-an (SOTA) method DRCT (ICML 2024), the proposed method achieves a 1.6% enhancement in accuracy. This clearly indicates that our method demonstrates superior generalization capa- bilities and achieves SOTA performance on the GenImage benchmark.
B. Additional Ablation Studies
A.2. Comparison with Existing Video Detectors
Impact of Different Vision Foundation Models We ini- tialize the VIT backbone with several widely used pre- trained weights from different vision foundation models, including BEIT-v2 (Peng et al., 2022), CLIP (Radford et al., 2021), and SigL.P (Zhai et al., 2023). The results are shown in Tab. 12 and Tab. 13. It is evident that our proposed approach improves the generalization performance of dif- ferent pre-trained ViTs. On the other hand, we note that different initialization significantly impacts generalization performance, indicating the importance of choosing suit- able pre- pre-trained initialization. Through empirical results, we discover that the ViT pre-trained on CLIP exhibits the highest performance in both deepfake detection and syn- thetic image detection tasks. Therefore, we choose CLIP as the default setting for our approach.
In the manuscript, we mainly compare our method with image detectors. Here, we provide an individual result to compare our approach with existing SOTA video detectors. Following (Zhang et al., 2024; Xu et al., 2023), we conduct evaluations on the widely-used CDF-v2 (Li et al., 2020b) and DFDC (detection challenge.. 2020) using the video- level AUC metric. We have considered both the classical detectors such as LipForensics and the latest SOTA detectors such as NACO (ECCV'24) for a comprehensive comparison. Results in Tab. 10 demonstrate that our image-based ap- proach achieves higher generalization performance in both CDF-v2 and DFDC, improving 4.5% and 3.9% points than the second-best video-basch models. This further validates
14

---

Orthogonal Subspace Decomposition for Generalizable AI-Generated Image Detection
Table Benchmarking results cross-method evaluations in terms Ace performance on the Genimage dataset. We follow (Zhu et al., 2024) and use the SDv1.4 as the training set while others as the testing sets. We directly cite the results of ResNet-50, DeiT-S, Sw CNNSpot, Spec, F3Net, GramNet (Zhu al., 2024). We obtain the results of UnivFD and DRCT from (Chen al., 2024), and Net NPR, and FatFormer by using the official checkpoints for reproduction. We report the Accuracy metric for comparison following (Chen al., 2024).
Methods
Venues
Midjourney
SDv1.4
SDvI.5
ADM
GLIDE
Wukong
VQDM
BigGAN
mAce
ResNect-50 (He al., 2016)
CVPR 2016
54,9
99.9
99.7
53.5
61.9
98.2
56.6
52.0
72.1
DeñT-S (Touvron ul., 2021)
ICML 2021
55.6
99.9
99.8
49.8
58.1
98.9
56.9
53.5
71.6
Swin-T (Liu al., 2021b)
ICCV 2021
62.1
99.9
99.8
49.8
67.6
99.1
62.3
57.6
74.8
CNNSpot (Wang et al. 2020b)
CVPR 2020
$2.8
96.3
95.9
50.1
39.8
78.6
53.4
46.8
64.2
Spec (Zhang et al., 2019)
WIFS 2019
52.0
99.4
99.2
49.7
49.8
94.8
53.6
49.8
68.8
F3Net (Qia et al., 2020)
ECCV 2020
50.1
99.9
99.9
49.9
50.0
99.9
49.9
49.9
68.7
GramNet (1.iu et al., 2020)
CVPR 2020
99.2
99.1
50.3
54.6
98.9
51.7
69.9
UnivFD (Ojha et al., 2023)
CVPR 2023
91.5
96.4
96.1
73.4
94.5
57.7
79.5
NPR (Tan et al.. 2024c)
CVPR 2024
98.2
97.9
76.9
89.8
96.9
88.6
FreyNet (Tan al., 2024b)
AAAI 2024
98.6
86.5
97.3
FatFormer (Liu
2024)
CVPR 2024
100.0
88.0
55.8
88.9
DRCT (Chen 2024)
ICML 2024
95.0
94.4
89.2
89.5
Ours
82.4
99.8
99.8
78.7
93.3
97.4
91.7
77.6
91.1
Block Wise
Change Contrast
JPEG Compression
Average
20
AUIC (%6
80
AUIC (%6
20
AUAC 0%6
00
AUI (%)
80
70
TWA
Face X-nay
Patehfiorensiis
FTCN
Upforeesice
Figure Robustness to unseen perturbations. We present video-level AUC five distinct degradation levels across three types of perturbations in (Jiang et al., 2020).
Before Training (initial feature space)
(a). StyleGAN
(b). CyeleGAN
Semantic
Semantic
Semantic
Semantic
Bedboon
Car
Apple
Bedroo
Summer
Figure 9. t-SNE visualizations of the initial latent feature spaces between vanilla CNN (Wang et al., 2020b) and CLIP (Radford al., 2021). We show that both the pre-trained CNN and CLIP identify different semantic objects.
Impact of Different ViT Backbones Here, we investi- gate the effects of different ViT architectures. Specifically, we consider two backbones that were implemented in the original paper of CL.PP: ViT-Base-16 16 and ViT-Large-1 14. We conduct evaluations both deepfake detection and syn- thetic image detection benchmarks. as shown in Tab. 14 and
Tab. 15. Compared to fully fine-tuning the CLIP model, our proposed approach consistently demonstrates substantial enhancements generalization performance across these VIT backbones. It is worth noting that CLIP--Large performs better than CLIP-Bas by notable margin. Based this ablation experiment, ultimately choose ViT-Large as our
15

---

Orthogonal Subspace Decompasition for Generalizable AI-Gencrated Image Detection
Seen Fake (ProGAN)
Semantic
Unseen Fake (StyleGAN)
Semontic
Forgery
Forgery
Ferpery
Forgery
Vanilia CNN (Only Forgery Space)
Ours (Semantlc and Fargery
Vanilla CNN (Oal Furgery Space)
Own (Semantic and Fargery Space) Acc: 99,77
Acc: 99.99
Acc: 100
Acc: 85.75
Real:
Car
Hors
Dog
Real:
Fake:
Home
Pake:
Figure 10. t-SNE visualizations of the latent feature spaces between vanilla CNN (Wang et al., 2020b) and ours. We use the testing set ProGAN and StyleGAN within UniversalFakeDeted Dataset (Wang et al., 2020b) for visualization. see that after fine-tuning on AIGI data, the baseline quickly shortcuts to the fake and "forgets" the pre-existing semantic knowledge, thereby resulting in highly constrained feature space.
Table 12. Ablation studies on deepfake image detection (Cross- dataset) regarding different vision foundation models (VFMs)
Table 10. Cross-dataset generalization evaluations with exist- ing SOTA video detectors. The results other detectors are directly cited from their original papers. The metric is video-level AUC.
were used. All models are trained on FF++ (c23) and tested on
CDF-v2 and SimS SimSwap.
VFMs
#Panams
#lmgSize
CDFvv2
SimSwap
Avg.
Methods
Venues
CDF-v2
DFDC
Avg.
BERT-v2 (Pong al., 2022)
303M
224
0.855
0.821
0.838
Oars
0.14M
224
0.894
0.850
0.872
Lipforemsics (Haliissoos al., 2021)
2021
0.780
IP (Zhai et al., 2023)
316M
0.795
FTCN (Zheng et al.. 2021)
ICCV 2021
0.869
0,740
0.805
Oarss
0.19M
256
0.895
0,778
0.867
HCIL (Gu et al., 20226)
BCCV 2022
0.790
0.692
0.741
CLIP (Radford al., 2021)
307M
224
0.857
0.860
0.859
RealForensies (Haliassos al., 2022)
CVPR 2022
0.857
0,759
0.808
Ours
0.19M
0.941
LITD al.,
NeudPS 2022
0,849
AltFreezing (Wang et al.,, 2023b)
CVPR 2023
0.895
TALL-Swin (Xu et al., 2023)
ICCV 2023
0.908
0.768
0.838
Table 13. Ablation studies on syuthetic image detection (Univer- salFake Dataset) regarding different vision foundation models (VFMs) were used, All models are trained on ProGAN's s images and tested on 19 different generative models' images.
StyleDFD (Choi et al., 2024)
CVPR 2024
0.890
NACO (Ziang al., 2024)
BCCV 2024
0.831
Ours
0.956
0.843
0.900
VFMs
#Params
#ingSize
mAP
mAce
BETT-V (Peng et al., 2022)
203M
224
93.50
79.11
0.14M
224
97.39
83.66
SigLIP (Zhai et al., 2023)
316M
256
94.30
81.23
Table 11. Additional results of cross-manipulation evaluation on FF++ (c23). Following (Miao al., 2023; Luo al., 2023a), we conduct evaluations by training on the other three manipulated methods while testing on the remaining one. Specifically, GID- DF means training on the other three manipulated methods (FF- F2F, FF-FS, FF-NT) while testing on the FF-DF. Results of other methods are cited from (Miao et al., 2023; Luo et al., 2023a).
+ Ours
0.19M
256
96.24
90.46
CLIP (Radford et al., 2021)
307M
224
97.95
86.22
Ours
0.19M
224
99.41
95.19
default backbone.
GID-DF
GID-F2F
C. Additional Analysis and Visualizations
Methods
AUC
EfficientNet (Tan 2019)
82.40
91.11
63.32
80.10
C.1. Additional t-SNE Visualizations
MLGD (Li al., 2018)
84.21
91.82
63.46
77.10
LTW (Sun al., 2021)
85.60
92.70
65.60
80.20
DCL (Sun al. 2022)
87.70
94.90
68.40
82.93
We further visualize the t-SNE of the seen fake ProGAN, unseen fake StyleGAN and useen CycleGAN for the com- parison of vanilla CNN (Res-50 (Wang et al., 2020b)) and ours (see Fig. 9, Fig. 10). As we can see from both two figures, our approach maximizes and preserves the pre- trained knowledge while fitting the forgery patterns during training, whereas the vanilla CNN overfits the seen fake
MZTR (Wang et al.
81.07
94.91
55.71
76.99
F3Net (Qian al., 2020)
83,57
94.95
61.07
81.20
F2Traans (Miao et al., 2023)
89.64
97.47
81.43
90.55
CFM (Luo et al., 2023b)
85.00
92.74
76.07
84.55
FA-VIT (Lao al., 2023g)
92.86
98,10
82.57
91.20
Ours
95.71
99.26
85.71
95.83
16

---

Orthogonal Subspace Decompasition for Generalizable AI-Generated Image Detection
learning fake patterns.
Table 14. Ablation studies on deepfake image detection (Cross- dataset) regarding different VIT architectures were used. We employ the two architectures implemented in the original paper of CLIP (Radford et al., 2021) for experiments. All models are
D. Limitation and Future Work
The core idea of this paper is to decompose the original feature space into two orthogonal subspaces for preserving pre-trained knowledge while learning the forgery. In our manuscript and supplementary, we have conducted exten- sive experiments and in-depth analysis on both deepfake and synthetic image detection benchmarks, showing the su- perior advantages in both generalization and efficiency. One limitation of our work is that our approach regards all fake methods in one class during training real/fake classifiers, po- tentially ignoring the specificity and generality of different fake methods.
trained on FT++ (c23).
VFMs
#Params
#ingSize
CDF-v2
SimSwap
Avg.
CLIP-Basc/16
86M
224
0,854
0,833
0.844
+ Ours
0.07M
224
0.915
0.919
0.917
CLIP-Large/14
307M
224
0,857
0.860
0.859
Ours
0.19M
224
0.956
0.926
0.941
Table 15. Ablation studies on synthetic image detection (Uni- Fakeeeett Dataset) regarding different architectures were used. All models are trained on ProGAN's images and lested on 19 different kinds of generative models' images.
In the future, we plan to expand our approach into a incre- mental learning framework, where each fake method will be regarded as "one SVD branch", ensuring the orthogonality between different fake methods, thereby avoiding the severe forgetting of previous learned fake methods. This extension design will help our approach better address the future deep- fake types in the real-world scenario. Additionally, although our work's 's scope mainly focuses on deepfake and synthetic image detection, our approach also has the potential to be applied to other similar fields such as face anti-spoofing, anomaly detection, etc. Furthermore, we hope our proposed approach can inspire future research in developing better orthogonal modeling strategies.
VPMs
#Params
#imgSize
mAP
mAcc
86M
224
96.25
82,52
Ours
0.07M
224
98.47
88.46
CLIP-Large/14
307M
224
97.95
86.22
Ours
0.19M
224
99.41
95.19
method, learning forgery patterns only, thereby resulting in highly low-ranked feature space (see Fig. 6 and Fig. 7 of the manuscript for details) and causing the overfitting to seen forgery patterns in the training set. Additionally, we see that the logit distribution of the vanilla CNN has larger overlapping between fake and real, while ours is highly smaller, suggesting that approach achieves better generalization performance.
Ethics & Reproducibility. All of the facial images that are utilized are sourced from publicly available datasets and are accompanied by appropriate citations. This guarantees that there is no infringement upon personal privacy. We will make all codes and checkpoints available for public access upon acceptance.
C.2. Gelf-Attention Map Visualizations
Here, we perform the self-attention maps visualization of the original CLIP-VIT ViT model (Original), the fully fine- tuned CLIP-ViT ViT model (FFT), the LoRA-trained CLIP-VI VIT model (LoRA), and our proposed orthogonal trained CLIP- ViT model (Ours) on the UniversalFakkeettot dataset (see Fig. 11, Fig. 12, Fig. 13 and Fig. 14). Specifically, for each block of the VÝT, the self-attention map denotes the self- attention coefficient matrix caleulated between the [CLS) token and the patch tokens. In the case of the LoRA compo- nent, the self- atffnattentinn maps are generated from left to right using the original + LoRA weights. the original weights, and the LoRA weights, respectively. In the case of the Ours component, the self-attention maps are generated from left to right using the principal + residual weights, the principal weights, and the residual weights, respectively. Surprisingly, observe that the semantic information is primarily con- centrated in the earlier blocks, and our proposed approach establishes orthogonality between the semantic subspace and the learned forgery subspace at the level of the self- attention map. It further explains that our proposed ap- proach can better preserve the pre-trained knowledge while
17

---

Orthogonal Subspace Decomposition for Generalizable AI-Generated Image Detection
Original (bed))
Criginal (iolod
(Fake)
Insk
Our (Fabe)
Figure 11. Self-attention map visualizations of CycleGAN part in fniversalFakeeettee Dataset (Wang al., 2020b). We visualize the fake image of CycleGAN part and the corresponding real image for each block of the CLIP-ViT model (there are a total of blocks, with IDs gradually increasing from top bottom).
18

---

Orthogonal Subspace Decomposition for Generalizable Al-Generated Image Detection
Original
Real
Original Slabe)
Figure 12. Self-attention map visualizations of ProGAN part in UniversalFake Dataset (Wang et al., 2020b). We visualize the fake image of ProGAN part and the corresponding real image for each block of the CLIP-VIT VIT model (there are total of 24 blocks, with IDs gradually increasing from top bottom).
19

---

Orthogonal Subspace Decomposition for Generalizable Al-Generated Image Detection
Realt
Original Slabe)
Figure 13. Self-attention map visualizations of DeepFake part in UniversalFakeDDted Dataset (Wang et al., 2020b). We visualize the fake image of DeepFake part and the corresponding real image for each block of the CLIP-VIT model (there are total of 24 blocks, with IDs gradually increasing from top bottom).
20

---

Orthogonal Subspace Decomposition for Generalizable AI-Generated Image Detection
Figure 14. Self-attention map visualizations of LDM part in UniversalFake Fake Detect Dataset (Wang et al., 2020b).We visualize the fake image of LDM part and the corresponding real image for each block of the CLIP-ViT model (there are total of 24 blocks, with IDs gradually increasing from top bottom).
21


Reviews:
[{"review_id": "2eVB9hRTdC", "reviewer": "Reviewer_cn1X", "summary": "The paper proposes a novel approach for detecting AI-generated images (AIGI), particularly deepfake and synthetic images. It highlights that existing detectors suffer from poor generalization ability when encountering unseen forgery methods, primarily due to overfitting to forgery patterns in the training set, resulting in a constrained and low-rank feature space.\n\nTo address this issue, the authors introduce a singular value decomposition (SVD)-based method that decomposes the feature space into two orthogonal subspaces: one for retaining pre-trained knowledge and the other for learning forgery-related patterns. This approach demonstrates superior generalization performance across multiple benchmark evaluations.\n\n## update after rebuttal. I thank the authors for carefully preparing the comments, my concerns have been addressed during the rebuttal, I would therefore be willing to raise my score.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**We sincerely thank Reviewer cn1X for the constructive comments, insightful questions, and useful suggestions.** We greatly appreciate and are encouraged by the reviewer's recognition of our motivation,  thorough experimental and theoretical analysis, methodological novelty, and superior experimental performance. Additionally, the reviewer raised several important concerns and questions, which we address in detail below.\n\n\n> **Q1.**  The paper treats all forgery methods as a single category during training. Has the consideration of the specificity and generalization of different forgery methods been explored? \n\n**R1.** Thanks for your very insightful question. As highlighted by the reviewer, treating all forgery methods as a single category in binary classification may potentially risk losing specificity and generalization—a concern we have not yet verified.\n\nHowever, please note that this limitation is inherent to binary classification tasks in general, **rather than specific to our proposed SVD-method in this work** (our SVD-based approach is designed specifically for adapting to new forgery types while preserving pre-trained knowledge).\n\nFollowing the reviewer's suggestion, we plan to propose an **extended version (multi-task) based on our current binary framework**, aiming explicitly at enhancing the balance between specificity (handling known forgeries effectively) and generalization (detecting unknown or unseen forgeries).\n\nThe preliminary idea of **our future multi-task learning framework** features:\n\n- (1) **Dual-head structure:** Two separate heads are employed—one dedicated to multi-class classification (specific head) and another for binary classification (general head).\n  - **Specific Head** (multi-class): Learns fine-grained differences among known forgery types, focusing on fitting to the training distribution (specificity, IID).\n  - **General Head** (binary): Captures shared features across forgery types, enabling the detection of unseen or novel manipulations (generality, OOD).\n- (2) **Adaptive Dynamic Inference:** Combining predictions from both heads based on confidence, balancing specificity and generality dynamically during inference.\n\nSpecifically, our **dynamic inference strategy** is as follows:\n\n- Compute the prediction probability (i.e., confidence) according to the specific head's logits.\n- If the maximal prediction confidence across classes is below a threshold  (indicating uncertainty or a \"flat\" distribution), then making decision based on the binary general head, otherwise based on the specific multi-class head. \n\nThis adaptive strategy effectively maintains specificity for known forgery methods while providing robust generalization against unseen or evolving forgeries.\n\n\nDue to the limited content in the rebuttal, we cannot provide exhaustive experimental details at this stage. But following the reviewer's suggestion, **we evaluate our binary model and multi-task model on the $\\text{Chameleon}$ dataset (see **R2** below for details)**.\n\n\n---\n\n> **Q2.** Chameleon is a high-quality dataset. Testing on this dataset would further validate the generalization ability.\n\n**R2.** Thanks for introducing and highlighting the high-quality and challenging dataset, $\\text{Chameleon}$. Following the suggestion, we have conducted additional evaluations using this dataset. We follow the proposed setting in the original paper for experiments, i.e., training on GenImage (whole) and testing on $\\text{Chameleon}$. \n\nAs shown in **Table 1** below, our method achieves superior generalization performance compared to baseline methods and other SOTA detectors. Also, the proposed extension version, i.e., multi-task framework, further boosts and refines the results, alleviating the potential loss of specificity and generality problem raised by the reviewer.\n\n\n***Table 1: Evaluation results on Chameleon. All detectors are trained on the GenImage dataset.***\n\n| CNNSpot | FreDect | Fusing | GramNet | LNP | UnivFD | DIRE | Patch | NPR | AIDE | Ours (binary) |Ours (multi-task)\n|-|-|-|-|-|-|-|--|-|-|-|-|\n|60.89|57.22|57.09| 59.81 | 58.52 | 60.42 | 57.83 | 55.70 | 57.81 | 65.77| 70.27 | 72.62\n\n\nFurthermore, all evaluated methods experience a notable performance decline on $\\text{Chameleon}$, highlighting the dataset's significant difficulty. Inspired, we plan to conduct an in-depth analysis of $\\text{Chameleon}$ in future research. Again, we greatly appreciate your suggestion.\n\n\n---\n\n\n> **Q3.** About the robustness evaluation.\n\n**R3.** Thank you for your question. We have already acknowledged this concern and have performed a robustness evaluation on the deepfake detection benchmark to assess our model's robustness, **as presented in Figure 7 of our appendix**. Following [1,2], we evaluate three types of image degradation: block-wise distortion, contrast changes, and JPEG compression. This experiment confirms the model's robustness against various perturbations.\n\n\n[1] LSDA, CVPR 24.\n\n[2] LipForensics, CVPR 21.", "claims_and_evidence": "The paper’s primary claim is that orthogonal subspace decomposition enhances the generalization ability of AI-generated image (AIGI) detection. This claim is well-supported by experimental results, particularly in cross-dataset and cross-forgery evaluations, where the proposed method significantly outperforms state-of-the-art approaches.\n\nThe results demonstrate that the method effectively preserves pre-trained knowledge while simultaneously learning forgery patterns, leading to improved generalization performance in synthetic image detection.", "methods_and_evaluation_criteria": "The proposed method is based on SVD decomposition, where the principal components are frozen, and the remaining components are adjusted to preserve pre-trained knowledge while learning forgery patterns. This approach is theoretically sound and has been empirically validated through experiments.\n\nThe evaluation metrics include cross-dataset and cross-forgery tests, using AUC and accuracy as key indicators. These evaluation criteria are reasonable and align with existing research in AIGI detection.", "theoretical_claims": "The paper presents a theoretical analysis explaining the asymmetry phenomenon in AIGC detection and demonstrates, through covariance spectrum analysis, the inevitable failure of symmetric classifiers in this task. The theoretical analysis is well-grounded and aligns with the experimental results.", "experimental_designs_or_analyses": "The experimental design is well-structured, covering multiple datasets and forgery methods, effectively validating the generalization ability of the proposed approach.\n\nThe authors also conducted ablation studies, confirming the effectiveness of the SVD-based method and loss constraints. Experimental results indicate that the SVD method is the primary driver of performance improvement, while orthogonality constraints and singular value constraints further optimize generalization performance.", "relation_to_broader_scientific_literature": "The paper is closely related to the existing literature on AIGI detection, particularly studies addressing generalization challenges. The authors cite a wide range of relevant works and highlight the limitations of existing methods.\n\nBuilding upon prior research, the proposed approach introduces an innovative solution by leveraging orthogonal subspace decomposition to enhance generalization performance.", "essential_references_not_discussed": "\"A Sanity Check for AI-Generated Image Detection\" is a high-quality dataset that could serve as an additional benchmark for evaluating the proposed method. Testing on this dataset would further validate the generalization ability of the approach and provide a more comprehensive comparison with existing methods.", "other_strengths_and_weaknesses": "Strengths:\nThe paper proposes an innovative approach that addresses the generalization problem in AIGI detection through orthogonal subspace decomposition.\nThe experimental design is comprehensive, covering multiple datasets and forgery methods, effectively validating the method's effectiveness.\nThe theoretical analysis is insightful, providing an explanation for the asymmetry phenomenon in AIGI detection.\n\nWeaknesses:\nThe paper treats all forgery methods as a single category during training, which may overlook the specificities and commonalities of different forgery techniques.\nThe proposed method may face additional challenges in real-world applications, such as shifts in data distribution and the emergence of new forgery techniques.", "other_comments_or_suggestions": "The overall quality of the paper is high, with thorough experimental and theoretical analysis. It is recommended that the authors include ablation studies on backbones and robustness testing in the experimental section, as well as add results from more high-quality datasets to further strengthen the findings.", "questions_for_authors": "The paper treats all forgery methods as a single category during training. Has the consideration of the specificity and generalization of different forgery methods been explored? Could this approach result in the loss of unique characteristics for different forgery types, potentially degrading detection performance? For instance, compared to methods like MoE, could this lead to a lack of specialized knowledge?\n\nIn practical applications, there are often many adversarial scenarios and a variety of robustness tests. How well does the method perform under these conditions?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "NA."}}, {"review_id": "5Vnk6sLNSY", "reviewer": "Reviewer_4w2k", "summary": "This paper investigates the failure of generalization in AI-generated image detection, identifying an asymmetry phenomenon where detectors overfit to limited fake patterns, resulting in a low-rank and constrained feature space. To mitigate this, the authors leverage the vision foundation models and propose a novel SVD-based tuning approach that freezes the principal components while adapting the remaining components, explicitly ensuring orthogonality to maintain a higher feature rank, thereby alleviating the overfitting problem.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**We sincerely thank Reviewer 4w2k for the constructive comments, insightful questions, and useful suggestions.** We greatly appreciate and are encouraged by the reviewer's recognition of our motivation with sufficient and reasonable evidence, methodological novelty, and interesting analysis method. Additionally, the reviewer raised several important concerns and questions, which we address in detail below.\n\n\n\n> **Q1.** What about the performance of ViT models trained on ImageNet-1K and other ViT-based VFMs?\n\n**R1.** Thank you for raising this. Following the reviewer's suggestion, we conduct additional experiments comparing ViT models pre-trained on ImageNet-1K and SigLIP using different adaptation methods. Results are summarized clearly in **Table 2** below:\n\n\n***Table 2: Results of other ViT-based baseline models.***\n| Pre-trained Model   | Tuning Methods | Mean Accuracy (%) |\n|--------|---------|--------|\n| SigLIP              | svd     | 90.46     |\n| SigLIP              | lora    | 83.42     |\n| SigLIP              | fft     | 81.23      |\n| CLIP                | svd     | 95.19      |\n| CLIP                | lora    | 93.03      |\n| CLIP                | fft     | 86.22      |\n| DINOv2              | svd     | 85.46    |\n| DINOv2              | lora    | 83.42    |\n| DINOv2              | fft     | 79.52     |\n| ViT-ImageNet-1K     | svd     | 72.77    |\n| ViT-ImageNet-1K     | lora    | 70.47    |\n| ViT-ImageNet-1K     | fft     | 68.41   |\n\nFrom these results, we observe that our SVD-based adaptation method consistently achieves the best performance over the LoRA-based adaptation and full fine-tuning (FFT) across different pre-training models.\n\n\n---\n\n> **Q2.** How can the authors implement their method in GenImage benchmark?\n\n**R2.** We sincerely appreciate the kind comment. Our experiments primarily follow the implementation details and experimental settings described in recent SOTAs [1, 2]. This alignment ensures consistency and transparency, making our results directly comparable with existing studies.\n\n[1] NPR. CVPR 2024.\n[2] FatFormer. CVPR 2024\n\n\n---\n\n\n> **Q3.** Also, can the detection method be applied to detect more complex and advanced fakes, such as the talking-head generation contents?\n\n**R3.** Thank you for your question. Following your suggestion, we have extended our evaluation to include highly realistic talking-head deepfake content, i.e., HeyGen, selected from the DF40 dataset. To ensure a fair comparison, we add four SOTA detection methods under identical experimental conditions for a fair comparison. \n\nThe results, summarized clearly in Table x below, demonstrate that our method achieves superior performance compared to the latest SOTA detectors, even on advanced commercial talking-head manipulations.\n\n\n***Table 3: Evaluation results on the advanced talking-head generation fake contents.***\n\n| LSDA (CVPR'24) | ProDet (NIPS'24) | FSFM (CVPR'25) | UDD (AAAI'25) | Ours\n|---------|---------|--------|---------|-------|\n| 46.7   | 41.0   | 70.8  | 75.4   | **79.7** |\n\nFrom the results above, we find that **some detectors trained on the *face-swapping data* fail to generalize the *talking head contents***. However, most previous works are conducted using only the face-swapping deepfake data. Inspired by the reviewer's comment, we plan to add this additional evaluation to the revision and further enlarge our evaluation in the future.\n\n---\n\n> **Q4.** The authors did not conduct an ablation study on the weights of the orthogonality loss and singular value loss. How are these weights allocated in the loss function?\n\n**R4.** Thanks for the comment. In our experiments, **we set the weights for both the orthogonality loss and singular value loss terms to 1.0 by default**. To further explore the influence of these loss terms, we adjust each hyper-parameter across a wide range. \n\nAs shown in **Table 4** below, **varying these parameters results in only slight fluctuations in performance,** indicating the stability of our method. Importantly, the inclusion of these loss terms consistently improves performance compared to models without them.\n\n\n***Table 4: Ablation studies regarding different weights of the loss terms.***\n\n| Orthogonality Loss Weight | Singular Value Loss Weight | AUC on SimSwap (%) |\n|-|-|-|\n| 0.0 (No loss, SVD only)    | 0.0 (No loss, SVD only)              | 94.0         |\n| 1.0 (Default) | 1.0 (Default)   | 95.6         |\n| 0.5  | 1.0   | 95.5         |\n| 1.0   | 0.5   | 95.0         |\n| 0.5    | 0.5       | 94.5         |\n| 2.0     | 2.0     | 95.1         |\n| 0.1    | 0.1      | 94.4         |\n\n\n---\n\n> **Q5.** I hope the authors can cite references [1] and [2] and include a brief discussion on their relevance.\n\n**R5.** Thanks for the valuable suggestion. Both [1] and [2] introduce the concept of effective rank to quantitatively measure the dimensionality of the feature space, which is similar to our case. We will provide a detailed discussion in our revision. Thank you again.", "claims_and_evidence": "The claims in this paper are well-supported by evidence and detailed explanations. Specifically, Figure 1 validates the existence of the asymmetry phenomenon and shortcut overfitting in AIGI detection. Figure 2 illustrates that the constrained feature space is predominantly influenced by fake data. Figures 3 and 5 further confirm that a baseline model trained naively on the AIGI dataset tends to be highly low-ranked, whereas the proposed approach effectively preserves most of the pre-trained knowledge.", "methods_and_evaluation_criteria": "The evaluation criteria and chosen benchmarks/datasets are appropriate for the problem. The paper demonstrates the effectiveness of the proposed method, achieving SOTA performance on both deepfake and AIGC detection benchmarks (Tables 1, 2, and 3). Also, the evaluation protocols are consistent with those used in many existing studies, ensuring a fair comparison.", "theoretical_claims": "The proposed theoretical claims are presented in Theorem 3.2, where the authors define the covariance similarity between real and fake images and prove that this similarity has a lower bound under certain assumptions. I did not identify any obvious errors or issues in the validity of the proof.", "experimental_designs_or_analyses": "Overall, the experimental designs are suitable. However, I have identified the following issues:\n(1) The authors did not conduct an ablation study on the weights of the orthogonality loss and singular value loss, so it is unclear how the weights are allocated in the loss function;\n(2) The authors primarily use CNNs as baselines. Why not consider ViT-based baselines, such as ViT models trained on ImageNet-1K?\n(3) The main tables (Tables 1, 2, and 3) lack bold or underlined results, which affects readability.", "relation_to_broader_scientific_literature": "This paper introduces the concept of effective rank to quantify the overfitting problem in AIGI detection, which I find interesting and relevant to key concepts in other fields. Notably, effective rank is widely used in continual learning [1] and signal processing [2], where it mainly helps assess how much knowledge a model retains for given tasks. In this paper, the authors cleverly use this concept to compute the “dimensionality” of the feature space, which makes sense to me.\n \n[1] Loss of plasticity in deep continual learning. Nature (2024).\n \n[2] THE EFFECTIVE RANK: A MEASURE OF EFFECTIVE DIMENSIONALITY. European Association For Signal Processing (2007).", "essential_references_not_discussed": "I hope the authors can cite references [1] and [2] and include a brief discussion on their relevance.", "other_strengths_and_weaknesses": "I have listed the strengths and weakness in my above comments. So, not additional comment here.", "other_comments_or_suggestions": "-       The authors did not conduct an ablation study on the weights of the orthogonality loss and singular value loss. How are these weights allocated in the loss function?\n-       The baselines primarily consist of CNN models. Why were ViT-based baselines, such as ViT models trained on ImageNet-1K, not considered?\n-       How can the authors implement their method in GenImage benchmark? It seems like the authors don’t provide any implementation details for that.\n-       Also, can the detection method be applied to detect more complex and advanced fakes, such as the talking-head generation contents?\n-       Furthermore, can this method be applied to broader fields beyond deepfake and AIGC detection? How does it perform in areas such as domain generalization and anomaly detection? I hope the authors can further clarify the broader applicability of their model.\n \nI suggest and hope the authors can address my above concerns one-by-one carefully.", "questions_for_authors": "No.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "faSJhjTpjk", "reviewer": "Reviewer_j2Rb", "summary": "This paper proposes Effort, a novel SVD-based adapter tuning method, for generalizable AI-generated image detection. The key idea is constructing two orthogonal subspaces, where the principal components preserve the pre-trained knowledge from the vision foundation models while the residual components are utilized to learn new domain-specific forgery patterns for detection. The paper also leverages PCA to quantify the effective rank of the learned feature space, explaining the failure of conventional detectors.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**We sincerely thank Reviewer j2Rb for the constructive comments, insightful questions, and useful suggestions.** We greatly appreciate and are encouraged by the reviewer's recognition of our insightful and in-depth analysis, methodological novelty, and comprehensive experiments with high generalization performance. Additionally, the reviewer raised several important concerns and questions, which we address in detail below.\n\n\n> **Q1.** The paper does not provide an intuitive visualization of residual and principal components using PCA. It would provide some new insights and findings.\n\n**R1.** We genuinely appreciate the suggestion. **We perform the visualization of the attention maps before and after our proposed orthogonal training** on the UniversalFakeDetect dataset. Specifically, for each block of the vision transformer, we visualize the attention map, which represents the attention coefficient matrix calculated between the cls token and the patch tokens. \n- Specifically, the attention map is computed across the multiple heads on average and is presented alongside the principal weights, residual weights, and total weights, respectively. \n- We have three discoveries in general: \n    - 1) The attention map of principal weights is almost identical to the attention map of the total weights for each block; \n    - 2) Before and after training, the attention maps of the residual weights in the earlier blocks do not respond (e.g., for ViT-L of CLIP model, the first 22 blocks do not respond); \n    - 3) Only the attention maps in the last blocks of the residual weights respond, which means they contain the real/fake discriminating information (e.g., for ViT-L of CLIP model, the last 2 blocks respond).\n\nFollowing the reviewer's suggestion, we will add these visualization results and the corresponding analysis into our revision.\n\n---\n\n\n> **Q2.** Some important implementation details of fine-tuning and evaluations are missing in the paper. For instance, how many layers are fine-tuned? What kinds of data augmentations are used in this paper?\n\n**R2.** Thanks for the suggestion. We list the implementation details below.\n- Fine-tuned layers:\nBy default, we fine-tune all query (Q), key (K), value (V), and output (Out) components across every layer of the given architecture, such as CLIP.\n- Data augmentation strategy:\nWe adopt standard data augmentation techniques commonly used in each specific benchmark. For instance, we follow [1, 2] for deepfake detection, [3] for the UniversalFakeDetect benchmark, and [4] for the GenImage benchmark.\n\nFollowing the reviewer's suggestion, we will present a detailed description in the revision.\n\n[1] LSDA. CVPR 2024.\n[2] DeepfakeBench. NeurIPS 2023.\n[3] UnivFD. CVPR 2023.\n[4] NPR. CVPR 2024.\n\n---\n\n> **Q3.** The observation in Table 6 that lower-rank SVD achieves better generalization results is intriguing. The authors should provide a detailed explanation for this.\n\n**R3.** We genuinely appreciate the suggestion. Our choice of using a lower rank (specifically, \"n-r\"=1) for fine-tuning DFD is primarily motivated by two critical factors:\n- First, **the nature of the real-fake classification task itself makes it relatively straightforward.** Specifically, fake samples in existing training sets tend to exhibit a limited number of distinctive forgery patterns (FF++ contains only four forgery types), each with relatively simple and consistent characteristics. \n    - Due to this simplicity and limited diversity, a low-rank adaptation with a small rank (e.g., \"n-r\"=1, 4, or 16) is sufficient for the model to effectively learn these forgery patterns. As demonstrated by Table 5 in our paper, choosing ranks of 1, 4, or 16 yields very similar performance results. Given this observation, **we prioritize efficiency and parameter economy, making rank 1 the optimal choice.**\n- Second, **the inherent characteristics of binary classification further justify selecting a smaller rank.** *Binary classification tasks typically do not require the model to learn extensive and nuanced patterns*, but rather to identify just enough distinctive features to separate the two classes, making the learned feature space inherently constrained. Thus, binary classification inherently simplifies the complexity of the learning problem, meaning that employing a higher rank would not provide significant additional benefit. \n    - In contrast, more complex tasks like multi-class classification (as discussed in **our response R1 of the Reviewer cn1X**) necessitate a higher rank (value of \"n-r\"). Indeed, our experimental results indicate that increasing the value of \"n-r\" in multi-class scenarios leads to improved performance, highlighting the relationship between task complexity and optimal rank selection.\n\n---\n\n> **Q4.** The improvement of the two proposed constrained losses shown in the ablation study is limited.\n\n**R4.** Thanks for your comment. Please refer to our response of **R4 of the reviewer 4w2k**.", "claims_and_evidence": "The authors have provided appropriate references, clear visualizations, and thorough analysis, making their work technically sound and well-justified.", "methods_and_evaluation_criteria": "The used evaluation methods and criteria are suitable and common in the related fields. The used benchmarks are commonly used in existing works.", "theoretical_claims": "Although the authors provide a theoretical explanation to the asymmetric phenomena, I believe this theorem does not significantly contribute to the overall paper. Instead, I suggest moving this content to the supplementary material and replacing it with Algorithm 1, which is much more helpful in understanding the core concepts of the paper.", "experimental_designs_or_analyses": "This paper has conducted a thorough evaluations on both deepfake and AIGC detection benchmarks, and most analysis experiments are insightful and carefully designed. However, I noticed in Table 6 that lower values of r yield better generalization results. This observation seems counterintuitive, and I believe the authors should provide a clear and reasonable explanation for this phenomenon.", "relation_to_broader_scientific_literature": "I believe the proposed method in this paper is quite general and may not be limited strictly to AIGC detection. I encourage the authors to provide a detailed discussion on how their approach could be applied to other related fields, which would further highlight its versatility and broader impact.", "essential_references_not_discussed": "Most related and essential works are properly cited in this paper.", "other_strengths_and_weaknesses": "Strengths:\n\n-\tThis paper provides an in-depth and reasonable analysis for the failure reason of existing detectors. The proposed analysis approach is insightful and new to me.\n\n-\tThis paper proposes a novel SVD-based method that can explicitly ensure the orthogonality between pretrained knowledge and deepfake-specific knowledge.\n\n-\tThis paper conducts comprehensive evaluations on both deepfake detection and AIGC detection benchmarks, achieving high generalization performance over existing methods.\n\nWeaknesses:\n\n-\tThe two proposed constrained losses seem not very necessary, as the improvement shown in the ablation study is limited. Instead, why not make a constrain to the effective rank, as the rank is an important concept in the paper?\n\n-\tThe paper does not provide an intuitive visualization of residual and principal components using PCA? It would provide some new insights and findings.\n\n-\tSome important implementation details of fine-tuning and evaluations are missing in the paper. For instance, how many layers are fine-tuned? What kinds of data augmentations are used in this paper?", "other_comments_or_suggestions": "Overall, I think the quality of the paper is high, with insightful analysis and new effective method proposed. My comments and suggestions are summarized below.\n\n-\tMoving Algorithm 1 to the main body of the paper would greatly enhance understanding, as it provides a clear and practical insight into the key methodology.\n\n-\tThe observation in Table 6 that lower-rank SVD achieves better generalization results is intriguing. The authors should provide a detailed explanation for this phenomenon to clarify its implications.\n\n-\tThe analysis of Figure 6 is particularly interesting and insightful. I encourage the authors to include more visualizations of existing detectors, as this could serve as a critical analytical tool for the entire field. It effectively reveals the \"discrimination dimension\" in deepfake detection, which is a valuable contribution. \n\n-\tTo provide a more intuitive understanding, I suggest the authors include visualizations of the principal and residual components. For example, what specific regions or features do the residual components focus on? This would help readers better grasp the underlying mechanisms of the proposed approach.\n\n-\tAdding more implementation details into the main paper or supplementary, as it helps the readers better understand some technical details within the method.", "questions_for_authors": "See the weakness", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "voJtQhLw7V", "reviewer": "Reviewer_SVkD", "summary": "In this paper, the authors explain the issue of lacking generalization in deepfake detection from the perspective of SVD. Specifically, they argue that due to the limited nature of fake features, models trained on deepfake datasets tend to produce low-rank matrices, which leads to a failure in capturing key components necessary for distinguishing real images and unseen samples. To address this issue, they propose freezing most of the parameters in a pre-trained model—preserving its ability to detect real images—while only adjusting a subset of parameters to adapt to forgery patterns. This parameter decomposition is achieved through SVD decomposition.", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**We sincerely thank Reviewer SVkD for the constructive comments, insightful questions, and useful suggestions.** We address the reviewer's concerns below.\n\n\n\n\n> **Q1.** The authors should provide additional visualizations for different testing datasets. For instance, face deepfakes might be less diverse than general AI-generated images, leading to inconsistent results.\n\n**R1.** Thanks for your valuable suggestion.\n- Indeed, the number of principal components (effective rank) varies across datasets because our method measures the \"effective dimension\" of the feature space, which **depends on the input data distribution**.\n- To validate this, we analyzed the effective rank on multiple test datasets, including **CDF-v2 (face-focused)** and **UniversalFakeDetect (general-content)**. Key observations from **Table 5** are:\n    - **General datasets exhibit a higher effective rank than face-specific datasets.** This is because general datasets contain diverse objects, leading to a more complex feature space.\n    - **Our SVD-based method outperforms LoRA and full fine-tuning (FFT)** in capturing a higher-rank feature space, preserving more discriminative information.\n\n\n\n\n\n***Table 5: Evaluation results on different testing datasets.***\n\nTuning Methods | Train Datasets | Test Datasets | Effective Rank | Mean Accuracy (%) |\n-|-|-|-|-|\n| svd    | FF++  (**face**) | CDF-v2 (**face**) | 159  | 95.60  |\n| lora    | FF++  (**face**) | CDF-v2 (**face**) | 137  | 89.40  |\n| fft     | FF++  (**face**) | CDF-v2 (**face**)  | 57   | 85.70   |\n| svd      | UniversalFakeDetect (**general**) | UniversalFakeDetect (**general**)  | 316  | 95.19    |\n| lora     | UniversalFakeDetect (**general**) | UniversalFakeDetect (**general**)   | 304  | 93.03    |\n| fft     | UniversalFakeDetect (**general**) | UniversalFakeDetect (**general**)   | 238    | 86.22     |\n\n\n\n\n---\n\n\n> **Q2.** The authors have not provided a clear explanation for why n-r=1 achieves the best results.\n\n**R2.** Thanks for the interesting questions. Our choice of using a lower rank (specifically, n-r=1) for fine-tuning DFD is primarily motivated by two critical factors:\n- First, **the nature of the real-fake classification task itself makes it relatively straightforward.** Specifically, fake samples in existing training sets tend to exhibit a limited number of distinctive forgery patterns (FF++ contains only four forgery types), each with relatively simple and consistent characteristics. \n    - Due to this simplicity and limited diversity, a low-rank adaptation with a small rank (e.g., \"n-r\"=1, 4, or 16) is sufficient for the model to effectively learn these forgery patterns. As demonstrated by Table 5 in our paper, choosing ranks of 1, 4, or 16 yields very similar performance results. Given this observation, **we prioritize efficiency and parameter economy, making rank 1 the optimal choice.**\n- Second, **the inherent characteristics of binary classification further justify selecting a smaller rank.** *Binary classification tasks typically do not require the model to learn extensive and nuanced patterns*, but rather to identify just enough distinctive features to separate the two classes, making the learned feature space inherently constrained. \n    - To illustrate, when distinguishing between cats and dogs, the classifier might achieve good accuracy simply by examining the tails, without needing detailed knowledge about specific breeds such as Corgis or Shibas. Thus, **binary classification inherently simplifies the complexity of the learning problem**, meaning that employing a higher rank would not provide significant additional benefit. \n    - In contrast, more complex tasks like multi-class classification (as discussed in our extended framework proposed in **our response R1 of the Reviewer cn1X**) necessitate a higher rank (the value of \"n-r\"). Indeed, our experiments indicate that **increasing the value of \"n-r\" in multi-class scenarios leads to improved performance**, highlighting the relationship between task complexity and optimal rank selection.\n\n***Table 6: Optimal rank selection based on the task complexity. We evaluate all the models on the Chameleon dataset.***\n\n| binary (n-r=1) | binary (n-r=16) | binary (n-r=32) | binary (n-r=64) | multi-task (n-r=1) | multi-task (n-r=16) | multi-task (n-r=32) | multi-task (n-r=64)\n|-|-|-|-|-|-|-|-|\n| 70.27   | 69.34   | 69.08  | 68.87   | 72.62 | 72.89 | 73.76 | 74.18 \n\n\n\n---\n\n> **Q3.** Many details in the analysis are missing, such as Figure 5 and Figure 6.\n\n**R3.** Thanks for the kind mention. All analysis figures, including Figure 5 and Figure 6, are tested on FF_c23 (test), which is commonly used as the within-domain testing dataset. We will clarify it in the revision.\n\n\n\n---\n\n> **Q4.** The implementation and experimental details of GenImage are missing.\n\n**R4.** Thanks for your kind mention. We will add the implementation details and clarify them clearly in the revision.", "claims_and_evidence": "The proof that explains generalization loss using the rank of the SVD decomposition is complete.", "methods_and_evaluation_criteria": "Yes, the authors adopted standard datasets and evaluation metrics commonly used in deepfake detection.", "theoretical_claims": "The reviewer examined the authors’ explanation of the “asymmetry phenomenon” and found it convincing.", "experimental_designs_or_analyses": "[1] The experimental setting of cross-dataset and cross-method evaluation is feasible, and the chosen baselines are appropriate.\n\n[2] The ablation study examines the impact of individual components, the effects of switching the backbone, and comparisons with other PEFT methods, making the experimental design relatively comprehensive.", "relation_to_broader_scientific_literature": "Extensive research has been conducted on the generalization issue in deepfake detection, with numerous studies addressing its existence and potential causes—including the distribution differences between real and fake features, as discussed in this paper. However, this paper provides a novel perspective by further analyzing the problem through the lens of SVD. In particular, the idea can be useful in addressing challenges such as analyzing overfitting problems, examining the expressivity of feature spaces, and preserving pre-trained knowledge while adapting to downstream tasks.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "The reviewer does not have major concerns, some minor weaknesses are listed below:\n\n[1] Many details in the analysis are missing. For instance, which testing datasets are used to generate the visualizations of Figure 5 and Figure 6\n\n[2] The implementation and experimental details of GenImage are missing. It is not specified whether the authors used the same settings as those in the UniversalFakeDetect benchmark.\n\n[3] In Figure 6, the number of main principal components is heavily influenced by the testing dataset. If the dataset changes, the values may differ. The authors should provide additional visualizations for different testing datasets. For instance, face deepfakes might be less diverse than general AI-generated images, leading to inconsistent results.\n\n[4] The average results in Table 4 don’t align with those mentioned in the text.\n\n[5] How is the retained rank r determined? Would adjusting r for different datasets lead to better or different results? The authors also have not provided a clear explanation for why n-r=1 achieves the best results.\n\n[6] The font size in Figure 6 is too small, particularly in the legend.", "other_comments_or_suggestions": "The reviewer has no further comments on the paper. However, the reviewer would like to offer one additional suggestion on writing: the concept of the “asymmetry phenomenon in AI-generated image detection” seems like a rebranding of the generalization issue and comes across as somewhat overly elaborate. This does not affect the reviewers’ assessment of the paper’s quality. However, it would be ideal if this section can be condensed a bit.", "questions_for_authors": "Please see weaknesses", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper has received three accept and one strong accept scores. All the reviwers acknowledge reading the rebuttals. Nothing more needs to be said.
Decision: Accept (oral)