Fair Deepfake Detectors Can Generalize
Authors: Harry Cheng
Ming-Hui Liu
Yangyang Guo
Tianyi Wang
Liqiang Nie
Mohan Kankanhalli
Venue: NeurIPS 2025 poster ()
Keywords: Deepfake Detection; Causal Inference; Generalization; Fairness; Back-door Adjustment
Primary area: social_and_economic_aspects_of_machine_learning
TL;DR: In this paper, we provide the first demonstration of a direct causal relationship between fairness and generalization. Furthermore, we show that both can be simultaneously improved by controlling for confounding factors.

Abstract: Deepfake detection models face two critical challenges: generalization to unseen manipulations and demographic fairness among population groups. However, existing approaches often demonstrate that these two objectives are inherently conflicting, revealing a trade-off between them.
In this paper, we, for the first time, uncover and formally define a causal relationship between fairness and generalization.
Building on the back-door adjustment, we show that controlling for confounders (data distribution and model capacity) enables improved generalization via fairness interventions.
Motivated by this insight, we propose Demographic Attribute-insensitive Intervention Detection (DAID), a plug-and-play framework composed of: i) Demographic-aware data rebalancing, which employs inverse-propensity weighting and subgroup-wise feature normalization to neutralize distributional biases; and ii) Demographic-agnostic feature aggregation, which uses a novel alignment loss to suppress sensitive-attribute signals.
Across three cross-domain benchmarks, DAID consistently achieves superior performance in both fairness and generalization compared to several state-of-the-art detectors, validating both its theoretical foundation and practical effectiveness.

Fair Deepfake Detectors Can Generalize
Harry Cheng National University of Singapore xaCheng1996@gmail. com
Ming-Hui Liu
Shandong University liuminghui @mail sdu edu.
Yangyang Guo" National University Singapore guoyang. eric@gmail. com
Tianyi Wang
National University of Singapore terry ai. wang@gnail. com
Liqiang Nie
Mohan Kankanhalli
Harbin Institute of Technology (Shenzhen) mieliqiangeel com
National University of Singapore mohan@comp. nus. edu. sg
Abstract
Deepfake detection models face two critical challenges: generalization to unseen manipulations and demographic fairesss among population groups. However, existing approaches often demonstrate that these two objectives are inherently conflicting, revealing a trade-off between them. In this paper, we, for the first time, uncover and formally define causal relationship between fairness and gen- eralization. Building on the back-door adjustment, we show that controlling for confounders (data distribution and model capacity) enables improved generaliza- tion via fairness interventions. Motivated by this insight, we propose Demographic Attrbuue-insensitive Intervention Detection (DAID), a plug-and-play framework data rebalancing, which employs inverse-
composed of: i) Demographic-awarr propensity weighting and subgroup-wizg wise feature normalization to neutralize dis- tributional biases; and ii) Demographic- aggnossi feature aggregation, which uses a novel aligmment loss to suppress cunitivee-attriuu signals. Across three cross- domain benchmarks, DAID consistently achieves superior performance in both fairness and generalization compared to several state-of-the-an detectors, validating both its theoretical foundation and practical effectiveness.
Introduction
With the advancement of cutting-edge facial synthesis models, attackers can generate high-quality forged faces at minimal cost [69 27]. resulting in serious negative social implications [65] In
response to these threats, numerous deepfake detection methods have been proposed [17
31 80
Employing binary real/fake classification 79 53], these approaches have achieved promising results when trained and tested on datasets with similar distributions (i.e., forged samples generated using the same manipulation techniques). However, their generalization ability remains limited when faced
with previously unseen forgery methods [28 52 63 39
5 73 19].
The problem lies in that a detector should maintain consistent performance across different demographic groups, such as gender and race. However, prior studies 22 12 34 have predominantly shown that simply improving cross- cross-domain generalization does not benefit all demographic subgroups equally
On the other hand, the faimess of deepfake detectors has also drawn increasing attention 13 35
"Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

0.70
Backbone Bockbone-FG
Data Imbalance
Distribution Shift
0.65
DD
Geranaaaaion ton
Redluced
72
0.33
dolf)
A
A
0.45
Overiiting Effect
68
Majority Bias
MC
MC
0.30
(a)
(b)
Figure 1: (a) Comparison of model performance on Celeb-DF on Skew [6 (fairness metric, the lower the better) and AUC (generalization metric, the higher the better). FG 33 is a method to improve fairness, but may compromise the detector's generalization ability. (b) Causal graph for relationship between fairness and generalization, where data distribution (DD) and model capacity (MC) act as confounders, i.e., they can affect both metrics, thereby obscuring the true causal relationship.
(i.e., generalization A faimnss).. Meanwhile, as shown in Figure D. pushing detectors to more fair can compromise generalizability, which arguably makes these trade-off (23).
Different from existing studies that treat fairness and generalization as competing objectives, our preliminary experiments show that improving detector faimess can occasionally lead to enhanced cross-domain generalization. This finding motivates our hypothesis that demographic fairness causally improves generalization performance (i.e., fairness + generalization), although this effect is often obscured by confounders. To formalize this intuition, we investigate two possible confounders (data distribution (DD) and model capacity (MC)) and construct the resulting causal graph (see Figure [p). In this graph, fairness (F) functions as a treatment variable exerting a causal influence on generalization (A). However, data distribution (DD) and model capacity (MC) act confounders affecting both metrics and potentially obscuring the true causal relationship. Using the back-door criterion 46). which blocks spurious paths, we demonstrate causal relationship between fairness and generalization under this causal model. Specifically, we explicitly stratify the dataset based on human demographic attributes and control for model capacity (see Section 3 for details).
To further validate our insight, we propose a novel Demographic Attribute- curibute-insensitiie Intervention Detection (DAID) approach. Rather than directly optimizing for cross-domain generalization [57] 70, DAID explicitly controls for both data distribution and model capacity confounders. In doing so, DAID elucidates the causal relationship between fairesss and generalization during training, and generalization can be improved by intervening on faireess. To this end, our DAID is equipped with complementary modules. First, apply demographic-awar data rebalancing module, which uses adaptive sample reweighting and per-group normalization to mitigate distributional bias. Second, we propose demographic-agnoe
feature aggregation, which aligns same-label samples across different demographic groups through demographic-agnonn agnostic optimization strategy. Together, these modules serve distinct but synergistic purposes: the data rebalancing module ensures equitable representation across subgroups, while the feature aggregation module enhances the model's ability mitigate the influence of human-related attributes. and model-level confounders, while achieving substantial improvements in fairness.
result, DAID effectively controls both data-
We conduct extensive experiments across multiple datasets and different backbones. The results demonstrate that our approach leads to improvements in both fairness and generalization. For instance, the DFDC (14), DFD [1, and Celeb-DF 32 datasets, method outperforms several the state-of-the-art (SoTA) approaches. Our contributions are threefold:
To the best of our knowledge, we are the first to establish causal relationship where enhancing fairness leads improved generalization in deepfake detection. This finding reveals a www-ssttoo--ttttt-----eee- strategies that also enhance robustness.
strategy: It enables the development of fairness-aware
We propose novel approach that improves generalization by promoting fairness. Our method controls the confounders, thereby isolating the causal relationship between fairness and generalization and achieving improvement in both objectives.
We evaluate our approach on multiple datasets and backbones, showing consistent improve- ments in faimess and generalization. Code is provided in the supplementary materials.

---

2 Related Work
2.1 Deepfake Detection
Generalization in Deepfake Detection. Deepfake detection 21
72
67
37
68 25
is gener-
ally cast as a binary classification task. Preliminary efforts often endeavor to detect the specific manipulation traces [2 42 66 77]. which have shown certain improvements on intra-dataset setting. However, these methods often encounter inferior performance when applied to data with different
distributions manipulation methods. To address this generalization issue [60] 29] [1]. subsequent
research has increasingly devoted efforts to learning more generalized features [19 73 36 57 [9].
For instance, D&L [24] introduces novel framework that jointly leverages semantic and noise
cues to achieve SOTA deepfake localization performance. RealForensics [18] exploits the visual and auditory correspondence in real videos to enhance detection performance [8. MoE-FFD [26]
represents the first innovative framework that utilizes MoE modules to achieve superior deepfake detection with significantly reduced computational cost.
Fairness in Deepfake Detection. Fairness in deepfake detection pertains to potential biases against certain demographic groups [62] 20, 55), particularly in terms of race and gender A4 13] For instance, Pu et al. (50) evaluate the fairness of the detector McsoInception-4 and find it to be unfair to both genders. Some recent approaches (35) have been proposed to address this problem by chasing for improved fairness metrics. For instance, Ju et al. [23] mitigate sharp loss landscapes during training improve fairness within the same data domain. Lin al. (33 aims to enhance cross-domain fairness by leveraging contrastive learning across different demographic subgroups. Furthermore, several approaches [56 43] distributionally robust optimization to improve worst- group performance, thereby addressing fairness and robustness together. Nevertheless, these methods treat fairness the main optimization objective, without establishing clear connection between fairness and generalization. Our DAID framework is tailored to visual deepfake data for robust cross-domain deepfake detection. Leveraging fairnsss as causal intervention, DAID simultaneously boosts fairness and cross-domain robustness (generalization).
2.2 Causality Inference
In recent years, causal inference has emerged as powerful tool to uncover causal relationships [4] growing body of research confirms that robust causal identification lead to substantial
38 75
improvements in model performance [40] 41 76). Causal inference methods can be categorized into
The backdoor adjustment removes the confounding bias by stratifying the data according to the values of the confounders [78). Li et al. (30) leverage back-door adjustment to mitigate inter- and intra-modal confounding, resulting in improved image- text matching accuracy. Chen et al. ? apply back-door causal intervention to neutralize the textual bias to detect fake news. In contrast, the front door adjustment recovers the causal effect of a treatment by conditioning an observed mediator that fully carries the influence of the treatment on the outcome [6). For instance, Zhang el al. (74 employ L.M--generatdd prompts as a mediator and calculate the causal effect between prompts and responses. In this paper, we apply back-door adjustment to block the influence of confounders, thus demonstrating the causal relationship between fairness and generalization.
back-door and front-door adjustment
49 48
3
Causal Analysis Between Fairness and Generalization
3.1 Causal Relationship Construction
Causal Graph. Figure D illustrates our assumed causal structure directed acyclic graph (DAG) over four variables: fairess (F), generalization performance (A), data distribution (DD), and model capacity (MC). F serves as a binary treatment variable: 'low fairness' vs. 'high fairness', based on the absolute value of Skew metric (smaller Skew indicates greater fairness). A is the testing-set AUC, reflecting the generalization capability. DD captures the distribution of sensitive attributes (e.g., race, gender), while MC denotes the model's architectural capacity (e.g., the number of parameters and performance on benchmarks). Since DD and MC influence both F and A, we must control for them to isolate the causal effect of fairness generalization.
3

---

This DAG contains two types of paths: i) Causal path: F + A represents our hypothesis that improving fairness boosts generalization; ii) Confounding paths: DD A), MC - (F. where data distribution and model capacity each affect both faimess and generalization. Confounding paths that simultaneously influence both and A, such as F <- DD + A and F MC A, can induce back-door effect, introducing a spurious association between F and A.
Therefore, is essential to block these back-door effects for recovering the true causal effect of F on A. To this end, we apply the back-door adjustment [46]. Specifically, if there exists a set of variables Z that satisfies the back-door criterion, we can estimate the causal relationship by conditioning on Z.
Definition (Back-door Criterion) Let & be a causal DAG and let X and Y be two nodes in G. A set of variables Z satisfies the back-door criterion relative X, Y.Y.o. if:
1. No element of Z is a descendant of X EGG
2. blocks every path between and Y that begins with an arrow pointing into X. In this study, Z is defined to include both the data and the model factors, i.e., 2 (DD,
Theorem (Back-door Adjustment Formula) If set Z satisfies the back-door criterion relative to X,Y in G, then the causal effect of X on Y is identifiable and given by:
Here, do(X =x) denotes an intervention that forcibly sets X I, disconnecting it from its natural causes. This allows us to distinguish causal effects from spurious associations in observational data. Theorem [ demonstrates that as long the conditional distribution I( X, Z) and the marginal distribution of the confounder set P(Z)) can be observed, the causal effect can be identified without experimental randomization, In our context, if the influence of varying fairness levels F on generalization performance A remains consistent when conditioned on different values of DD and MC, then a direct causal relationship between fairness and generalization can be established.
3.2 Causal Effect Estimation
According to the back-door criterion, adjusting for Z = (DD, suffices:
P(A do(F=f)) = Σ F(A dd P(DD dd, ddddMMM (CC=mc), (2)
dd
where J, dd, and me represent the values of F, DD, and MC, respectively For simplicity,
discretize the two levels fairness with a binary variable (0, 1}, where f = denotes low fairness.
To examine the causal effect of F on A, we define the Average Causal Effect (ACE) (55) as follows:
ACE = (AA| do = 1)) - P(A do(F = 0))
Σ
P(A 1, me) - P(A| F = 0, dd, mc) P(dd, me).
dd,mc
In other words, the causal effect is defined as the weighted average of the performance differences observed between high and low faimess conditions within each subgroup. Moreover, define
μi = ((A do(F = 0)). for any fairness level f. we can apply a simple substitution:
P(A do n) = pn + f. |P = 1)) PPA do 0))
ACE
pn + f ACE.
This leads to a straightforward linear formulation: When f 0, we have IP(A| do = 0)) = μo- When f = 1, we have P(A) do(F 1)) = μo + ACE. As long as ACE + 0, we can assert that
We approximate P(DD, MC) an i.i.d. sample from the deployment population.
MC) by the empirical frequency in the held-owt test set, assuming that this set is
'It is worth noting that the confounding factors may vary depending on the task setting, potentially expanding beyond DD and M Nevertheless, this does not affect the applicability of Equation??

---

Demographh
Demographic Attributes
Weight
Rebalanced Classifier
Backbone
Feature
Feature h
Real
Deepfake Dataset
Demeggraphic-tpnoeee Feature Agpregation
Fake
Feature h1
Feature
Lumr
ht
Xy
Figure 2: Overview of the proposed DAID method. Top: Demographic-awart Data Rehalancing. We utilize human attributes perform demographic normalization and classifier rebalancing, which suppresses the confounding effects of DD Bottom: Demographic-Agnosti We introduce demographic-agnosti loss that enhances the model's is ability filter out demographic- related information, which mitigates the confounding influence of M while improving fairness.
Feature Aggregation.
fairness F has a causal effect on generalization performance A: ACE > 0 implies that improving fairness leads to better model performance, and ACE 0 indicates the opposite.
We further design a concrete experiment to estimate the ACE to establish the causal relationship between fairness and generalization (more details are provided in the supplementary materials).
Confounder Stratification. For DD, we stratify the dataset based on the intersection of gender and race. Specifically, the dataset is first divided into two groups according to binary gender: Male and Female. Within each gender group, samples are further categorized by skin tone into three subgroups: White, Black, and Asian. Each intersection of gender and race is treated as distinct demographic distribution. For MC, we employ two different architectures: Xception (4 (lower capacity) and EfficientNet [6] (higher capacity), the latter of which is known for stronger cross-domain performance [72.
Fairness Intervention (d)( We implement two training regimes to approximate do(F= = 0) and do(F 1) 47 1) Low faimess (F 0): Standard cross-entropy training. 2) High faireess (F = 1): Cross-entropy loss with a simple resampling strategy [9] where each sample in the cross-entropy loss is assigned a weight to suppress the rvereeepresnntation of majority groups.
ACE Estimation Results, Based on the above procedure, we observe an average ACE gain of 2.35 percentage points (stratified bootstrap resampling with B 1000, A 0.0235, 95% CI [0.0186, 0.0280], two-sided < 0.001). This result indicates that, after removing the influence confounders, direct relationship between fairness and generalization emerges.
3.3 Demographic Attribute-Insensitivee Intervention Detection
Motivated by our causal findings, we conclude that, as long as confounders are properly controlled, the clear causal pathway can be leveraged enhance generalization by intervening on more readily mea- surable fairness. Therefore, we introduce Demographic Attribute-Insensitvee Intervention Detection (DAID), a training approach that uses fairness interventions boost cross-domain generalization.
As illustrated in Figure 2 DAID counteracts two key confounders: data distribution (DD) and model capacity (MC) via two complementary modules: Demographic-aware Data Rebalancing, and ii) caoogrraphicAAnnosii Feature Aggregation.
Demographic-aware Data Rebalancing. To neutralize the spurious dependency induced by the data distribution confounder DD, our rebalancing module includes two key components: sample- wise reweighting and representation-level normalization, that jointly calibrate both the optimization direction and the feature space geometry
Firstly, we employ the probbbiiyy reweighting strategy. Let x, denote an input sample with sensitive demographic attributes s; (e.g. gender, race). To equalize the influence majority and minority groups, we compute a sample-specific importance weight:
II P (k((
Wi
(5)
5

---

where a/4) is the k-th sensitive attribute of x,, and P(s(") is the empirical marginal frequency estimated from the training data. This inverse propensity weighting ensures that the expected contribution of each demographic subgroup to the loss function is approximately uniform, thus suppressing spurious correlations between DD and the optimization target.
Beyond reweighting, we further mitigate DD- DD-induced feature shifts by normalizing latent features within each subgroup. Denote the feature vector for x; as h, For each DD group dd, we estimate the first and second moments:
Pdd = dd aa = ddd [h:];
and apply the following demographic-condicie conditoone normalization:
h; ad,
h:
+
This operation aligns the group-conditioned feature distributions, removing systematic shifts induced by demographic imbalance and restoring feature comparability across subgroups.
In summary, these strategies decouple the confounding influence of DD from both model updates and representation space, yielding unbiased learning that better reflect the intrinsic relationship between fairness (F") and generalization (A).
Bonographic-Agnosti Feature Aggregation. To eliminate the confounding influence of MC, we propose to encourage the model to focus on task-relevant cues while marginalizing residual demographic signals. Therefore, we perform demographic-inaarian representation space. The key intuition is that cumipulltiinnconsiittt samples, L.e, those with the same class label but differing sensitive attributes, should lead to similar internal representations.
optimization in the learned
Formally, let P = {(x,, xj)} be a set of sample pairs such that yi = Yj (same task label) and dd, # dd, (different demographic attributes). We define:
Cattr
Σ (i.5)EP
Leus Coaa, h,),
(8)
where h, and hy are normalized feature vectors, and (ooe(;,) denotes a cosine similarity loss: Lcos (h;, h2) cos (h;, hT) +e
where cos(-) denotes the cosine similarity between feature vectors. To ensure this alignment occurs in semantically meaningful subspace, factorize h Rd via low-rank projection layer:
U= h,
(10)
where U is trainable orthonormal basis, used filter out irrelevant directions. To avoid collapsing to trivial solutions, we regularize the projected features with:
Lottho = |IUT - II
(11)
where the identity matrix, and
||r denotes the Frobenius norm.
By enforcing demographic- invariant structure in a filtered representation space, this module sup- presses the model's reliance on demographic features, thereby neutralizing MC as a confounder and sharpening the causal interpretability of huiness-drivee generalization.
Training Objective. We adopt fully end-to-end optimization strategy that preserves the backbone architecture the base detector. Specifically, we only insert our proposed modules before the classification head. It is worth noting that our approach is model-agnostic and can be seamlessly integrated into various deepfake detection backbones, which ensures inference efficiency.
Let fo x + h denote the backbone encoder, and ye h + ý denote the binary classifier. Our total objective integrates the classification loss with two fairness-enhancing regularizers:
Liotal Lets + Aaur + Aortho Carthon
(12)
where Less = E(x, y) [w: ((go(fo(x)) y)] is the weighted binary cross-entropy loss over labels and sample-specific importance weight (see Equation (5)); Lattr enforces demographic-invariann alignment between same-label samples across subgroups (see Equation (8); Lortha ensures that
the projected representation remains compact and expressive (see Equation Aattr, Aortho are hyperparameters that modulate the contribution of each loss.

---

DFDC
DFD
Celeb-DF
Method
Skew 4
AUC
Skew
AUC
Skew 4
AUC f
Xception
2.221
60.63
80.69
0.597
70.91
EffcientNet
61
2,011
60.49
0.351
83.12
0.437
75.36
F-NNet 53
2.143
60.17
0.589
77.68
0.556
74.36
Face X-ray
28
1.982
62.00
0.821
80.46
0.491
74.20
SBI
2,385
63,39
0.757
86.43
0.715
79.76
RECCE 3
2.622
61.63
80.13
0.644
70.55
GRU [1]
2,432
62.63
0.551
86,48
0.405
76.00
CADDM 15
2,183
63.77
0.547
88.59
0.391
81.75
UCF 71
2,272
60.03
0.510
81.01
0.619
71.73
ProDet 10
2.306
65.89
89.18
0.569
82.71
VLFFD 58
2,411
65,21
0.669
90.08
0.526
81.17
!DAW-FDD 23
2.127
59.96
0.528
71.40
0.509
69.55
IFG [33
1,932
60.11
0.447
80.42
0.498
68.30
DAID
1.460
66.85
0.263
91.15
0.289
84.39
Table 1:
Frame-level cross-datases performance comparison on fairness and generalization of baselines and our approach. We reproduced all baselines on three datasets and reported their Skew and AUC values. 1; This method proposed enhance the fairness of the detector.
4 Experiments
4.1 Datasets and Metrics
Datasets. Following prior work [72 59 58 employed FuceForensiest+ (FF++) the training set and evaluate the generalization performance on three other datasets: DFDC [4], DFD [ and Celeb-DF 32 Since none of these datasets contain native demographic annotations, we follow the data processing, annotation protocol, and sensitive attribute intersection strategy of previous fairness studies [33 70 (3). Specifically, annotated each face with combination of gender and race attributes, resulting in six demographic subgroups: Male-Asian (M-A), Male-White (M-W), Male-Black (M-B), Female-Asian (F-A), Female-White (F-W), and Female-Blas (F-B).
Metrics. We used AUC as the primary metric to evaluate the generalizability of the model and adopted Skew as the fairness metric [6 64 Skew is commonly used indicator for measuring model fairness, which quantifies the performance disparity across different demographic subgroups. In our context, lower Skew value indicates better fairness, with Skew 0 representing perfectly fair predictions. The detailed computation of Skew is provided in the supplementary materials.
4.2 Implementation details
We used several deepfake detectors backbone models, including Xception 54 (=222.M pa- rameters), -eeNet (53) (2:37.3M parameters), EfficientNet-B4 [61] 1Ax.. parameters), and
CADDM 11 (==21 11.5M parameters), to evaluate the effectiveness of DAID. Training employs AdamW (learning rate x 10 weight decay 4 x 10-3) until convergence, with batch size of 64. All input images are resized to 224 x 224 and normalized using ImageNet statistics. All experiments are conducted on a single NVIDIA H100 GPU.
4.3 Main Results
In Table we reported comparison of our method, DAID, against several SoTA baselines in terms of both fairness and generalization performance. It can be seen that DAID consistently achieves the best results in all three datasets. For instance, on Celeb-DF, our method improves faimesss by 26% compared to the best best-performing baseline. On the DFDC and DFD datasets, DAID achieves AUC scores of 66.85% and 91.15% outperforming all competing methods. By controlling for confounding factors, we successfully achieve simultaneous improvements both fairness and generalization.
It can be observed that achieving a high AUC does not necessarily imply high faireess. For example, VLFFD attains an AUC of 90.08% on the DFD dataset. However, its fairness performance lagged behind that of UCF, which exhibits significantly lower generalizability than VLFFD but demonstrates better fairness as indicated by lower skew. Moreover, fairness-oriented methods, i.e., DAW-FDD and FG, effectively enhance the fairness of the model. Nevertheless, this improvement may come

---

Module
Dataset
Data Rebalancing
Feature Aggregation
DFDC
DFD
Celeb-DF
Reweight
Normalization
Latt
Loetho
Skew
AUC
Skew
1
Skew
AUC
2.183
63.77
0.547
88,59
0.391
81.75
1.719
64.94
0.295
89.63
0.340
83,07
1.574
65.96
0.274
90.67
0.319
83,98
1.750
65.40
0.273
89.38
0.327
83.59
1.715
64.96
0.271
89.55
0.321
83.88
1.495
66.49
0.266
91.05
0.292
84.12
1.460
66.85
0.263
91.15
0.289
84.39
Table 2: Performance of ablation studies on each module of DAID.
at the cost of reduced generalization. For instance, on the Celeb-DF dataset, FG outperforms most baselines in terms of fairness, yet its AUC score only around 68%, significantly lower than those achieved by other methods.
4.4
Ablation Studies
4.4.1
Comparison Modules
We reported the ablation studies the modules of our DAID in Table Specifically, we incrementally integrate each DAID module into the backbone model to assess their individual contributions. The results indicate that omitting any single module negatively impacts performance. For instance, removing the data rebalancing module, i.e., no longer controlling the confounding factor DD, leads to significant performance drop across all three datasets. Overall, the integration of all DAID modules yields the best performance both generalization and fairness.
4.4.2 Comparison Hyperparameters
We employ two hyperparameters, Aattr and Aortho- to control the relative weights of the corresponding loss functions. To investigate
Performance
Performance
their impact on model generalization, we
AUC 190
conducted parameter sensitivity analysis, with the results shown Figure [ As both parameters increase, model performance ini- tially improves and then stabilizes. Based on empirical observations,
86
0.2
0.4
0.5
0.8
1.0
0.05
0.30
0.15
0.20
0.25
0.30
select Anter 0.7 and Aortho 0.2 as default values. It worth
Autte
Aooto
Figure 3: Hyperparameter analysis.
noting that our method demonstrates robustness to hyperparameter selection.
4.4.3 Comparison on Demographic Attributes
In Figure we reported a radar plot that illus- trates the performance of the model the DFDC dataset at different intersections between gender and race, e.g., White-Female. The left subfig-
Masl Who
Mala Weke
TOTOR Bick
ure presents the AUC performance for evaluating generalization. Our DAID model outperforms the baseline across all six demographic intersec- tions, with particularly notable improvement on the Male-Asian subgroup. where AUC increases by 30%. The right subfigure assesses fairness via the Skew metric, where our model demonstrates significantly lower skew values. This indicates
Female:
Asiso
Aoniaaa Whlte
Mille Soom
CADIN DAID
Figure 4: Radar plot for DAID. Left: AUCt (%) for generalization. Right: Skew] for faimess.
that DAID achieves greater fairness in various demographic dimensions.

---

FF++
DFDC
DFD
Celeb-Di
Method
Skew
AUC
Skew J
AUC
Skew
AUC
Skew
AUC 1
Xception
S4
0.177
97.85
2.221
60.63
0.564
80.69
0,597
70.91
+DAID
0.122
98.64
1.772
63.36
0.398
82.54
0.467
75.23
EffcientNet 61
0.185
98.08
2.011
60.49
0.351
83.12
0.437
75.36
+DAID
0.136
98.72
1.697
63.43
0.264
84.31
0.352
78.49
Net [53
0.219
97.32
2.143
60.17
0.589
77.68
0.556
74.36
+DAID
0.127
97.63
1.544
62.68
0.220
78.53
0.541
76.54
CADDM 05
0,220
99.15
2.183
63.77
0.547
88.59
0.391
81.75
99.26
1.460
66.85
0.263
91.15
84.39
Table 3: Performance comparison after applying our DAID different backbones. All models are trained on the FF+ dataset and evaluated on four datasets. Our method consistently leads to significant improvements across all backbone architectures.
FF++
DFDC
Backbone
DAID
M-W
M-A
M-B
F-W
F-A
F-B
M-W
M-A
M-B
F-W
F-A
F-B
Figure 5:
Non-chemry-iccked Heatmaps. We included heatmaps for six demographic subgroups across two datasets: Male-Asian (M-A), Male-White (M-W), Male-Black (M-B), Female-Asian
(F-A), Female-White (F-W), and Female-Black (F-B).
4.4.4 Comparison on Backbones
Table presents the performance when applying the DAID) to different backbone architectures. Specifically, we compare the performance of the four backbones, i.e., Xception, EfficientNet, F Net, and CADDM. As shown in the table, our method consistently enhances both fairness and generalization across all backbones. For instance, on Celeb-DF, applying our DAID to the Xception backbone yields a 5% increase in AUC and nearly 20% improvement in faimess. It worth noting that this process does not require any architectural modifications to the model, leading to synergistic gains greater than the sum of individual improvements.
4.5 Visualization Results
In Figure we present the heatmap results of the backbone model without faireess enhancement and our proposed DAID method. It can be seen that the backbone exhibits markedly different attention regions for different attributes. For instance, it focuses primarily on the lips for male subjects, while emphasizing the upper faces for female subjects. Furthermore, within the same gender, subtle differ- ences in attention regions are also observed across different racial groups. For example, the backbone tends to focus more on the left side of the lips for the Male White group, whereas for the Male-Black group, the nose is more frequently included in the attention region. This indicates that the backbone model conflates demographic attributes with cues for deepfake detection, potentially undermining reliable decision-making In contrast, DAID demonstrates consistent detection patterns across both gender and race groups, effectively indicating that our method is insensitive to demographic attributes. Moreover, compared to the backbone, DAID generally focuses broader regions of the image, reflected in its superior generalization capability.
4.6 Efficiency Analysis
We assess the additional computation introduced by DAID's two modules on single NVIDIA H100 GPU (batch size 64, input resolution 224x224). For the data rebalancing module, the reweighting step adjusts only the classification loss based on subgroup frequencies, and subgroup-wisg feature normalization operates directly on batch statistics. Neither requires extra gradient computations beyond standard training, resulting in negligible run-time impact. For feature aggregation module, we introduce two regularization losses and a low-rank projection layer. These involve only light

---

matrix multiplications and loss evaluations, resulting in minimal extra cost. On EfficientNet, standard training takes 233 min for the full session. Incorporating DAID increases this 243 min relative overhead of 4.3%. Therefore, DAID's fairness-driven interventions add under 5% to total training time, making the framework practical for large-scale use.
5
Discussion
5.1 Why Fairness and Generalization May at Odds
The conflict between faimess and generalization may arise from both data and model characteristics. Pertaining to the data aspect, the most representative factor, i.e., imbalanced distribution can lead to increased bias toward the majority group, thus improving generalization under certain limited datasets or scenarios. This however, poses the fairness problem a great challenge. Even worse, existing models tend amplify this imbalance distribution problem, making the prediction biased. Unlike the existing methods, in this work, we propose to leverage the causal theory with the confounder controlling guidance. Informed by this, our proposed method in fact aims to rebuild balance from imbalance. Therefore, we can maintain the generalization capability of vanilla models, and can also improve the prediction fairness.
Conflicts may be difficult to resolve. For instance, in high-security systems, developers might prioritize reducing overall overall misclassification, thereby sacrificing the performance on minority groups (e.g., individuals wearing unusual clothing). In contrast, for judicial models such as sentencing decisions or crime risk prediction systems, fairness across different demographic groups must be prioritized, even if it comes at the cost of reduced generalizability.
5.2 Connection with General Fairness Definitions
The primary fairness goal of DAID is to reduce performance disparities across demographic subgroups in other words, to achieve form of group fairness. In terms of common definitions, this aligns most with pursuing equalized performance (e.g., smaller gaps in accuracy between groups), which what the used Skew metric captures. By making feature embeddings invariant to sensitive attributes, our method mitigates the model's s reliance on those attributes, thus helping to satisfy criteria related to demographic parity (outcomes independent of demographics).
5.3 Comprehensive General Proof for Causal Relationship
As introduced in Section D our causal claim regarding the relationship between fairness and gener- alization is defined with respect the causal graph presented in Figure This graph is built on the assumption that data distribution (DD) and model capacity (MC) constitute an exhaustive set of confounders. If additional confounding factors argument might be limited. Nevertheless, our experiments with DAID demonstrate that controlling for DD and MC is sufficient to effectively reveal positive causal relationship between fairness and generalization. In future work, we plan to investigate whether a comprehensive and fully general proof can be established to formally substantiate this causal relationship.
be identified, the generality of causal
6 Conclusion
In this paper, we demonstrate that improving fairness can causally lead to a better generalization in deepfake detection. Building on this insight, We propose the Demographic Attribute-nnsensitive Intervention Detection (DAID), novel plug-and-play approach that jointly ensures demographic fairness and generalization without modifying base architectures. Extensive experiments on various benchmarks validate the theoretical foundation and practical value of DAID. Our findings reframe faimeess from a mere ethical concern into a strategic lever for enhancing model robustness. By harnessing fairness as a means to improve generalization, we offer a new perspective and a practical path toward building more robust and equitable deepfake detectors. However, one limitation of our current framework is reliance on demographic annotations. Extending DAID to operate under unlabeled or
faimess settings remains important direction for future work.
10

---

Acknowledgments and Disclosure of Funding
This document supported by the Ministry of Education, Singapore, under its MOE AcRF TIER Grant (MOE-MOET33022-0001)
References
[1] Google Al Blog. Contributing data to deepfake detection rescarch, 2019. URL hhttps///aa googleblog 1ttp://2199/0//ootrro111/0
data toad-topeapaakeee detection. html
[2] Joy Buolamwini and Timnit Gebru. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountubility and Iranspurency, pages 77-91, 2018.
[3] Junyi Cao, Chao Ma, Taiping Yao, Shen Chen, Shouhong Ding, and Xiaokang Yang. End-to-end reconsiaction-laasffcttoonn learning for face forgery detection. In CVPR, pages 4103-4112, 2022.
[4] Krzysztof Chalupka, Frederick Eberhardt, and Pietro Perona. Causal feature learning: an overview. Behaviormetrika, 44:137-164, 2017.
[5] Liang Chen, Yong Zhang, Yibing Song. Linggiao Liu, and Jue Wang. Self-supervised learning of adversarial example: Towards good generalizations for deepfake detection. In CVPR, pages 18689-18698, 2022.
[6] Tieyuan Chen, Huabin Liu, Tianyao He, Yihang Chen, Chaofan Gan, Xiao Ma, Cheng Zhong, Yang Zhang, Yingxue Wang, Hui Lin, et al. Meed: Unlocking multi-event causal discovery in video reasoning. NeurIPS, 77:92554-995588 2024.
[7] Ziwei Chen, Linmei Hu, Weixin Li, Yingxia Shao, and Liqiang Nie. Causal intervention and counterfactual reasoning for multi-modal fake news detection. In ACL, pages 627-638, 2023,
[8] Harry Cheng, Yangyang Guo, Tianyi Wang, Qi Li, Xiaojun Chang, and Liqiang Nie. Voice-face homogeneity tells deepfake. ACM Transactions on Multimedia Computing, Communications and Applications, 20(3): 1-22, 2023.
[9] Harry Cheng, Yangyang Guo, Qingpei Guo, Ming Yang, Tian Gan, and Liqiang Nie. Social debiasing for fair multi-modal llms. arXiv preprint arXiv:2408.065:: 2024.
[10] Jikang Cheng, Zhiyuan Yan, Ying Zhang, Yuhao Luo, Zhongyuan Wang, and Chen Li. Can leave deepfake data behind in training deepfake detector? In NeurlPS, pages 1-12, 2024.
[11] Jongwook Choi, Tachoon Kim, Yonghyun Jeong. Seungryul Back, and Jongwon Choi. Exploit- ing style latent flows for generalizing deepfake video detection. In CVPR, pages 1 1133-1143, 2024.
[12] Ramon Correa, Mahtab Shaan, Hari Trivedi, Bhavik Patel, Leo Anthony G Celi, Judy W Gichoya, and Imon Banerjee. A systematic review of 'fair' ai model development for image classification and prediction. Journal of Medical and Biological Engineering, 42(6):816-827, 2022.
[13] Feng Ding, Jun Zhang, Xinan He, and Jianfeng Xu. Fairadapter: Detecting ai ai-generated images with improved fairness. In /CASSP, pages 1-5, 2025.
[14] Brian Dolhansky, Joanna Bitton, Ben Pflaum, Jikuo Lu, Russ Howes, Menglin Wang, and Cristian Canton-Ferrer. The deepfake detection challenge dataset. CoRR, pages 1-13, 2020.
[15] Shichao Dong, Jin Wang. Renhe Ji, Jiajun Liang, Haoqiang Fan, and Zheng Ge. Implicit identity leakage: The stumbling block to improving deepfake detection generalization. CVPR, pages 3994-4004, 2023.
[16] Sahin Cem Geyik, Stuart Ambler, and Krishnaram Kenthapadi. Fairness-aware ranking in search & recommendation systems with application to linkedin talent search. In KDD, pages 2019.

---

[17] Weinan Guan, Wei Wang, Jing Dong, and Bo Peng. Improving generalization of deepfake detectors by imposing gradient regularization. IEEE TIFS. 19:5345-5356, 2024.
[18]
Alexandros Haliassos, Rodrigo Mira, Stavros Petridis, and Maja Pantic. Leveraging real talking faces via self-supervision for robust forgery detection. In CVPR, pages
2022.
[19] Yue-Hua Han, Tai-Ming Huang, Shu-Tzu Lo, Po-Han Huang, Kai-Lung Hua, and Jun-Cheng Chen. Towards more general video-based deepfake detection through facial feature guided adaptation for foundation model. CVPR, 2025.
[20] Caner Hazirbas, Joanna Bitton, Brian Dolhansky, Jacqueline Pan, Albert Gordo, and Cristian Canton-Ferrer. Towards measuring faimess in AI: the casual conversations dataset. IEEE TBIOM, 4(3):324-332, 2022.
[21] Cheng- Yao Hong, Yen-Chi Hsu, and Tyng-Luh Liu. Contrastive learning for deepfake classifi- cation and localization via multi-label ranking. In CVPR, pages 17627-17637, 2024.
[22] Shuai Jia, Chao Ma, Taiping Yao, Bangjie Yin, Shouhong Ding, and Xiaokang Yang. Exploring frequency adversarial attacks for face forgery detection. In CVPR, pages 4093-4102, 2022.
[23] Yan Ju, Shu Hu, Shan Jia, George H Chen, and Siwei Lyu. Improving faimesss in deepfake detection. In WACV, pages 4655 4655-4665, 2024.
[24] Chenqi Kong, Baoliang Chen, Haoliang Li, Shiqi Wang, Anderson Rocha, and Sam Kwong. Detect and locate: Exposing face manipulation by semantic-and noise-level telltales. IEEE TIFS, 17:1741-1756, 2022.
[25] Chenqi Kong, Anwei Luo, Peijun Bao, Haoliang Li, Renjie Wan, Zengwei Zheng, Anderson Rocha, and Alex Kot. Open-set deepfake detection: a parameter-efficien adaptation method with forgery style mixture. arXiv preprint urXiv:2408. 12791, 2024.
[26] Chenqi Kong, Anwei Luo, Peijun Bao, Yi Yu, Haoliang Li, Zengwei Zheng, Shiqi Wang, and Alex Kot. Moe-ffd: Mixture of experts for generalized and parameter-efficient face forgery detection. IEEE TDSC, 2025.
[27] Lingzhi Li, Jianmin Bao, Hao Yang, Dong Chen, and Fang Wen. Advancing high fidelity identity swapping for forgery detection. In CVPR, pages 5073-5082, 2020.
[28] Lingzhi Li, Jianmin Bao, Ting Zhang, Hao Yang, Dong Chen, Fang Wen, and Baining Guo. Face x-ray for more general face forgery detection. CVPR, pages 5000-5009, 2020.
(29] Shuang Li, Fan Li, Jinxing Li, Huafeng Li, Bob Zhang, Dapeng Tao, and Xinbo Gao. Logical relation inference and multiview information interaction for domain adaptation person re- identification. IEEE TNNLS, 2023.
[30] Wenhui Li, Xinqi Su, Dan Song, Lanjun Wang, Kun Zhang, and An-An Liu. Towards decon- founded image-text matching with causal inference. ACM MM, pages 6264-6273, 2023.
[31] Yuezun Li, videos by detecting eye blinking. In WIFS, 2018
Chang, and Siwei Lyu. In ictu oculi: Exposing ai generated fake face
[32] Yuezun Li, Xin Yang, Pu Sun, Honggang Qi, and Siwei Lyu. Celeb-df: large-scale challenging dataset for deepfake forensics. In CVPR, pages 3204-3213, 3213, 2020.
[33] Li Lin, Xinan He, Yan Ju, Xin Wang, Feng Ding, and Shu Hu. Preserving fairness generalization in deepfake detection. In CVPR, pages 16815-16825, 2024.
[34] Mingquan Lin, Tianhao Li, Yifan Yang, Gregory Holste, Ying Ding, Sarah H Van Tassel, Kyle Kovacs, George Shih, Zhangyang Wang. Zhiyong Lu, al. Improving model fairness in basge computer-aided diagnosis. Nature communications, 14(1):6261, 2023.
[35] Decheng Liu, Zongqi Wang, Chunlei Peng, Nannan Wang, Ruimin Hu, and Xinbo Gao. Thinking racial bias in fair forgery detection: Models, datasets and evaluations. AAAI, pages 5379-
5387, 2025.
12

---

[36] Ming-Hor Hui Liu, Harry Cheng, Tianyi Wang, Xin Luo, and Xin-Shun Xu. Learning real facial concepts for independent deepfake detection. arXiv preprint arXiv:2505.04480 00.50000000, 2025.
[37] Ming-Hul Liu, Xiao-Qias Qian Liu, Xin Luo, and Xin-Shun Xu. Data: Multi sulttiddsenttngemment based contrastive learning for open-wonee world semi-supervised deepfake attribution. arXiv preprint arXiv:2505.043844 2025.
[38] David Lopez-Paz, Robert Nishihara, Soumith Chintala, Bernhard Scholkopf, and Léon Bottou. Discovering causal signals in images. In CVPR, pages 6979-69999999 6979969972228 2017.
[39] Yuchen Luo, Yong Zhang, Junchi Yan, and Wei Liu. Generalizing face forgery detection with fighenryy features. In CVPR, pages 16317-16326, 2021.
[40] Fangrui Lv, Jian Liang, Shuang Li, Bin Zang, Chi Harold Liu, Ziteng Wang, and Di Liu. Causality inspired representation learning for domain generalization. In CVPR, pages 8046- 8056, 2022.
[41] Divyat Mahajan, Shruti Tople, and Amit Sharma. Domain generalization using causal matching. In ICML, pages 7313-7324, 2021.
[42] lacopo Masi, Aditya Killekar, Royston Marian Mascarenhas, Shenoy Pratik Gurudatt, and Wael AbdAlmageed. Two-branch recurrent network for isolating deepfakes in videos. In ECCV, pages 667-684, 2020.
[43] Debarghya Mukherjee, Mikhail Yurochkin, Moulinath Banerjee, and Yuekai Sun. Two simple ways to learn individual faimeess metrics from data. In ICML, pages 7097-7107, 2020.
[44] Aakash Varma Nadimpalli and Ajita Rattani. GBDF: gender balanced deepfake dataset towards fair deepfake detection. In ICPR Workshop, volume 13644, pages 320-337, 2022.
[45] Sungho Park, Jewook Lee, Pilhyeon Lee, Sunhee Hwang, Dohyung Kim, and Hyeran Byun. Fair contrastive learning for facial attribute classification. In CVPR, pages 10379-10388, 2022.
[46] Judea Pearl. Causality: Models, Reasoning and Inference. Cambridge University Press, 2009. ISBN 052189560X.
[47] Judea Pearl. The do-calculus revisited. In UAI, pages 3-11, 2012.
[48] Judea Pearl. Does obesity shorten life? or is it the soda? on non-manipulable causes. Journal of Causal Inference, 6(2):20182001. 2018.
[49] Judea Pearl, Madelyn Glymour, and Nicholas Jewell. Causall inference statistics: A primer. John Wiley & Sons, 2016
[50] Muxin Pu, Meng Yi Kuan, Nyee Thoang Lim, Chun Yong Chong, and Mei Kuan Lim. Fairness evaluation in deepfake detection models using metamorphic testing. In MET@ ICSE, pages 7-14, 2022.
[51] Zhuang Qi, Lei Meng, Zitan Chen, Han Hu, Hui Lin, and Xiangxu Meng. Cross-silo prototypical calibration for federated learning with non-iid data. In ACMMM, pages 3099-3107, 2023,
[52] Zhuang Qi, Lei Meng, Zhaochuan Li, Han Hu, and Xiangxu Meng. Cross-silo feature space alignment for federated learning on clients with imbalanced data. In AAAI, pages 19986-19994, 2025.
[53] Yuyang Qian, Guojun Yin, Lu Sheng, Zixuan Chen, and Jing Shao. Thinking in frequency: Face forgery detection by mining frequency-awar clues. In ECCV, pages 86-103, 2020.
[54] Andreas Rössler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Mathhis Nießner. Faceforensics*#: Learning to detect manipulated facial images. In ICCV, pages 1-11, 2019.
[55] Joseph Schafer and Joseph Kang. Average causal effects from nonrandomized studies: practical guide and simulated example. Psychological methods, 13(4):279, 2008.
13

---

[56] Shubham Sharma. Alan H. Gee. David Paydarfar, and Joydeep Ghosh. Fair-n: Fair and robust neural networks for structured data. In AIES, pages 946-955. ACM, 2021.
[57] Kaede Shiohara and Toshihiko Yamasaki. Detecting deepfakes with self-blended images. In CVPR, pages 18699-18708, 2022.
[58] Ke Sun, Shen Chen, Taiping Yao. Ziyin Zhou, Jiayi Ji, Xiaoshuai Sun, Chia-Wen Lin, and Rongrong Ji. Towards general visual-linguistic face forgery detection. arXiv preprint arXiv:2502.20669, pages 1-10, 2025.
[59] Zhimin Sun, Shen Chen, Taiping Yao, Ran Yi, Shouhong Ding, and Lizhuang Ma. Rethinking open-world deepfake attribution with multi-perspective sensory learning. IJC,, pages 1-24, 2024.
[60] Chuangchuang Tan, Huan Liu, Yao Zhao, Shikui Wei, Guanghua Gu, Ping Liu, and Yunchao Wei. Rethinking the up-sampling operations in enn-based generative network for generalizable deepfake detection. In CVPR, pages 281 28130-28139 2024.
[61] Mingxing Tan and Quoe V. Le. Efficientnet: Rethinking model scaling for convolutional neural networks. ICML, volume 97, pages 6105-6114, 2019.
[62] Loc Trinh and Yan Liu, An examination of fairness of models for deepfake detection. In IJCAI, pages 567-574,202
[63] Chengrui Wang and Weihong Deng. Representative forgery mining for fake face detection. In CVPR, pages 14923-14932, 2021.
[64] Mei Wang and Weihong Deng. Mitigating bias in face recognition using skewness- reinforcement learning. In CVPR, pages 1319-9328 2020.
[65] Tianyi Wang, Xin Liao, Kam Pui Chow, Xiaodong Lin, and Yinglong Wang. Deepfake detection: comprehensive survey from the reliability perspective. ACM CSUR, --53) 2024.
[66] Xi Wu, Zhen Xie, YuTao Gao, and Yu Xiao. Sstnet: Detecting manipulated faces through spatial, steganalysis and temporal features. In ICASSP, pages 2952-29552 2020.
[67] Ruiyang Xia, Decheng Liu, Jie Li, Lin Yuan, Nannan Wang, and Xinbo Gao. Mmnet: multi- collaboration and multi-supervision network for sequential deepfake detection. IEEE TIFS, 2024.
[68] Ruiyang Xia, Dawei Zhou, Decheng Liu, Lin Yuan, Jie Li, Nannan Wang, and Xinbo Gao. Towards generalized proactive defense against face swappingwith contour- hybrid watermark. arXiv preprint arXiv:2505. 19081, 2025.
[69] Chao Xu, Jiangning Zhang, Miao Hua, Qian He, Zili Yi, and Yong Liu. Region-aware face swapping. In CVPR, pages 7622-7631, 2022.
[70] Ying Xu, Philipp Terhörst, Marius Pedersen, and Kiran Raja. Analyzing fairness in deepfake detection with massively annotated databases. IEFE Transactions on Technology and Society, 5 (1):93-106, 2024.
[71] Zhiyuan Yan, Yong Zhang, Yanbo Fan, and Baoyuan Wu. UCF: uncovering common features for generalizable deepfake detection. In ICCV, pages 22355-22366. 2023.
[72] Zhiyuan Yan, Yuhao Luo, Siwei Lyu, Qingshan Liu, and Baoyuan Wu. Transcending forgery specificity with latent space augmentation for generalizable deepfake detection. In CVPR, pages 8984-8994, 2024.
[73] Zhiyuan Yan, Yandan Zhao, Shen Chen, Mingyi Guo, Xinghe Fu, Taiping Yao, Shouhong Ding, and Li Yuan. Generalizing deepfake video detection with plug-and-play. Video-level blending and spatiotemporal adapter tuning. CVPR, 2025.
[74] Congzhi Zhang, Linhai Zhang, Jialong Wu, Yulan He, and Deyu Zhou. Causal prompting: Debiasing large language model prompting based on front-door adjustment. In AAAI, pages 25842-25855 2025.
14

---

[75] Dong Zhang. Hanwang Zhang, Jinhui Tang, Kian-Sheng Hua, and Qianru Sun. Causal interven- tion for
semantic segmentation. NeurIPS, 33:655- 666, 2020.
[76] Haoyu Zhang, Meng Liu, Yuhong Li, Ming Yan, Zan Gao, Xiaojun Chang, and Liqiang Nie. Attribute-guided collaborative learning for partial person re-identification. IEEE TPAMI, 45 (12):4414-14100) 2023.
[77] Haoyu Zhang, Meng Liu, Zixin Liu, Xuemeng Song, Yaowei Wang, and Liqiang Nie. Multi- factor adaptive vision selection for egocentrie video question answering. In ICML, volume 235, pages 59310-59328, 2024.
[78] Shengyu Zhang. Tan Jiang, Tan Wang, Kun Kuang, Zhou Zhao, Jianke Zhu, Jin Yu, Hongxia Yang. and Fei Wu. Devlbert: Learning deconfounded visio-linguistic representations. In ACMMM, pages 0177-4382,, 2020.
[79] Hanqing Zhao, Wenbo Zhou, Dongdong Chen, Tianyi Wei, Weiming Zhang, and Nenghai Yu. Multi-attentional deepfake detection. CVPR, pages 2185-2194, 2021.
[80] Yipin Zhou and Ser-Nam Lim. Joint audio-visusa visual deepfake detection. In ICCV, pages 14800- 14809, 2021.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The concluding paragraphs of both the abstract and the introduction explicitly enumerate and elaborate on the three contributions of this work.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations in the final section of the paper.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
16

---

Justification: Complete proofs for all the theorems and formula definitions are provided in the appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Sections 3 and 4 describe all the methods and implementation details.
Guidelines:
The answer NA means that the paper does include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
17

---

Answer: [Yes]
Justification:
The code will be uploaded as supplementary material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https guwdes/ddeeeeee Cooeeeoodeesssseeeeeee //nips.com htce///bbb//uudddd///eeeeeueisss ssioiolicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification: All training and testing details are provided Section 4.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Significance analysis is incorporated. Specifically, Section 3.2 presents 95% confidence intervals to quantify uncertainty.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
18

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπхо rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Sections 4.2 and 4.6 report the hardware environment and runtime efficiency.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform. in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
There are no ethical concerns, provided that the guidelines have been read and followed.
Justification:
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations in their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: This paper addresses the issues of fairness and generalization in deepfake detection and does not pose any significant negative societal impact.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
19

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: Our method does not involve generative models or the construction of high-risk datasets.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All external resources properly cited.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the Datasets.papersstt paperswi thcode. cww/ datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
20

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
The new code is accompanied by documentation.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/cood/mmodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: No subjective human studies.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
No subjective human studies.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
21

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: LLMs are used solely to write and format the manuscript and are not involved in the core technical contributions.
Guidelines:
The answer NA
that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
22

---

A Skew Calculation
To assess fairness at the subgroup level, introduce log-ratio skew metric that quantifies the deviation between predicted and ground-truth label distributions across demographic attributes. We compute this skew separately for the real and fake classes, across both marginal (e.g., gender, race) and intersectional (e.g., female-Asian) groups.
A.1 Definition
Given subgroup S and class label {real, fake), we define the skew as:
P c P(y = c
Skew(s, c) = log
(13)
where (P(y = c s)) denotes the empirical proportion of samples with ground-trun truth label in group 8, and (P = e 8)) denotes the corresponding proportion in model predictions. This skew reflects the relative distortion introduced by the model's predictions:
Skew(s, e) > group s overrepresented in predicted class c, Skew(s, < 0: group is underrepresented in predicted class ck = 0: perfect parity between prediction and ground truth.
To capture extreme disparities, detine:
maxskew max (Skew(s, c).;
(14)
.ES,c
The maxskew metric corresponds to the Skew metric reported in paper. It measures the degree of bias in the most skewed group, regardless whether that group is overrepresented or underrepre- sented. A lower value Skew indicates lower level of the most severe group bias and thus reflects fairer model overall.
A.2 Implementation Summary
The calculation procedure is summarized in Algorithm [ It is applied to both marginal groups (gender and race) and intersectional groups (gender & race).
Algorithm Skew Computation for Each Demographic Group
Require:
Ground-truth labels y. predicted probabilities p. binary predictions ÿ. demographic at- tributes gender, race
1: Convert predictions: ÿt = Ill;; > 0.5]
2: for each group E S do
3: for each class E {real, fake] do
4: Compute: P(y
s) from ground-truth labels
5: Compute: P = 8) from predicted labels
6: Compute: Skew(s. loog P()=cls) (()=ds)
7: end for
8: end for
9: Collect: maxskew, minskew across all
We discard any subgroup for which P(y s) 0, avoid numerical instability.
A.3 Illustrative Example
Consider scenario where Female-Asia individuals constitute 10% of all real samples, but the model predicts 20% of real instances as belonging to that group. Then:
0.20
Skew(female-Asian, real) log
log(2) 0.693,
(15)
0.10
indicating an overrepresentation of that subgroup in the predicted class. Conversely, a skew of -0.693 would indicate underrepresentation.
23

---

Strr
Femil
Mady Aenn
Veroow Fremale Srriee
pear due Menie
Final Jaan Farnale soite
comak
Adde
CAGCH D6D
105p
Figure 6: The performance of CADDM and DAID in terms of AUC (left) and Skew (right) across all demographic groups and their intersections.
A.4 Application and Utility
This skew metric is used to:
Audit group-level disparities in prediction outcomes;
Reveal intersectional bias not captured by marginal analysis;
Guide fairness-aware model interventions and reweighting strategies.
Figure 6 presents the fairnsss and generalization performance of CADDM and DAID across all groups, revealing a substantial improvement the consistency of DAID.
B Details for Causal Effect Estimation
B.1 Experiment Settings
Data Stratification (Controlling DD). Following prior research [33, we apply the FF++ dataset 54 and the DFDC dataset [14] as the training set and the testing set, respectively. Both datasets are augmented with additional demographic annotations [70], i.e., they contain diverse demographic attributes. In our training-testing pipeline, the entire training set is used for model training, following standard practices (63 39.. The testing set, on the other hand, stratified based on the intersection of gender and race. Specifically, the dataset is first partitioned based on binary gender into Male and Female groups. Within each gender group, samples are further categorized according to skin tone into three subgroups: White, Black, and Asian. Each intersection of gender and race is treated as distinct distribution dd DD, with its proportion can be computed using conditional probability:
P(dd = (g,, r:)) P(rslgs) x P(g:),
(16)
where the 9: and r; are the value of gender and race, respectively.
Model Capacity (Controlling MC). We employ two different architectures: Xception [54] and EfficientNet [61] to simulate the effect of model capacity. The latter, EfficientNet, has demonstrated superior generalization performance 72) and therefore serves the representative of a complex model.
Fairness Intervention (do(F)). We create two fairness levels by in-processing de-biasing [47 Specifically, we employ standard cross-entropy training to conduct the training process with low-fairness (flo)):
Liow
yi log(ū.) - (1 y1) log(1 - 9.)
(17)
where yi 1) and û E (0, 1) are the ground truth and predicted labels for the i-th sample, respectively. Moreover, we adopt a simple resampling strategy (9), where each sample the cross- entropy loss is assigned a weight to suppress the over-representation of majority groups, to formulate

---

Faimess Level
Method
Male Black
Male_Asian
Female,
Female Asian
foo
2.22
56.61
64.19
44.95
58.98
58.35
60.91
Xception
faigh
2.06
61.74
62.86
49.46
65.83
58.81
62.34
EfficientNet
Row
2.01
55.36
65.00
37.32
53.35
60.49
51.36
fnigh
1.91
53.51
65.29
39.44
60.89
61.81
74.95
Table 4: Observed model AUC (%) under different data distribution (DD) and model capacity (MC). The 'Skew' column represents the fairness metric of the model, where lower values indicate better fairness.
the high-fairness (Shigh) version:
N
Snigh E0- 1- log(ū:) v1) log(1 - 0;]]]
(18)
where A; is weighting factor defined based the proportion of specific group in the dataset. For example, if the subgroup with dd = (male, white) constitutes 70% of the dataset, then all 'male-white' samples assigned weight of 0.3 during training.
Measurement and Computation. To estimate the causal effect of our bias intervention, we first compute the conditional AUC within each stratum defined by DD and MC factors. Specifically, for each combination (dd, mc) and for each bias setting f E {flow fuihh, we evaluate the empirical stratum AUC on the held-out testing dataset as
Aad.mo
P(A F= DD dd, MC =mc).
(19)
This is obtained by selecting all test samples whose sensitive-atriibute stratum equals dd and whose model architecture equals m.. then measuring the fraction correctly classified under the fairness intervention f. Subsequently, we compute the marginal weight each stratum from the full test set,
mc = P(DD = dd, MC = me).
(20)
The wed,me is the proportion test samples that fall into stratum (dd, mc). We then apply the back-door adjustment formula recover the interventional AUC under each fairness level:
A(f)
Σ
Aad.me Wdd.me-
(21)
dd, me
From Equation 21 Wad,me to emulate the causal effect of setting F = for the entire population.
sum the stratum accuracies (J) weighted by their marginal frequencies
Finally, we perform a causal comparison between the two faimess settings. Specifically, we compare Á(tow) against Á(Jigh). A statistically significant increase in Á when moving from the low-fairness to the high-fairness model provides strong empirical evidence that improving fairness causally improves overall AUC.
B.2 Numerical Illustration
Table 5 lists the observed AUCs under every combination of DD and MC. We then aggregate each stratum using Equation 21) and obtain A = 52.08% and Angg = 53 38%.. Stratified bootstrap resampling (B 1000) further shows that moving from the low-fairness to the high-farrness model yields an average gain of 2.35 percentage points (A 0.0235, 95 % CI [0.0186, 0.0280], two- sided p < 0.001). Thus, irrespective of DD or MC, higher fairness consistently translates into better generalisation. Combining (i) the rigorously specified DAG, (ii) back-door adjustment for identification, and (iii) stratified empirical estimates under controlled bias interventions, we obtain clear, quantitative evidence that reducing model bias causally improves overall performance.
25

---

Method
ACC
TPR
FPR
DAW-FDD
57.89
60.69
43.33
FG
61.27
65,85
40.73
DAID
64.99
72.62
38.34
Table 5: Comparison of different methods ACC, TPR, and FPR.
More Experiments
C.1
Other Metrics
We have conducted additional metrics such as Accuracy (ACC), True Positive Rate (TPR), and False Positive Rate (FPR). Our method significantly outperforms previous fairness approaches.
C.2
Hyperparameter and Fairness
Aattr
Aortho
Value
Performance
Value
Performance
(Reference)
1.574
(Reference)
1.495
0.1
1.559
0.05
1,483
0.2
1.541
0.1
1.475
0.3
1.529
0.15
1.464
0.4
1.513
0.2
1.460
0.5
1.509
0.25
1.462
0.6
1.497
0.3
1.461
0.7
1.495
0.8
1.496
0.9
1.498
1.0
1.497
Table 6: Ablation Aattr and Northo
We have added the results between the two hyperparameters for loss functions and skew in Table 6. Overall, improvements in faimess are generally positively correlated with improvements in generalization. The best performance achieved when Aattr reaches 0.7 and Aortho reaches 0.2.
26


Reviews:
[{"review_id": "63N1sCI9Qy", "reviewer": "Reviewer_4jNC", "summary": "The paper motivates that demographic fairness and generalization to unseen data are not always at odds with each other. Based on this motivation, the paper proposes DAID to improve fairness between demographic groups and generalization to unseen data on deepfake detection models. Experiments show the effectiveness of the method.", "questions": "Please refer to weaknesses", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- I like this work. The method proposed is well explained, and useful for the deepfake detection problem. The problem being addressed is an important one.\n- Experiments are thorough: Beyond basic experiments, effects of hyperparameters is shown. The comparison on demographic attributes is also interesting.\n\nWeaknesses:\n- This paper needs a discussion on why exactly fairness and generalization may be at odds with each other. Specifically, when is fairness and generalization at odds with each other? Is this a characteristic of the data, the model, or both? When can we observe that these results will hold and when is it simply not feasible to improve in both dimensions? I would like to see a detailed discussion on this.\n- There are several works in the fairness and robustness literature in general, eg., [1], targeted towards tabular dataset that shows models can be more robust and fair. While robustness in [1] is considered as adversarial robustness, more adversarially robust models could improve generalization. I think its important to discuss works in fairness and robustness and draw a (qualitative) comparison.\n- Minor issues in writing. Example: Line 273-\"it worth noting\". I also find it odd to explicitly list the venues of papers in Table 1; since the references are already there, it is not needed. \n\n[1] Sharma, Shubham, Alan H. Gee, David Paydarfar, and Joydeep Ghosh. \"Fair-n: Fair and robust neural networks for structured data.\" In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, pp. 946-955. 2021.", "was_revised": true, "final_justification": "The paper is a valuable addition to the field, discussing generalization and fairness. The authors addressed my feedback in the rebuttal, and I think the paper should be accepted", "rebuttal": "We would like to express our sincere gratitude to the reviewers for the valuable feedback on our work. Below, we address each of the concerns one by one.\n\n**Weakness 1 - Discussion on Fairness and Generalization May Be at Odds:** \n\nWe sincerely appreciate this comment. Clarifying the inherent conflict between fairness and generalization is imperative. \n\n**When and why they are at odds.** We totally agree that this conflict may arise from both data and model characteristics. Pertaining to the data aspect, the most representative factor, i.e., imbalanced distribution can lead to increased bias toward the majority group, thus improving generalization under certain limited datasets or scenarios. This however, poses the fairness problem a great challenge. Even worse, existing `models` tend to amplify this imbalance distribution problem, making the prediction biased. Though some efforts have tried to equalize performance across demographic groups, the improved fairness often comes with degraded generalization. Unlike the existing methods, in this work, we propose to leverage the causal theory with the confounder controlling guidance. Enlightened by this, our proposed method in fact aims to `rebuild balance` from `imbalance`. Therefore, we can maintain the generalization capability of vanilla models, and can also improve the prediction fairness. \n\n**When conflicts are difficult to resolve.** The mitigation of such conflict also has its limitations. For instance, in certain highly specialized application scenarios, trade-offs between fairness and generalization may be inevitable. For instance, in high-security systems, developers might prioritize reducing overall overall misclassification, thereby sacrificing the performance on minority groups (e.g., individuals wearing unusual clothing). In contrast, for judicial models such as sentencing decisions or crime risk prediction systems, fairness across different demographic groups must be prioritized, even if it comes at the cost of reduced generalizability.\n\nWe greatly appreciate the reviewer’s insightful discussion on this issue. These discussions help us strenghthen the quality and rigor of this work. We will highlight these new points in the revised manuscript.\n\n[1] Improving model fairness in image-based computer-aided diagnosis\n\n[2] The cost of fairness in classification\n\n\n\n**Weakness 2 - Related Work:** \n\nThank you very much for highlighting related work on joint fairness and robustness. We will **cite such work**, including the tabular-data methods [1], and **expand Section 2** of the original manuscript to discuss them. These updated discussions are following: \n\n`several of these approaches, especially [1], use distributionally robust optimization to improve worst-group performance, thereby addressing fairness and robustness together. Unlike these fairness techniques that operate on tabular data, our DAID framework is tailored to visual deepfake data for robust cross-domain deepfake detection. Leveraging fairness as a causal intervention, DAID simultaneously boosts fairness and cross‐domain robustness (generalization). `\n\n`[1] Fair-n: Fair and robust neural networks for structured data.`\n\n**Weakness 3 - Minor Issues:** \n\nWe sincerely apologize for these errors and will comprehensively revise all inaccuracies in the manuscript in accordance with the reviewer's comments. For instance, we will correct the typo “it worth noting” to **“it is worth noting”** on Line 273 . And we will remove venue names from Table 1 (and elsewhere) to avoid redundancy. We thank the reviewer for pointing them out.\n\nFinally, we once again extend our sincere gratitude to the reviewer for the insightful comments."}, {"review_id": "6N7ylMiATe", "reviewer": "Reviewer_VvJy", "summary": "This paper proposes a novel Demographic Attribute-insensitive Intervention Detection (DAID) approach to address the trade-off between improving fairness and generalization in Deepfake image detection. A causal relationship between fairness and generalization is assumed and established through back-door adjustment, implemented via two modules that counteract the confounders of data distribution and model capacity. Experiments are conducted on various datasets and DAID is compared against multiple baselines. The results suggest that the proposed DAID approach can improve both fairness and generalization simultaneously.", "questions": "(Q1) I agree that it is somewhat intuitive to assume a causal relationship from fairness to generalization, but could the authors provide more explanation and literature to support their choice of this assumption, rather than the other way around?\n\n(Q2) The authors are encouraged to respond to the weaknesses mentioned above, especially (W1) and (W3), which will greatly affect my future decisions.", "limitations": "yes", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n(S1) This paper demonstrates that a direct relationship between fairness and generalization can be established after reducing the influence of confounders, as shown by a preliminary experiment in Section 3.2. Based on this finding, the proposed method looks sound to improve generalization by promoting fairness through the two proposed confounder-reduction modules, and therefore address the trade-off between fairness and generalization, as found in previous works.\n\n(S2) The experiments are rich, and the authors provide valuable details and discussions on many aspects in the ablation studies.\n\n(S3) In general, the paper is easy to follow and understand. \n\nWeaknesses:\n\n(W1) While the authors claim that \"DAID elucidates the causal relationship between fairness and generalization during training,\" neither this causal relationship nor the confounder-reduced features are evaluated in the experiments, making it unclear whether the proposed two modules work as intended. Additional comments after the reviewers/ACs discussion stage: In my opinion, if the authors claim a specific approach leads to improved outcomes of something else, that approach itself should be directly evaluated, and evidence should be provided to show that it works as intended. In this work, while the results demonstrate great gains in model generalizability, the underlying causal relationship that is claimed to support these gains is not evaluated. The authors are encouraged to further investigate this matter, or to de-emphasize the claim about \"establishing a causal relationship\", to enhance the contributions of this work and to avoid any potential confusion.\n\n(W2) The pre-processing and annotation pipeline for the Deepfake datasets used in this paper's experiments is unclear, although the authors claim that they follow previous studies (line 225). For example, four races are considered in [24] (Asian, White, Black, and Others), but only three of them are considered in this paper (Asian, White, Black). The choice of subgroups is very important in fairness works and can have a large impact on evaluating the model performance.\n\n(W3) Table 1 and Table 2 do not report any standard deviations or statistical tests, making it unclear whether the minor number differences represent true marginal improvements or if they actually follow the same distribution. This affects the authors' claim that the proposed method outperforms the baselines.", "was_revised": true, "final_justification": "W3 was resovled.\n\nW1 remains unresolved. It still lacks direct experiments and evidence that a causal relationship is created by the proposed method.", "rebuttal": "We thank the reviewers’ thorough evaluation very much and apologize for any confusion resulting from our oversight. Please find below our detailed responses, which address each of the reviewers’ concerns.\n\n**W1 & Q1 & Q2 - Causal Relationship Establishment and Confounder-Reduced Feature Validation**: \n\nWe sincerely appreciate these insightful comments and would like to address the concerns from three perspectives:\n\n1. **Causal relationship establishment.** Traditionally, fairness and generalization are regarded as being in a trade-off relationship. However, our preliminary experiments reveal that, in certain cases, enhancing the fairness of deepfake detectors can improve cross-domain generalization. For instance, random partitioning of the training set into distributionally distinct subsets or enhancing the backbone with additional regularization techniques or data augmentation modules can yield superior performance compared to the baseline model. This intriguing observation lead us to hypothesize that some latent factors may obscure the causal  link between fairness and generalization. Thanks to some causal analysis cases [1] [2], we find that the variability in model performance bears a striking resemblance to the influence of confounders. Therefore, we explore the interplay between fairness and generalization through the lens of causal graphs, and obtain empirical support for the notion that improving fairness can lead to enhanced generalization—provided that confounders are properly addressed (As discussed in Section 3.2). \n\n   [1] Domain Generalization via Causal Adjustment for Cross-Domain Sentiment Analysis\n\n   [2] Graph Out-of-Distribution Generalization via Causal Intervention\n\n   **The performance of The performance of `causal relationships`.** As shown in Table 1 of Section 4, once confounding effects are mitigated (e.g., using the Data Rebalancing module to control DD and the Feature Aggregation module to control MC), our method achieves improvements in both generalization and fairness compared to all baselines, including conventional generalization detectors and fairness detectors.\n\n2. **Confounder-reduced features.** The proposed DAID approach comprises two modules that can be incorporated into the backbone model to mitigate the influence of confounders on the extracted features. \n\n   **The effectiveness of the resulting `confounder-reduced features` can be evaluated from two perspectives:**\n\n   - **The ablation results presented in Table 2** of the original manuscript demonstrate the effectiveness of our confounder-control strategy. Specifically, applying either the Data Rebalancing module to mitigate DD or the Feature Aggregation module to address MC individually yields notable improvements in both fairness and generalization. These findings validate that that each of our two modules effectively controls its respective confounder, and that combining them yields the best overall performance.\n   - **The heatmaps in Figure 5** of the original manuscript further illustrates how our method adjusts confounders. Specifically, attention maps of baseline models are biased. For instance, the baseline model focus on the lips when processing male subjects but on the upper faces for female subjects, implying reliance on gender‐related cues. In contrast, the DAID model exhibits a consistent attention distribution across all groups. This provides a validation of our proposed approach: After controlling for confounders, our method ensures that the model does not rely on sensitive-attribute proxies when making decisions. Instead, it attends to more generalizable, forgery-related features, thereby enhancing cross-domain performance.\n\nWe sincerely thank the reviewer once again for their insightful comments. We will highlight these valuable insights more in the revised manuscript.\n\n**W2 - Exclusion of the “Others” Category:** \n\nWe sincerely apologize for any confusion caused by this experimental choice and would like to offer the following two-fold clarification:\n\n1. **Data Annotation and Rationale for omitting “Others.”**\n   We directly adopt the annotations from previously studies [1]. In the annotations, only three racial annotations—White, Black, and Asian—are explicitly defined. Images that do not fit these three groups are collectively labeled as “Others.” This “Others” group is both `quantitatively minor` (e.g, <5% in the FF++ dataset) and `qualitatively heterogeneous` (potentially comprising multiple smaller ethnic subgroups). Therefore, we restrict our analysis to the three racial groups to ensure that our evaluation is statistically sound.\n\n2. **Impact on experimental results.** To ensure a fair comparison, we have `reimplemented` all baseline models. As illustrated in Figure 4 of the original manuscript, our method consistently achieves improvements across all subgroups. \n\n   [1] Analyzing fairness in deepfake detection with massively annotated databases.\n\n**W3 & Q2 – Statistical Testing:**\n\nWe sincerely appreciate the insightful comment. In the final manuscript, we will augment our results with multiple independent trials to assess stability. Specifically, we will train the primary experiments five times using different random seeds and report the mean ± standard deviation across these runs. A concrete illustration of this format is provided in the table below.\n\n| Method      |    **DFDC (Skew / AUC)**     |     **DFD (Skew / AUC)**     |  **Celeb-DF (Skew / AUC)**   |\n| ----------- | :--------------------------: | :--------------------------: | :--------------------------: |\n| **CADDM**   | 2.174 ± 0.018 / 63.76 ± 0.63 | 0.563 ± 0.024 / 88.52 ± 0.31 | 0.386 ± 0.016 / 80.90 ± 0.60 |\n| **DAW-FDD** | 2.181 ± 0.077 / 59.92 ± 0.17 | 0.576 ± 0.051 / 71.16 ± 0.81 | 0.529 ± 0.065 / 69.86 ± 0.66 |\n| **FG**      | 1.930 ± 0.023 / 60.13 ± 0.35 | 0.451 ± 0.029 / 80.06 ± 0.47 | 0.502 ± 0.038 / 68.67 ± 0.59 |\n| **DAID**    | 1.458 ± 0.024 / 66.94 ± 0.27 | 0.268 ± 0.004 / 91.33 ± 0.36 | 0.293 ± 0.004 / 84.09 ± 0.40 |\n\nIn summary, we sincerely thank the reviewer for these revision suggestions and apologize for the confusion we have caused. We will correct these issues and add new clarifications."}, {"review_id": "yBGykKtSzU", "reviewer": "Reviewer_sqEo", "summary": "The paper introduces a new method, namely DAID, for distinguishing between real and deepfake-generated images, while enforcing algorithmic fairness. The main contribution lies in framing the problem through a causal lens; the authors argue that the causal effect of fairness on generalization performance can be confounded by two observable confounders: (i) the distribution of sensitive attributes (e.g., gender, race) in the dataset, and (ii) the complexity of the predictive model. To address this, the paper investigates whether conditioning on these two confounders reveals a causal effect of fairness on generalization error. To this end, the proposed approach incorporates a loss function that (i) reweights data points based on group membership and (ii) includes a regularization term that enforces demographic invariance in the latent representations. The effectiveness of DAID is validated through empirical experiments on real-world datasets.", "questions": "Please see the weakness section.", "limitations": "yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strengths** \n\n1. The paper is well-written and easy to read. \n2. The motivation for the problem is clear. \n\n\n**Weakness** \n\nWhile the paper brings together several components relevant to enforcing fairness, such as reweighting samples and encouraging latent representations of similar images (real or deepfake) to be close, I am a bit concerned about the novelty. From my perspective, the primary contribution appears to lie in combining two existing strands of fairness approaches in an attempt to improve performance, rather than introducing a fundamentally new methodological insight. Below, I outline my main concerns in more detail:\n\n1. The idea behind the proposed $L_{\\rm attr}$ loss appears closely related to the FACE method introduced in [1] for enforcing individual fairness. However, the motivation for introducing this loss is not clear; to my understanding, the goal of this loss function is to ensure all latent representation of same $Y$ should be close to each other irrespective of their sensitive attributes, but it not clear what type of fairness/model complexity normalization it would bring. I recommend that the authors include a discussion situating their approach within the broader landscape of fairness definitions, such as demographic parity, equalized odds, or individual fairness. This would help make its fairness objectives more transparent. \n\n2. The description of observations is not clear to me. What is $X$ and $Y$? Does $Y  = 1$ mean real or fake images? What are we observing? \n\n3. The ACE estimation result is not clearly presented. It is unclear what is meant by an \"ACE gain of 2.35 percentage points\", does this imply that the estimated ACE is 0.0235? Additionally, the dataset used for computing the bootstrap result is not specified. The authors should clarify both the interpretation of the reported ACE value and the experimental setup used to obtain the bootstrap estimate.\n\n4. I am also confused between $\\tilde h$ in Equation (10) and $\\hat h$. Which on is being used in the loss? It would be great if the authors write the loss function explicitly in terms of all the trainable parameters. \n\n5. In the experimental section, some key evaluation metrics appear to be missing. For instance, [2] reports standard metrics such as False Positive Rate, True Positive Rate, and Accuracy, which are generally more interpretable and informative. I suggest that the authors consider including these metrics for improved clarity and comparability. Additionally, there are some discrepancies in the reported AUC scores; for example, [2] reports an AUC of 94.88 for DAW-FDD on the DFDC dataset, whereas this paper reports a significantly lower value of 59.96. Similar inconsistencies are observed for some other values. Why is this discrepancy? \n\n[1] Two Simple Ways to Learn Individual Fairness Metrics from Data. Debarghya Mukherjee,  Mikhail Yurochkin, Moulinath Banerjee and Yuekai Sun.\n\n[2] Improving Fairness in Deepfake Detection. Yan Ju, Shu Hu, Shan Jia, George H. Chen, and Siwei Lyu.", "was_revised": true, "final_justification": "The paper has a nice idea, but it seems to be a bit incremental given the large existing literature on algorithmic fairness. Notably, the method appears to be a combination of straightforward approaches from group fairness and individual fairness. Therefore, though I like the idea, I tend to keep my score 4 due to the novelty of the new method and the absence of proper theoretical guarantees.", "rebuttal": "We would like to express our sincere gratitude to the reviewer for the positive assessment of our work. Below, we address each of the concerns in detail.\n\n**Weakness 1 - Contributions & Fairness Objective:**\n\nWe sincerely appreciate your insightful comments. We would like to first outline our contributions and then discuss the fairness objective.\n\n**(Contributions)** Our contributions can be broadly divided into two parts.  **`First`**, we employ causal graph to elucidate the relationship between fairness and generalization in deepfake detection. Building on this, we uncover the previously obscured potential for improving both of the two objectives. **`Second`,** we present DAID, a framework that leverages causal analysis to explicitly control confounding factors, thereby concurrently enhancing both the backbone model’s generalization and fairness. Empirical results demonstrate that our approach consistently outperforms baseline methods.\n\n**(Fairness objective)** The proposed demographic-agnostic alignment loss L_attr is indeed related to the notion of individual fairness – ensuring that *similar individuals receive similar outcomes*. In our context, “similar individuals” are those sharing the same *class label* (real or fake) but differing in sensitive attributes (e.g., gender or race). By encouraging the latent representations of same-class samples to be close regardless of demographic group, the proposed alignment loss pushes the model towards treating these individuals equivalently. **`We will cite the related work, such as the FACE method [1], in Section 2 and add a discussion on clarifying how our loss extends the concept to deepfake detection.`**\n\nMoreover, we will explicitly situate our approach within general fairness definitions. Our primary fairness goal is to reduce performance disparities across demographic subgroups – in other words, to achieve a form of **`group fairness`**. In terms of common definitions, this aligns most with pursuing equalized performance (e.g., smaller gaps in accuracy between groups), which is what the used Skew metric captures. By making feature embeddings *invariant* to sensitive attributes, our method mitigates the model’s reliance on those attributes, thus helping to satisfy criteria related to demographic parity (outcomes independent of demographics). In the revised paper, we will include a dedicated discussion of these fairness notions and how DAID’s objectives relate to them.\n\n[1] Two Simple Ways to Learn Individual Fairness Metrics from Data.\n\n**Weakness 2 - Meaning of X and Y, and Clarification of “Observations”:**\n\nWe are sorry for the confusion. X and Y are abstract variables (i.e., `nodes`) in the causal graph. In the context of our study, i.e., the relationship between fairness and generalization, the variables X and Y are in fact F (fairness) and A (generalization), respectively. In particular, as demonstrated in Section 3.2, we define X to quantify the level of fairness intervention (i.e., whether an intervention method is employed to enhance fairness), while Y measures the model’s generalization performance (measured by AUC, which ranges from 0 to 1).\n\nIn Sections 3.1 and 3.2, the term “observe” denotes examining how the generalization performance A varies across different levels of fairness intervention F, thereby establishing the interplay between fairness and generalization. The `“observational data”` refers to the dataset and evaluation metrics used in this analysis. In our experiments, we employ the DFDC dataset and report results in terms of our fairness metric (Skew) and generalization metric (AUC).\n\n**Weakness 3 - Interpretation of ACE:** \n\nWe sincerely apologize for this confusion. The ACE refers to the weighted improvement in the AUC achieved by the high-fairness model over the baseline low-fairness model, where the high-fairness model is obtained by intervention. Specifically, on the DFDC dataset, we first record each model’s prediction scores and compute the AUC within each each demographic stratum (dd, mc). Aggregating these stratum-specific AUCs using empirical weights yields $A_{\\text{low}}$ = 52.08\\%, and $A_{\\text{high}}$ = 53.98\\%, implying a raw gain of 1.90 percentage points (The detailed results for each stratified group are presented in Table 4 of the appendix). To assess statistical reliability, we perform `stratified bootstrap resampling (B=1000) on the DFDC dataset`, and recompute the weighted AUC difference. The resulting mean effect is $\\Delta$ = 0.0235 (i.e., 2.35 percentage points) with a 95% confidence interval of [0.0186,0.0280] and two-sided p<0.001. We will incorporate these details into Section 3.2 to clearly present the ACE definition, its computation, and its statistical significance.\n\n**Weakness 4 - Interpretation of  Equation (10):** \n\nOur sincere apologies for the confusion. In Equation (10) of the manuscript, we define $\\tilde{h} = U^{\\top}\\hat{h}$, where $\\hat{h}$ is the normalized feature vector and $U$ is a trainable orthonormal projection matrix. This $\\tilde{h}$ is indeed the feature representation used in the alignment loss $L{\\text{attr}}$. In other words, our demographic-agnostic alignment loss $L{\\text{attr}}$ is computed on the projected features $\\tilde{h}i$ and $\\tilde{h}j$ (for pairs of samples with the same class label but different demographics). In the current manuscript, we acknowledge that the use of $\\tilde{h}$ and $\\hat{h}$ has led to confusion. To resolve this, we will reorder Equation (10) and use $\\tilde{h}$ in the $L{\\text{attr}}$ to avoid any ambiguity.\n\nWe will also provide a clear description of all trainable parameters and variables in the loss function. In the revised version, we will list: $\\theta$ (parameters of the backbone encoder), $\\phi$ (parameters of the classifier), $U$ (parameters of the projection matrix in the feature aggregation module), along with definitions of $h$, $\\hat{h}$, and $\\tilde{h}$. The training objective can then be written explicitly as a function $L_{\\text{total}}(\\theta, \\phi, U) = L_{\\text{cls}}(\\theta,\\phi) + \\lambda_{\\text{attr}}L_{\\text{attr}}(\\theta, U) + \\lambda_{\\text{ortho}}L_{\\text{ortho}}(U)$. We hope that can clarify the use of $\\tilde{h}$ and loss function. \n\n**Weakness 5 - Additional Metrics and Discrepancies in AUC:**\n\nWe totally agree that including these evaluation metrics can improve the clarity and completeness of the results. In the revised manuscript, we will incorporate additional metrics such as **`Accuracy, True Positive Rate (TPR), and False Positive Rate (FPR)`**. One sample table is shown below:\n\n| Method      | ACC ↑     | TPR ↑     | FPR ↓     |\n| ----------- | --------- | --------- | --------- |\n| **DAW-FDD** | 57.89     | 60.69     | 43.33     |\n| **FG**      | 61.27     | 65.85     | 40.73     |\n| **DAID**    | **64.99** | **72.62** | **38.34** |\n\nRegarding the **discrepancies in AUC scores**, this is a result of different experimental settings. In our experiments (as stated in Section 4.1), *all models are trained on FaceForensics++ (FF++)* and then tested on unseen datasets (DFDC, DFD, Celeb-DF) to assess generalization across domains. This  cross-dataset generalization scenario is a significantly more challenging setting, which naturally yields lower absolute AUC values. In contrast, the 94.88% AUC from [2] is achieved when training and testing on DFDC itself (within-dataset evaluation). `The two numbers are not directly comparable due to this difference in protocol.` We have reproduced** baseline methods in our cross-domain setup for a fair side-by-side comparison. We will explicitly clarify this in the revised manuscript to avoid confusion. For instance, we will add a note such as: *The baseline results reported in our tables are obtained under cross-dataset evaluation protocol (train on FF++ and test on other sets).”* \n\n[2] Improving Fairness in Deepfake Detection\n\nFinally, we once again extend our gratitude to the reviewer for meticulous review and recognition of our work."}, {"review_id": "fxMXHQRsJm", "reviewer": "Reviewer_V3RS", "summary": "This submission argues that when model capacity and data distribution are controlled for, improvements to the fairness of a deepfake detector lead to improvements in their generalizability. They provide theoretical arguments for this claim based on the back-door criterion, and develop a training regime which attempts to counteract these confounders. Their results show that their technique achieves SOTA fairness and generalizability.", "questions": "a. Am I correct in understanding that the latent normalization is just performed once, as opposed to e.g. at every hidden layer of a neural network architecture?\n\nb. Can you explain the DD stratification going on in lines 151-155? (related to weakness 2).\n\nc. In Fig. 3 you show the relationship between hyperparameters (meant to improve fairness) and resulting generalizability. Would you be able to provide the relationship between the hyperparameters and the resulting fairness (skew) too?", "limitations": "Yes, with the exception of potentially-unstated or understated key assumptions of their theoretical results that limit the scope of their causality claim.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1. The writing is very clear.\n\n2. The proposed method achieves state of the art on both fairness and generalizability.\n\n3. The ablation study at the end is very comprehensive, and provides an exhaustive analysis of their proposal’s contributions and impact.\n\nWeaknesses:\n\nI am not fully convinced of the theoretical results. In particular:\n\n1. The backdoor criterion is defined with respect to a causal graph. Therefore, the theoretical results are built on the assumption that data distribution (DD) and model capacity (MC) are an exhaustive set of confounders. If a different researcher assumed a larger causal graph with more confounders, then the back-door adjustment formula, Eq (2), would not be correct (i.e. it would require more marginalization). \n\n2. While the DD variable is called “data distribution”, but my understanding of lines 151-155 is that it refers to the proportion of the data set belonging to each sensitive attribute. There are many other properties of the data distribution that can impact fairness, e.g. correlation between sensitive attribute and spurious correlations such as clothing, picture setting etc. This broader definition of data distribution is implied in the latent normalization of Sec. 3.3, and if it is believed to be a confounding variable should be added to the DAG and marginalized out in the causal analysis.\n\n3. Lines 158-161 seem to depart from the theory that precedes it. Rather than defining the do intervention as a successful flipping of F from 0 to 1, it is defined in terms of training with or without a resampling strategy. Without understanding the relationship this has to the eventual F (which is not rigorously defined in the main paper as the threshold between low and high fairness is not established) it is hard to accept the numeric conclusions of the analysis.\n\nIn conclusion, I remain unconvinced of the claimed causal relationship. I am, however, convinced of a weaker claim: when DD and MC are controlled for, improvements in fairness lead to improvements in test AUC in the setting studied. This weaker claim is enough to motivate experimental choices, but weakens some of the stated contributions.\n\nIn general, this paper is very well written and thorough in its examination of its theoretical results. I lack the confidence to assert whether the primary experiment is lacking baselines or data sets/settings, but based on the results shown the improvements to generalizability and fairness are clearly-important contributions.\n\nSmall notes:\n\n“it worth” line 212\n\nLines 200-209 are a bit confusing without the context of Eq (12), which comes later. e.g. on line 200, you state “We enforce” which is a confusing construction. What is being enforced? It might be more appropriate to say “we define”, or make clear what you are enforcing wrt L_attr.\n\n\n\n**EDIT BASED ON DISCUSSION WITH AC**\n\nThis submission has a great deal of merit but does not adequately establish the causal relationship claimed in the abstract and introduction. In particular\n\n- The back-door criterion only establishes a causal relationship with respect to an underlying DAG, i.e. it assumes the DAG is correct. It remains unproven that there are not other confounders.\n- In particular, dataset reweighting (lines 158-161) certainly has a positive effect on fairness, but it is not exactly a do intervention. It affects something upstream of fairness, and could have unexplored confounding effects.\n\nI think the emphasis should be shifted to instead focus on the fact that, when model capacity and data distribution are controlled for, fairness-promoting interventions also improve generalization. This is a sufficient observation to motivate the experimental sections of this work.\n\nThe broad framework of causal inference and back-door criterion does not need to be scrapped to perform this change-in-emphasis; it can be done by softening some of the claims. For example, I suggest text to the effect of \"we investigate two possible confounders (data distribution and model capacity) and construct the resulting causal graph. Using the back-door criterion, we demonstrate a causal relationship between fairness and generalization under this causal model\". That way, the framework used in Sec 3 can be retained, but the insufficiently-substantiated claim that the authors demonstrate a (general) causal relationship between fairness and generalization is replaced by a claim that limits the scope to the DAG in question.", "was_revised": true, "final_justification": "See author rebuttal and my response. The added table and explanation for their methodology is sufficient to demonstrate theoretical and experimental contributions. I have minor concerns about their claim of establishing a causal relationship and about confusion as to what exactly the DD confounder refers to, but I believe these can be addressed by editing of the text. As a result, I have raised my score from 4 to 5.", "rebuttal": "We sincerely thank the reviewer for the recognition of our work. In the following, we address each of the concerns in detail.\n\n**Weakness 1 - Confounding Factors and Correction of Eq (2)**:  \n\nWe are very grateful for the reviewer's insightful comments. We totally agree that in a more comprehensive causal graph, one could incorporate more confounding factors (e.g., scene context, image style). In our current work, we focus on DD and MC to demonstrate that, once these two confounders are controlled, fairness positively influences generalization. As demonstrated in Section 3.2, controlling these confounders enables an estimate of the fairness→generalization effect, which is significantly positive. Specifically, we observe an improvement in AUC (≈+2.35 points **↑**) when moving from low fairness to high fairness, with statistical significance (p < 0.001). Section 4 further confirms that fairness interventions lead to consistent generalization gains under our DD and MC controls. We fully agree  that Eq. (2) can be extended to handle more confounders. We would like to introduce a unified confounder set CF, and formulate an updated equation: `P(A|do(F = f)) = ΣP(A|F=f, CF=cf)·P(CF=cf)`. We will clarify that the original Eq. (2) is simply its special case when `CF = {DD, MC}`.\n\n**We are very grateful for the reviewer’s insightful comment on the more diverse confounders, and we fully agree that this is both sensible and practically valuable.**\n\n**Weakness 2 & Question b - The Clarification of DD:** \n\nWe apolpgize for this confusion, and we would like to first detail the procedure of DD stratification, and then discuss the other properties of the data distribution related to Weakness 2.\n\n**“DD stratification” means dividing the dataset into demographic subgroups** based on the sensitive attributes (e.g. gender and race) and treating each subgroup as a distinct stratum. In our implementation, we considered the intersection of *Gender* (Male, Female) and *Race* (White, Black, Asian), yielding 6 subgroups in total :\n\n- **Male-White**, **Male-Black**, **Male-Asian**\n- **Female-White**, **Female-Black**, **Female-Asian**\n\nEach of these subgroups represents a distinct portion of the overall data distribution (DD). We **condition on these subgroups when estimating the causal effect** of fairness on generalization. In other words, during our causal analysis we calculate AUC within each stratum and then aggregate . By accounting for the data distribution this way, we “neutralize” its confounding influence – any observed improvement in generalization can be attributed to improved fairness rather than differences in demographic data balance.\n\n***(Related to Weakness 2:)***  We appreciate the reviewer’s suggestion to consider additional latent confounders beyond DD. In our formulation, DD is defined by demographic groupings. This choice follows from our motivation to estimate fairness-driven causal effects through demographic disparities, which are the most salient factors in fairness.\n\nThat said, we agree that other latent properties of the data – such as image background, clothing, or picture setting – can correlate with sensitive attributes and potentially act as confounders. While these broader factors are not explicitly represented in our causal DAG, we address their influence implicitly through two mechanisms: **1) Latent feature normalization** within demographic groups, which aligns feature distributions across subgroups and thus neutralizes subgroup-specific visual patterns that co-occur with sensitive attributes (e.g., if a race group is predominantly photographed in a specific setting). **2) Feature alignment loss**, which encourages latent features of samples belonging to different subgroups but sharing the same real/fake label to be close in the feature space, thereby mitigating the model’s dependence on spurious correlations such as background-related biases. These modules functionally mitigate the effect of style- or context-related biases (even if not explicitly modeled as DAG nodes). \n\n**We appreciate the suggestion to broaden the DD definition in the DAG. Incorporating richer confounder sets would allow more fine-grained causal modeling.** \n\n**Weakness 3 - Clarification on do(F) Intervention:** \n\nWe sincerely apologize for this confusion. Our do(F) intervention is intended to enhance the fairness of an initially low-fairness model, and establish a low-fairness (F=0) versus high-fairness (F=1) conditions. This is achieved by the resampling strategy, which is a widely adopted approach that has been used in prior fairness research [1] [2]. This resampling-based intervention effectively breaks the usual link between sensitive attributes and model predictions, approximating an idealized scenario of improved fairness. Specifically, in our implementation, the low-fairness setting is achieved through standard training (baseline). We then apply the resampling strategy to enhance fairness, thereby obtaining a high-fairness model. Quantitatively, this resampling reduced the Skew metric of **`Xception from 2.22 to 2.06 and that of EfficientNet from 2.01 to 1.91`**. The full set of numerical results from our causal analysis has been presented in the appendix to reinforce the robustness of our findings.\n\n[1] Social debiasing for fair multi-modal llms.\n\n[2] Enhancing Fairness through Reweighting: A Path to Attain the Sufficiency Rule\n\n**Small Notes:** \n\nWe apologize for the confusion caused by these oversights. We will correct them accordingly. For instance, we will revise “We enforce” in line 200 to “We define” to better align with the subsequent introduction of L_attr.\n\n**Question a -  Latent Normalization:** \n\nWe sincerely appreciate the insightful feedback, and the reviewer’s understanding is absolutely correct. The latent feature normalization** (subgroup-wise feature normalization) in our method is performed as a one-time operation on the final representation, **`not at every hidden layer`**. This design keeps the approach simple and focuses the normalization where group-wise distributional shifts manifest most – in the final embeddings – without repeatedly normalizing at each network layer (which may unnecessarily constrain the model’s internal representations).\n\n**Question c - Hyperparameter and Fairness:** \n\nWe have added the results between these two hyperparameters and skew in the table below. Overall, improvements in fairness are generally positively correlated with improvements in generalization. The best performance is achieved when λ_attr reaches 0.7 and λ_ortho reaches 0.2.\n\n| λ_attr        | Performance | λ_ortho       | Performance |\n| ------------- | ----------- | ------------- | ----------- |\n| *(Reference)* | *1.574*     | *(Reference)* | *1.495*     |\n| 0.1           | 1.559       | 0.05          | 1.483       |\n| 0.2           | 1.541       | 0.1           | 1.475       |\n| 0.3           | 1.529       | 0.15          | 1.464       |\n| 0.4           | 1.513       | 0.2           | **1.460**   |\n| 0.5           | 1.509       | 0.25          | 1.462       |\n| 0.6           | 1.497       | 0.3           | 1.461       |\n| 0.7           | **1.495**   |               |             |\n| 0.8           | 1.496       |               |             |\n| 0.9           | 1.498       |               |             |\n| 1.0           | 1.497       |               |             |\n\nIn conclusion, we sincerely appreciate the reviewer’s  positive perspectives of our work and will provide additional clarifications to address the concerns."}]

Author rebuttal:

Meta-review: This paper simultaneously tackles domain generalization and fairness in deepfake image detection from a causal perspective. It first discusses the causal effect of fairness on generalization, suggesting that improving fairness can lead to improved generalization when a quantity called average causal effective is positive. Then, it presents a new training approach RAID, which aims to learn a fair model by jointly minimizing a classification loss rebalanced between different groups and a prediction gap between individuals. Experimental results show RAID achieves the best fairness and generalization performance compared to many state-of-the-art baseline methods. 

---

Overall, the work is interesting and the proposed approach is effective. Most reviewers are on the positive side. 

A notable concern shared by many reviewers, including the most positive ones, is the suggested (causal) relations between fairness and generalization is not thoroughly investigated or discussed. Authors did a diligent job in defense, but the responses remain somewhat intuitive. Authors should incorporate the feedback in revision, including 

-- discuss why exactly fairness and generalization may be at odds with each other (Reviewer 4jNC)

-- de-emphasize the claim that "DAID elucidates the causal relationship between fairness and generalization during training"  and related statements. Consider making a weaker claim such as that suggested by Reviewer V3RS. 

-- discuss connections to studies on fairness-robustness relation (Reviewer 4jNC), and to the broader literature of algorithmic fairness, especially with group fairness and individual fairness techniques (Reviewer sqEo). 

I also notice that the concept of Average Causal Effect (AVE) is also discussed in some other contexts e.g. [1,2]. Is there any relation between the AVE defined in this paper and those? 

[1] Average Causal Effects From Nonrandomized Studies: A Practical Guide and Simulated Example, 2008. 

[2] Disease Modeling and Public Health, Part B, Handbook of Statistics 2017. 

---
 
Despite the concerns, reviewers are on the positive side. I think the proposed causal perspective is interesting and the proposed RAID has impressive performance. This work may serve a good step towards a new direction.
Decision: Accept (poster)