Kernel Space Conditional Distribution Alignment for Improving Group Fairness in Deepfake Detection
Authors: Sayantan Das
Mojtaba Kolahdouzi
Ali Etemad
Venue: Accepted by TMLR ()

Abstract: We introduce FairAlign, a new method to reduce bias and improve group fairness in deepfake detection by aligning conditional distributions of embeddings in a high-dimensional kernel space. Our approach reduces information related to sensitive attributes in the embedding space that could potentially bias the detection process, thus promoting fairness. FairAlign is a versatile plug-and-play loss term compatible with various deepfake detection networks and is capable of enhancing group fairness without compromising detection performance. In addition to applying FairAlign for reducing gender bias, we implement a systematic pipeline for the annotation of skin tones and promotion of fairness in deepfake detection related to this sensitive attribute. Finally, we perform the first comprehensive study toward quantifying and understanding the trade-off between fairness and accuracy in the context of deepfake detection. We use three public deepfake datasets FaceForensics++, CelebDF, and WildDeepfake to evaluate our method. Through various experiments, we observe that FairAlign outperforms other bias-mitigating methods across various deepfake detection backbones for both gender and skin tone, setting a new state-of-the-art. Moreover, our fairness-accuracy trade-off analysis demonstrates that our approach demonstrates the best overall performance when considering effectiveness in both deepfake detection and reducing bias. We release the code at: https://github.com/Mkolahdoozi/FairAlign.

Published in Transactions on Machine Learning Research (10/2025)
Kernel Space Conditional Distribution Alignment for Improv- ing Group Fairness in Deepfake Detection
Sayantan Das, Mojtaba Kolahdouzi, Ali Etemad /sayantan.do das, m. kouohoh ali. ali.etemad) Suueensu..uueeuo Queen's University, Canada
Reviewed on OpenReview: https: openreview. net/ forun? id= 68Lv6u4,N9J
Abstract
We introduce FairAlign, method to reduce bias and improve group fairness in deeplake detection by aligning conditional distributions of embeddings in a high-dimensional kernel space. Our approach reduces information related to sensitive attributes in the embedding space that could potentially bias the detection process, thus promoting fairness. FairAlign versatile aung-and-llay loss term compatible with various deepfake detection networks and is capable of enhancing group fairness without compromising detection performance. In addition to applying FairAlign for reducing gender bias, implement systematic pipeline for the annotation of skin tones and promotion of fairness in deepfake detection related to this sensitive attribute. Finally, we perform the first comprehensive study toward quantifying and understanding the tradc-off between fairness and accuracy in the context of deepfake detection. We use three public deepfake datasets FaceForensics+ CelebDF, and WildDeepfake to evaluate our method. Through various experiments, we observe that FairAlign outperforms other bias-mitig methods across various deepfake detection backbones for both gender and skin tone, setting new state-of-the-art. Morcover, our fairness-accurac, trade-off analysis demonstrates that our approach demonstrates the best overall performance when considering effectiveness in both deepfake detection and reducing bias. We release the code at: https://gits /owooooaaaooooi///aaaaa Paiiiiln.n....nnn......
1
Introduction
The case of creating deepfakes necessitates the development of sophisticated methods for dotection of content produced by generative AI models. To detect deepfakes, deep learning models have been recently utilized discern subtle inconsistencies and artifacts common in synthetic content (Das al. 2023 Wang al.)
2023 Cao et al.) 2022). Although recent deepfake detectors achieve high detection rates, several studies
TTrnn & Liu 2021 Xu et al. 2022b Nadimpalli & Rattani 2022 Masood et al. 2022 Ju et al. 2024 have shown that detectors can exhibit bias toward specific groups with regards to sensitive attributes like gender, racial background, and others. For instance, it has been shown that certain state-of-the-at deepfake detectors output higher accuracies when deepfake content involve individuals with lighter skin tones (Hazirbas et al. 2021: Trinh & Liu 2022). ) This can enable attackers to create malicious deepfakes aimed at more vulnerable groups. This highlights the need to consider group fairness as an equally important factor alongside detection accuracy deepfake detectors.
We identify three key open problems with the current state of deepfake detection research: (1) Although number of bias-mitigating strategics have been proposed for deepfake detection (Ju et al. 2024 Nadimpalli & Rattani 2022), the embeddings generated by these detectors continue to retain information cioomaation related to sensitive attributes, which could cause biases in detection of deepfake content. To demonstrate this empirically, we extract embeddings from the final layer of an Ceficiinttet-BB(Cocoooo
al. 2022) detector trainod with the DAG-FDD (Ju al.) 2024) bias mitigation method on the Celeb-DF dataset. We then project these embeddings into 2D space using t-SNE and colored the resulting points based the subjects' gender.

---

Published in Transactions on Machine Learning Research (10/2025)
5
t-SNE 2
0
-5
Male
Female
-5
0
5
t-SNE 1
Figure Representations for sample images from the Celeb-DI dataset obtained from the EfficientNet-BA delector encoder (Coccomini al. 2022) with DAG-FDD (Ju el al. 2024 bias-mitigation. The male/female clusters indicate
the existence of gender-related information in the embeddings.
Figure Songuresshore shows the result. It is evident that the embeddings corresponding to male and female subjects occupy different regions in the 2D space, demonstrating that gender-related information persists in the embeddings. As result, we suggest. that there is substantial room for improvement at the core of this problem. (2) Despite efforts to develop less biased deepfake detectors that work fairly across different populations, gender has been the main area of focus. While few works have focused on 'ethnicity' as second factor, we argue that 'skin tone' (Hazirbas al.) 2021 is an equally critical and, at the same time, more practical factor focus We believe skin tone is possibly different source for bias, since: (i) Each ethnicity might itself contain variety of skin tones; (ii) Encoders are highly likely to discriminate based on color given its strong visual prominence; (ii) Ethnicity labels are prone to label noise (Heldreth et al.) 2023). (3) Lastly, there exists phenomenon referred to as the
trade-off", which indicates that enhanced fairness may come at the cost reduced accuracy [Little et al.) 2022 Hazirbas et al.] 2021 b. While some studies suggest variability the existence of this trade-off (Maily et al. 2021 Wick al. 2019 Dulta el al.) 2020). the intertwined relationship of fairness and accuracy has been widely confirmed in prior works (Wang al.,
2021a Li ct al. 2021 Zhang et al. 2021). However, to knowlodge, this trado-off has not been studied in prior works in the context of deepfake detectors.
In this paper, to address the first problem mentioned above, we propose a novel loss term called FairAlign, for enhancing fairness via the alignment of conditional distributions of the embeddings in higher-dimennsional kernel space. Our method aims to reduce the gap in detection performance across different sensitive attributes such as gender, thus mitigating the risk of biased outcomes. By leveraging the kernel space, our method integrates the cross-covariance and covariance operators of the conditional distributions of the embeddings given sensitive attributes obtained from deepfake detectors into the training process. Our method is plug- and-play technique that can be integrated with other existing loss functions used in deepfake detection. Using our method, we perform multiple bias-mitigation experiments in deepfake detection three public, real-world datasets (Celeb-DF (Li et al. 2020), PaceForensicc+++ (FF++) (Rossler et al.) 2019), and WildDeepfake (Zi et al. 2020)), wherein we demonstrate the effectiveness of our method in improving the fairness of several state-of-tho-ar deepfake detectors while retaining strong detection performance. For the second problem, we propose simple yet effective pipeline for detecting skin tones and using them mitigate bias for this factor. Our method uses ArcFace (Deng et al. 2018 to detect and extract the face. Subsequently,
2

---

Published in Transactions on Machine Learning Research (10/2025)
select the facial skin regions using a pre-trained U-Net model (Xu et al. 2022a), based which the average skin color is measured. Finally, we use the shortest Euclidean distance between the skin tone with respect to Monk Skin Tone (MST) (Heldreth et al., 2023) scale to determine the final skin tone. We then apply this pipeline to deepfake detection datasets for the first time, following which we perform skin tone bias mitigation experiments. We find that our method, Alig,, is effective at reducing skin lone biases in deepfake detection datasots. Lastly, to address the third problem, we utilize two different metrics, Fairea (Hort et al.) 2021) and Harmonic Mean (Lesota et al. 2022: Li et al. 2023), to combine both fairness and accuracy into unified indices, for the first time in the area of bias mitigation for deepfake detection. Our analysis shows that while some existing lairness-promotin this improvement comes at the cost of accuracy, hence not ideal for practical applications. The analysis further demonstrates that our proposed FairAligu maintains the highest, performance in lerms of both fairness and accuracy as per the unified metrics.
techniques do indeed reduce bias to good degree,
Our contributions are summarized as follows:
(1) For promoting group fairness in deepfake detection we propose new loss term, FairAlign, that operates in the kernel space to reduce the distance between distributions of the representations learned by deepfake detectors given different sensitive attributes. Our method demonstrates effectiveness in improving the group fairness for state-of-the-art deepfake detectors while maintaining strong detection performance on three
large-scale datasets, FF++ (Rossler et al. 2019 CelebDF (Li et al. 2020 and WildDeepfake (Zi et al., 2020
(2) We analyze and improve fairness based on skin tones for deepfake detection tasks. We extract skin tones from existing deepfake datascts using the guidelines given by the MST scale(Helrreth et al. 2023). and apply our proposed FairAligu method for enhanced fairness. Our experiments demonstrate that Align improves skin tone fairness across all state-of-the-ar deepfake detectors. To our knowledge, this is the first attempt at reducing bias against different skin tones in the context of deepfake detection. Additionally, we enhance fairness based on the intersection of gender and skin tone in the context of deepfake detection for the first time.
(3) To objectivcly quantify the fairness-accuracy trade-off, we analyze two unified metries for the first time in the realm of fair deepfake detection. Results show that our method is highly favorable a bias-mitigating method that strikes healthy balance between fairness and accuracy.
(4) To enable fast and reliable reproducibility, https://anonynnt Aopen /r/FairAlignn170F.
we
release the code
at:
2 Related work
Fairness in deepfake detection. Prior works have demonstrated the existence of bias in deepfake detection tasks with respect to gender, age, and ethnicity (Hazirbas et al.) 2021 Trinh & Liu 2021 Pu et al.) 2022 Xu et al.] 2022b), while few works have proposed solutions to mitigate this bias Nadimmplli & Rattani 2022 Ju et al. 2024). The work in (Trinh& & Liu 2021) evaluates bias in existing deepfake datasets and detection models for the first time in the literature; however, it doesn't take into account the intersectional bias. The evaluation of bias for a popular detection model (Mesolnceetionn-4) on FF++ dataset is done in (Pu et al. )022). This work is limited in terms of the mumber of detection methods evaluated. A more comprehensive study is proposed (Xu et al. 2022b) which evaluates fairness over three deepfake detection models. One recent work (Nadimpalli & Rattani (222) has attempted to mitigate the aforementioned biases through data-centric approaches, i.e., making the datasets like FF++ balanced with regards to different sensitive attributes like gender. The process of also showed limited improvement in fairness. The work in (Ju et al.) 2024) applics conditionall-vluu-at--- loss to mitigate bias with regard to both gender and ethnicity in the context of deepfake detection. To our knowledge, this paper is the first and only one to directly provide a solution for bias in deepfake detection.
via data annotation is
and
Skin tone fairness. While cthnicity has been considered for promoting fairness (Ju et al. 2024 Trinh & Liu 2021: Xu et al. (2022b) in several deepfake detection literature, skin tone as factor in reducing bias is more commonly addressed in areas like skin lesion classification (Kinyanjui et al. 2019 Heldreth et al. 2023
3

---

Published in Transactions on Machine Learning Research (10/2025)
To our knowledge, (Hazirbas et al. 2021 is the only prior work to study skin tone in the context of deepfake detection. In their study, researchers evaluate biases in deepfake detection by analyzing top models from the DeepFake Detection Challenge (DFDC) (Dolhansky et al. 2020) on the Casual Conversations dataset, which is rich in diversity across age, gender, and skin tone. Their analysis confirms the importance of skin tone as a crucial sensitive attribute for bias mitigation in deepfake detection.
Fairness-accuracy trade-off. While aairneys-accuracy trade-off is well-known phenomenon (Little et al, 2022 Dutta et al. 2020 Maity et al., 2021), only handful of works have focused on introducing a quantitative
measure to assess this trade-off (Dutta et al. 2020 Little et al. 2022 Wang et al. 2021b), although none are positioned in the context of deepfake detection. One such work defines the Fairnoss- Ceeis-AAraaaa Area-Unddrtthh-Cuuve
(FAUC) to empirically define the fairness-accuracy Pareto frontier (Little et al.] 2022]. FAUC provides a model-agnostic metric to measure the Pareto frontier. However, as mentioned in their work, FAUC is ineffective when intersectional fairness is involved or in cases where fairness and accuracy typically do not have an inversely proportional relationship. Another work (Wang et al. 2021b) approaches this trade-off through the lens of multi-task learning by proposing two metrics: Average Relative Fairness Gap and Average Relative Error. These metrics compare the
Rate Gap and error rates of multi-task models to those of single-task models with the same architecture, providing a muanced assessment of the balance between fairness and accuracy in multi-task learning. With a different perspective, (Dutta et al. 2020 approaches the trade-off between fairness and accuracy by quantifying separability with Chernoff information, challenging the use of biased datasets for performance measures, and advocating for ideal, unbiased datasets. Our work utilizes Fairea [Hort et al. 2021) and Harmonic Mean (Lesota et al., 2022), which were previously not explored in the context of deepfake detection. Fairea quantifies the trade-off by computing the area within enclosed region formed by the bascline fairness values and the coordinates of any bias-mitigation method in a two-dimenusinaal fairness-accuracy space. Harmonic Mean computes the trade-off using a straightforward formula that balances accuracy and fairness, ensuring neither is overlooked in the evaluation process.
3 Proposed method
We introduce a novel loss term called FairAlign for enhancing group fairness in deepfake detection. Training a deepfake detector using FairAlign causes the aligument of conditional distributions of embeddings given different sensitive attributes. Our (kirness-ehanccnng approach is inspired by domain adaptation tasks MMukherjee et al.) 2022 Truong et al. 2023 Joshi & Burlina 2021) and is considered an in-processing fairness technique HHan et al.) 2024), as it intervenes directly within the learning algorithm to promote fairness.
Problem setup. Let us represent a triplet {(r,, Wi, =;) =)) where x; denotes the embedding generated by deepfake detection model for the ith input image, yi is its corresponding forgery label (real, fake), and 2; is the associated sensitive attribute (c.g., female, malc, etc.). For simplicity, we can assume that the embeddings x; are independent and identically distributed. Additionally, let X, Y, and Z be the random variables associated with the embeddings, their labels, and the sensitive attributes, respectively. Let the distribution of X be defined over the set X' and the distribution of Z be defined over the set Z. Also, let the cardinality of the set Z be equal to Ç; for example, in the FF++ dataset, Ç ç 2 for gender, which corresponds to male and female. We alternatively denote embeddings x; as 2t and x. where a and b denote elements in the set Z. Let us denote P(.) as the probability distribution of an arbitrary random variable. Accordingly, the optimum condition for a fair classifier is denoted by
Po = z.) (0lz- 2b) Vzas Zh E Z,
(1)
where & represents the random variable associated with the classifier's output, i.e., the predicted forgery category. This optimum condition is also known demographic parity, which is the objective of many bias-mitigating methods (Srivastava et al., 2019). In this paper, we aim to reduce the information related to the sensitive attribute 2; from the embeddings using our proposed loss.
Fair Align Our overall goal is to accurately capture and minimize the distance between the distributions of embeddings given different sensitive attributes, through a novel loss term. The discrepancy between two distributions can be captured with regard to different statistical measures like expected value, covariance, etc. However, it has been shown that simply considering the arithmetic difference of such statistical measures,
4

---

Published in Transactions on Machine Learning Research (10/2025)
particularly in lower dimensions, cannot effectively capture the discrepancy (Luo & Ren 2021 As an example, Maximum Mean Discrepancy (MMD) (Gretton et al. 2012) uses the difference in expected values of two distributions in high dimensions (specifically, Reproducing Kernel Hilbert Space (RKHS) (Steinke & Schölkopf, 2008)) to render the discrepancy between two distributions. As another example, the Bures metric (Bhatia et al. 2017,, which defines the discrepancy between two distributions as the difference between their covariance matrices, cannot effectively capture the discrepancy unless the data are mapped onto a. high dimensional space. To avoid explicit data projection onto higher dimensions and the additional computational load required to measure the difference in high-dimensional space, kernel functions (Zhang al.) 2020) can be used to operate directly in low-dimensional space. Usage of kernel functions allows establishing the Bures metric in RKHS (Steinke & Schölkopf 2008) where it is viable to be used as a distance metric (Zhang et al, 2020). This is also termed the Kernel Bures metric. Building on this concept, we utilize the Conditional Kernel Bures (CKB) metric, which is particularly designed for conditional distributions (Fukumizu et al. 2009 Luo & Ren 2021). Another noteworthy point is that other domain adaptation mothods like MMD are effective in aligning marginal discrepaney. Although effective, this can lead to discarding valuable discriminating information in label distributions (Luo & Ren 2021). As CKB aligns conditional distributions, it preserves such information leading to higher classification accuracy.
As defined earlier, 2 li=1 and attributes, drawn from the conditional distributions Pxiz=za and Pxiz= respectively. Let us define kernel functions kx and liz on the space of embeddings and Z, respectively. Further, we define (xx) = xxx (x, and xb2) (2..) as feature mappings from X' to RKHS Hx and Z to RKHS Hz respectively. Now, let denote cither a or b by the notation a/b. Accordingly Xx: K22 and K are the kernel matricos, where //b ( = kx(x;" . 2 11 = kz(z, 12g
are sets of embeddings corresponding to different sensitive
a/b, and xxxx = kx x9)))
The feature mappings can therefore be represented by Paʃb = bb //b ,.... oax (x and Wa/b =
( ................................ ( (
Consequently, as illustrated in [Luo & Ren 2021 the empirical cross-covariance matrices are denoted by A& 1 Pa and A = m PoJm V,, with J, = (In n 11 as the centering matrix of size n X n, In as the identity matrix, and I, a vector of ones of dimension n (similarly for Jm, Im. and 1m). The covariance matrices 2 and 1 are defined in a similar fashion. Morcover, the empirical conditional covariance is defined as:
a
22
XXIZ = A4/b xx - A0/b
j0/b Az + el)
Aa/b
(2)
where el acts as regularizer to the Azz matrix and > 0 is the regularization factor. The regularization is done due to the rank deficiency of the matrix Âzz. We denote the matrices
Qa
In
1
M2 - ME (MZ + enIn) Mž
(3)
TEE
1
Q6 me
M2 - Ma (M² + mmmm) M2
(4)
where the centralized kernel matrices are defined as: M9. = Jn, K K2 and M = Jm Jm Jm Using the
Cholesky decomposition (Schulman & Cramer 1975) Qa/b = Sa/bS a/b: where Q is a positive-defiitee matrix
(uo & Ren 2021) and i' the lower-triangular matrix obtained from the decomposition. Accordingly, we can reformulate the conditional covariance operator Ax as:
1 -. (Da))Sa) n1
(5)
and respectively for A x The empirical CKB metric is accordingly defined as:
(oxo(PxIz=z, Soixiz,,,PPIi---- PxIz=z,)= = decol A Axx(z) =etr M& (enI, + --2-1111 + etr MM& (emIm + +M2)
2
(JmSs)" k (JrS
(6)
vm x
5

---

Published in Transactions on Machine Learning Research (10/2025)
where | || is the muclear norm. The empirical CKB metric is differentiable and highly suitable for usage
loss function.
Total loss. Based on Eq. we define Fair as:
L Align
Σ
8xKe(PxIZ=201 Px Px(z=zj)
V(sLay)E2
Additionally, we use a binary cross-cntropy loss, Lee for supervising the deepfake detector to discriminate between real and fake samples, defined as:
R My log(O)) + Un) log(1 0;)]
Lee - Ω
Σ j=1
(8)
Here, denotes the total count of samples in the batch. Finally, we define the total loss as:
Con + CrairAlign
(9)
where A controls the contribution of the CKB term.
Skin tone fairness enhancement. As indicated earlier, in addition to a fair deepfake detection solution, we aim to detect skin tone. To this end, we first perform face detection using MobilcFaceNet backbone CChn jel al. 2018) along with the ArcFace loss (Deng et al.] 2018). We then employ the U-Net model presented (Xu al. 2022a) segment skin regions from the extracted facial image. Next, we compute the average color of the facial skin pixels to obtain the overall skin tone. Finally, intend to
standard dcfinition for characterizing the estimated skin tone. To do so, we use the MST scale (Heldreth et al. 2023). Therefore, to identify the corresponding tone from the MST scale, we compute the closest neighbour based on the Euclidean norm between the average tone and the tones in the MST scale. See Appendix for visual examples of skin tone bins along with the representative face crops for each.
Pairness-accurracy trade-off assessment. The final goal of our work is to conduct a comprehensive analysis of fairness-accuracy trade-off in the context of deepfake detection. Multiple studies have previously indicated the presence of an intrinsic trade-off between fairness and accuracy (Little et al., 2022: Hazirbas
2020: Wick et al. 2019), although not in the area of deepfake detection. To this end, we employ two metrics to characterize this tradc-off in this context for the first time. (1) Fairca: The Fairca approach (Hort et al. 2021 first assesses how a model's predictions would change if it were less biased. This is done by manipulating the model's predictions to reflect a range of hypothetical scenarios from slightly to fully unbiased, which referred to as 'mutation' in (Hort et al. 2021). ). The range of mutation can be from 10% to 100%, with 10% increments at each step. These adjusted predictions create a spectrum of potential fairness values within the model, referred to the 'baseline.. This baseline, along with the coordinates of arbitrary bias- miatmaitii method, form enclosed region whose area quantifies the trade-off. When evaluating two bias mitigation methods, the one with the larger area is considered to have achieved better lairnes-accuraccy trade-off. We illustrate this approach in Fig. 2 (HM) takes into account both accuracy and fairness in the form of HM = 24XF A+F: where A represents accuracy and stands for fairness. We apply the same rationale as various works that use Fl score (Lesota et al., 2022); wherein harmonic mean formulation has been used to balance two diverging objectives (Li al, 2023
et al. 2021 Dutta et al.
(2) Harmonic Mean: The Harmonic Mean
4 Experiment setup
Datasets. We conduct all the experiments based on three popular datasets, FF++ (Rossler et al. 2019
CelebDF (Li et al.) 2020), and WildDecpfake (Zi ct al. 2020). FF++ compriscs 1000 Bascling and 4000 forged videos with several visual quality levels, raw (no compression). high quality, and low quality. Ce 7314 face sequences obtained from 707 doepfake vidoos. Since both the FF++ and CelebDF provide only the video-level labels, we sample
contains 590 real and 5639 fake videos. WildDeepfake comprises
frames out of these videos using FFMPEG Tomar 2006 and perform facial cropping on these frames using
the ArcFace detection model (Deng et al. 2018)

---

Published in Transactions on Machine Learning Research (10/2025)
Table 1: Results on FF++. Best results in each column are bold and second-best results are underlined.
Bias Metrics (%6)
Detection Metrics
Methods
Backhones
Conder
Skin Tane
Intersection
Overall
Gyen
Feee
FEo
GEPR
PBo
GyPR
FFPR
FEO
AUCT
FPR]
TPRT
ACCT
EllicientNet-B3 B3
1.97
1.97
8.15
11.05
10.88
32.19
14.33
22.65
44.13
91.72
20..25
97.21
94.09
RECCE
1.27
1.27
9.14
18.81
29.65
25.07
30.26
69.38
52.34
98.05
21.20
98.21
94.74
Raseline
EffiientNet-B4
1.97
1.97
7-86
11.56
10.86
34.12
23.89
20.36
42.13
95.91
20.22
97.21
94.09
MASDT
1.38
19.71
11.61
18.07
11.32
11.46
96.21
3.65
97.13
97.
2.82
2.82
10.54
18:37
91.85
18.09
12.02
33.74
40.77
97.84
8.44
96.27
U8. 10
Avsrage
1.88
1.88
11.08
14.89
14.02
24.17
19.72
32.
50.16
96.55
14.75
97. 21
95.72
ElficientNet-B B3
0:23
0.23
1:42
1.71
0.58
12.38
6.30
12.32
12.85
91.37
8.06
89.60
89.66
DRO,"
BfleientNet-R44
RECCE
0.54
0.33
0.34
0.33
3.84
5.46
11.40
6.15
17:00
9.08
19.80
11.71
20.27
15.03
35.28
24.97
51.11
64.89
93.90
98.32
1,96
7.99
98.10
96.48
96.
95. .8
Hashimoto ot al. 2018
MASDT
0.64
0.64
15.08
6.51
9.04
7:24
12.99
13:04
41.15
98.22
3:22
98.966
97.
AltFrerzing
2.67
9.53
17.36
11.55
31.60
26.19
98.86
8.03
97.
98. 75
Aneraee
0.88
0.88
7.81
6.77
10:18
13.72
13:222
23.44
45.24
96.775
6.85
96.
95.71
EllacieniNet-B B3
0.35
0.35
6.65
5.88
1.78
12.91
5.62
12.89
14.95
93.37
8.19
91.11
91.01
MMD
RECCE
0.16
0.16
6.19
7.14
10.18
9.19
18.98
20.08
06.46
96.96
8.42
93.65
93.53
EffeientNet-B44
0.39
0.30
2.50
8.72
13.00
15.05
10.34
25.47
30.50
92.64
1.58
97.08
95.60
Deka & Suiherland
MASDT
0.32
14.20
5.87
10.20
11.44
29.18
97.04
AltFreezing
2.02
2.02
7:06
3.74
7.18
12.61
7.78
22.30
21.01
97.82
5.78
96.14
98.05
Anerae
0.65
0.65
7.34
6:27
8.42
11.13
10.58
411.22
95.61
5.10
95.80
95,000
Eelicien B3
0.67
0.67
6.36
11:48
9.58
13.50
12.887
19.34
46.08
97.01
8.40
9/2.87
12265
DAG-FDD
RECCE
0.75
0.75
5.71
11.08
19:41
19.33
25.40
38.17
76.21
98.33
12.01
96,50
95.23
0:61
0.61
4.86
13:81
20.35
17.50
30.88
95.50
912
Ju et ml. 2021
MASDT
0.58
0.58
18:70
10.00
9.81
7.36
16.00
11.27
30.39
96.95
5.67
97.63
96. 29
AltFreezing
2.64
10.01
11:30
32.62
37.66 69
7.87
95.59
97.44
Auerage
1.05
1.05
8.53
12.35
12.90
15.63
16.62
27.06
48.61
96.71
11.11
95.70
95.31
NelcientNet-B3
0.34
0.34
0:53
6.79
11.67
12.63
8:43
12:37
43.72
95.26
8.222
91.43
91.49
DAW-FDD
RECCE
0.45
0.45
7.95
6.90
9.96
13.95
21.54
25.44
54.95
98.35
8.15
94.59
94.10
RffelentNetB44
0.55
0.55
3.71
13.65
17:35
20.30
15.34
36.000
56.19
90.44
2.00
96.91
95540
Ju 2024
MASDT
0.45
17.71
8.62
9.87
12.05
13:29
AltFreezing
2.72
2.72
10.03
6.20
9.39
17.71
11.79
52.21
37.54
97.61
8.19
95.91
97.81
Average
0.90
11.45
14.89
14:23
23.90
45.16
95.85
6.30
95.33
95.30
AieieenNettni 13
0.16
0.16
5.78
3.307
3.59
10.15
4.74
13.08
45.74
92.87
8.50
96.19
96366
AnirAlign (Ours)
RECCE
0.19
0.19
4.96
6.02
10.03
10.50
14.21
21.58
57.54
96.74
8.53
93.60
93.03
BfficienNee-144
0.39
0.39
2.07
6.54
12:29
11:79
8.30
22.95
45.08
91.78
1.47
97.24
95.75
MASDT
0.29
0.29
12:00
4:01
5.28
5.38
8.26
11.72
23.177
97.22
1.67
98.11
98%
AltFreeiing
6.03
10.
5.90
23.
5.36
96.26
Average
0.55
0.55
6.17
4.93
7.65
9.60
8.24
18.62
00.53
95.32
5.12
96.08
96.81
Evaluation metrics. To assess fairness, we employ five bias metries. First, following (Wang et al.
2022
et al. 2024), we use the maximum difference in false positive rate (FPR) gap, equal FPR, and equal odds, denoted by GFPR, FPPR, and Fgo respectively. These metrics are defined as:
GFPR == max V2,,yE2
|PR.. - FPR2,1
(10)
Em Il 1M=11,=
FFPR ==
Σ
(11)
E [1 X(=
Em [Y;=0)
2,EZ
1 11,=112=
Σm 18=11== E [[)
FEO = Σ
(12)
Gi=1 1 Ymmg
4E2g==
where FPR2, represents the FPR scores of group 2i, 1 denotes the indicator function, and q represents the forgery label (1 is real and is fake). Also note that in the special case when Z corresponds to gender, i.e.
(= metrics GFTR and Frrn return the same value. See the Appendix for the definition of demographic parity difference (DPD) and demographic parity ratio (DPR) fairness metrics. We report DPD and DPR results in the Appendix.
Finally, to assess the performance of different deepfake detectors, we utilize four widely-used metrics (Ju et al. 2024): ; the area under the curve (AUC), FPR, true positive rate (TPR), and top-1 accuracy (ACC).
Baseline methods. To validate the efficacy of our proposed loss term, Alsrs we integrate it into the training process of 5 statc-of-the-art deepfake detector backbones: RECCE (Cao et al. 2022 MASDT Das et al. 2023). AltFreezing (Wang et al. 2023). EfficientNet-B. (Tan & Le 2019), and EfficientNet-BB Coccomini et al. 2022). The objective is to assess the impact of L Fair on the fairness of these models.
7

---

Published in Transactions on Machine Learning Research (10/2025)
Table Results on CelebDF. Best results in each column are in bold and second- best results are underlined.
Bins Metries (%61
Detection Metries (%)
Metbods
Backbones
Gender
Skin Tome
Intersection
Overall
Gyra
Fren
Fisu
Gyenn
Fvru
Fizo
Gyrn
Fyen
Euo
AUCT
FPR.
TPR
ACC1
EllicientNee-B
3.82
3.62
T.51
11.37
9.85
14.09
21.25
45.47
70.71
91.13
9:25
92.27
85.01
RECCE
2.71
2.71
3.14
18.81
27.65
30.07
30.26
67.38
80.34
94.05
12.20
95.21
95.74
Roseline
BffrientNet-B4
1.21
1.21
5.15
10:05
20.86
34.12
13.38
22.65
40.13
93.91
10.25
97.21
93.009
MASDT
0.48
0.48
3.71
8.64
10.89
9:39
12.07
15.32
16.46
96.61
5.05
94.13
96. 60
AltFreesing
1.71
8.52
7.25
12:63
22.65
23.83
40.13
95.91
8.25
96.09
Aoeruge
1.59
1.99
5.61
11:22
16:38
22.06
20.16
38. 09
400.06
94.92
9.00
94. 50
96.
EifiieentNe--33 B3
3.40
3.40
6.83
8.10
9.20
12.36
11.55
39.60
66.95
93.63
6.32
93.
96.62
DRO,*
RECCE
0.84
0.84
4.66
6.85
7,80
10.71
13:27
22.97
71.89
91.32
90:98
BllicieutNet-B -4t
0.14
0.44
3.64
12:10
17.00
19.89
15.00
35.28
53.11
90.90
11.96
98.10
94.77
Hushimoto
2018
MASDYT
0.22
0.22
3.68
10.77
9.21
9.93
15.04
16.15
95.29
96.37
Alt Freesing
0.33
0.33
6:23
6.16
10.83
17.91
11.14
32.32
42.85
941.37
6.02
90.01
95, 95.66
Averuge
1.05
1.05
4.81
8.10
11.11
14.022
12:18
29.01
50.19
93.10
8:29
93.75
94,88
EllicieniNet-R B3
3422
3.02
6.06
6.74
11.18
10:61
7.78
51.17
92.11
96.25
MMD
RECCE
0.76
0.76
1.89
7.11
8.18
12.19
13.98
21.08
61.16
9/,
10.12 12
92.65
91. 91.53
ElicienttNe-
0.29
2.59
8.72
13.03
15.05
10.34
37.50
92.64
97.00
94.60
Deka & Sutberland 2022
MASDT
0.31
0.31
3:20
6.89
7.91
7.87
9.20
12.43
11.18
96.01
4.40
92.02
97. .2
AltFreezing
0.52
0.52
1.54
6.85
10:99
18.83
11.32
28.84
4...55
94.557
6.91
96.15
97.01
Average
0.98
0.98
4.06
7.27
10.26
12.91
10.52
23.852
42.45
94.08
8:24
94.01
95,
ElicientNee-- B3
3.65
3.65
8,01
9.18
9.27
11:12
12,30
12.62
67.96
B3.072
10.72
92.59
95.20
DAG-FDD
RECCE
1.45
3.71
12.68
17.41
19.33
36.17
64.24
94.33
10.01
933.23
FffclentNee-B44
0.61
0.61
4.86
12.81
16.85
20.55
17.50
30.88
52.63
92.15
11.58
95.60
94.92
Ju et al. 2024
MASIET
0.58
0.58
3.70
8.60
10.84
9.36
12.04
15.27
16.39
95.95
5.55
93.63
96.29
AltFroriing
0.97
0.97
6.63
7.81
11.83
18.24
20.81
39.34
46.08
97.20
7.02
94.75
96.65
Anerage
1.45
1.45
5.38
10:22
13.24
16.78
155661
32: 96
40.46
94.53
8.98
94-47
95.26
EffrieenNee-I3
3.56
3.56
7.00
7.20
6.39
9.18
11.79
42.24
67.45
93.41
95.17
DAW-FDD
RECCE
0.95
0.95
4.75
6.119
7.96
11.95
13.54
23.44
02.95
92.35
12:15
94.559
92.10
BllicieuNNet-B B4
0.45
0.45
3.71
12.65
17.35
20.30
15.34
36.00
54.19
91.44
12.00
96.90
93,-10
Ju 2024
MASTYT
0.38
0.38
3.71
8-62
10.86
9.37
12.005
15:29
16.002
95.56
4.55
94.81
96.13
Freesingg
0.43
0.43
5.34
5.96
10.00 02.62
17.82
11.53
37.57
433722
96:30
7.26
93.33
97-49
Average
1.15
8.28
13.72
30.91
18.887
93.81
9.19
EllicierNNet-B B3
2.67
2.67
4.39
6.19
5.54
7.77
9.08
23.78
41.339
94.71
6.61
133.26
966.3
FaizAlign
RECCE
0.86
0.86
1.98
7.32
8.30
10.50
14.21
21.58
55.51
93.71
7.53
91.60
91.03
BfliceenNee-B44
0.29
0.29
2.77
6.54
12:99
11.79
33.00
93.75
11.47
97.24
94.75
(Our)
MASDT
0.19
0.19
3.00
5.01
7.88
7.38
9.16
12.72
122.07
96.23
5:10
92.11
27. 1G
AltEreezing
0.29
1.86
5.78
10.17
16.95
31.09
45.74
97.87
97.86
Anerage
0.56
0.86
4.00
5.97
8.98
10.88
11.20
23.02
37.56
95.27
7.26
94.23
96.05
Table 3: Bias for ethnicity vs. skin tone.
Ethnicity
Skin tone
Methods
BB
GePR
FFPR
FEO
GFPR
FFPR
Fgo
MASDT
10.56
12.99
11.18
14.64
11.89
11.39
Baseline
AltFreezing
8.11
7.21
21.48
18.37
9.85
18.09
For a thorough analysis, we benchmark our approach against 4 state-of-the-art mitimmiggg methods: DAG-FDD (Ju et al. 2024), DAW-FDD (Ju et al. 2024). DRO-x² (Chai ct al. 2022), and MMD loss (Deka & Sutherland 2022p. Furthermore, to establish baseline performances, each model is also trained without any fairness-enanncinn module.
Implementation details. All experiments are conducted using the PyTorch framework Paszke et al. 2019 on up to 8 NVIDIA A100 PCIE GPU cards. We train all methods using the AdamW optimizer (Lshhhilov & Hutter 2017) with a batch size of 32, a maximum of 100 epochs, and a learning rate of 0.001. The optimizer employs first and second momentum decays of 0.9 and 0.999, respectively. Additionally, we use a weight decay of 0.01 to refine the training process. The learning rate is adjusted using a step scheduler, which decreases the learning rate by factor of 0.5 every 5 epochs. The video frame input size is set to 380 pixels, with training augmentations including resizing, normalization and horizontal flipping. For the face detection process meneioned in Section 3 we use the InsightFace loolkit (Deng el al. 2022). For the e used in Equation Eq. (2) we adhere to 0.01 as per the design choice outlined in (Luo & Ren 2021). Furthermore, we employ Gaussian kernel for both the embodding space and the sensitive attribute space. The sigma paramcter in the Gaussian kernel is dynamically set to the mean of the pairwise squared Euclidean distances for each mini-batch.
8

---

Published in Transactions on Machine Learning Research (10/2025)
Oriinnl
97.75
0.5
12%
0.5
Uittmnection tint
Fairea
91.25
Fox Ikferrodiiis
0.4
ALC
91.00
25% Manako
96.75
0.2
Marorin
96.50
Raseline
96.25
0.3
0.2
0.1
0.5
0.25
0.5
0.75
1.0
Fro (Interaction)
Figure 2: Left: schematic of the Fairea Hort et al.
2021
Right: effect of À on bias mitigation.
Table 4: Cross-dataset results.
Methods
BB
GFer
FEO
AUC
Acc
Baseline
MASDT
1.55
10.51
80.21
88.76
AltFreezing
4.11
18.67
92.61
95.24
Ours
MASDT
0.78
6.34
82.78
91.34
AltFreezing
0.91
6.78
94.35
95.87
5 Results
Performance. We present. thorough analysis of the performance of approach on the FF++ and CelebDF datasets in Tables 1 and respectively. Additionally, we present the results of our method on the WildDeepfake dataset in the Appendix. First, evaluating the average results of FairAligu against the baselines demonstrates that our approach substantially promotes fairness across all three metrics for both gender and skin tone, as well as the intersoction of the two. A similar trend is observed for the additional two metries (DPD and DPR) presented in the Appendix. Comparing the performance of our method averaged across backbones, against other chirness-promolien fairness scores, with exceptions in the intersection group, where MMD achieves marginally higher scores for FEPR and Fgo the FF++ datasct. Similarly, MMD obtains slightly better results in the interscction group with GFPR CelebDF. While our method achieves the highest detection AUC on the CelebDF dalaset with the AltFreezing backbone, it exhibits a slightly lower AUC compared to the baseline on the FF++ dataset. In the next subsection, we will analyze this outcome in detail and demonstrate that, considering the trade-off between detection and fairness, our method remains the strongest.
solutions, we observe that our method generally achieves the best.
Delving deeper into the results, we observe that on FF++, Fair applied to Efficient.Nt--33 achieves the lowest GFpR (and similarly FFPR) of 0.16% in gender, in a tie with MMD when applied to the RECCE network. Similarly in CelebDF, FairAlign with MASDT achieves the lowest GFPR (and FFPR) in gender with a score of () 0.19.. For FEO our method obtains the best performance on gender when coupled with EfficientNet-B4 on FF FF++, while on CelebDF, MMD results in the best outcome with the same backbone. For skin tone, on FF++, MMD achieves the best performance for GFpr along with the AliFreezing deepfake detection method. However, for FFPR and FEO, our method obtains the best results with EfficientNet-BB and MASDT backbones respectively. On CelebDF, our method consistently achieves the lowest bias when coupled with MASDT, ElficientNei-B3, and MASDT, for the three metrics respectively. For the intersection of the two
Table 5: Faimnsss-accuracy trade-off on the FF+ and CelebDF datasets.
RECCE
EIL-B4
MASDIT
Celobii
FF
ColebD
FF+++
CoatDF
FF++
CoRtDF
FF++
CalebD
Paires?
Fairos" HM-
Falredt
HM1
Falrest
HM1
Fairws
HM
Falres?
HM
Fairest HM
Pairas
HM]
Paires?
HMI
Patres?
HM
DROC
Hashimote
2018
1.36
1.22
1.30
0.03
1.65
0.05
0.08
MMDD
Deca & Sucled aus A62
0.06
1.31
0.03
1.27
0.05
1.17
0.03
1.16
0.06
1.35
0.08
1.36
0.04
1.51
0.03
1.50
0.03
1.62
0.03
1.32
DAG-FDI Lm 47
0.04
1.36
0.03
1.13
0.04
1.12
0.02
1.17
0.06
1.25
0.00
1.24
0.04
1.49
0.08
1.65
0.01
1.42
0.05
1.34
0.05
0.04
1.33
FairAlgg 1OBlS
0.07
1:31
0.04
1.36
0.05
1.24
0.04
1.23
0.44
1.30
0.8
1.43
0.06
1.59
0.04
1.72
0.06
1.50
0.04
1.36

---

Published in Transactions on Machine Learning Research (10/2025)
Table Fairness metrics for Gaussian vs, linear kernel.
Backbone
Gender
Skin tone
GppR
FFPR
FEO
Gppr
FFpr
FEO
Fair Align (Caussian Kernel)
Nfficient B3
0.16
0.16
5.78
3.97
3.59
10.15
DificientNet-BRe B4
0.39
0.39
2.07
6.54
12.29
11.79
Linear Kernel
Effiient B3
0.46
0.46
6.69
6.18
5.81
13.51
ElficientNet-Bl
0.48
0.48
2.75
10.42
13.78
16.64
(gender and skin tone), on FF++, our approach outperforms the others based on GreR using Ffficient while MMD shows better performance on FFPr and FEO using MASDT and AltFreezing respectively. On CelebDF, MMD achieves better intersection results based on GFPr and FFPR using EfficientNet-BB and MASDT, while ours outperforms others based on Feo using the MASDT method.
Comparing the bias metries for gender with those of skin tone, we notice considerably higher, i.e, more biased, values for skin tone. We believe this due to two main reasons. First, gender is currently defined as binary class in the dataset, whereas between the number of classes is an important reason behind skin tone showing more bias as mcasured by the metrics. The second reason could be that skin lone is inherently more challenging in terms of bias mitigation, for instance due to the heavily imbalanced nature of the datasets in this regard. We present the distributions for gender and measured skin tones the Appendix, where we observe a less balancod, i.c., long-tailen distribution for skin tones. To compare the bias related skin color and ethnicity, we present the fairness metrics for both in Table 3 for the MASDT and AltFreezing backbones. As demonstrated, there is significant difference in the bias associated with ethnicity compared to that associated with skin tone. For instance, the AltFreczing backbone achieves a GepR of 8.11 for ethnicity, whereas it achieves a GFPR of 18.37 for skin lone. This indicates that skin lone is a distinct/ setpinc//coopleentrryy source of bias, introducing more severe bias into the system. We also test the performance of Fair on ethnicity sensitive attribute. The results are presented in Appendix.
definition of skin tones consist of 10 unique classes. This difference
Our method demonstrates strong deepfake detection performances across all four metrics for FF++ and CelebDF datasets. On the FF++ dataset, FairAlign obtains very competitive results, while on CelebDF, it generally achieves better performances with respect to others. A detailed comparison between both performance aspects (fairness alongside deepfake detection) highlights that drawing a high-level conclusion about the best fairness promoting approach remains complicated and nuanced when considering the deepfake detection results. This is especially the case on FF++ where the tradc-off between fairness and performance seems more complex. The next subsection discusses this phenomenon.
To investigate the generalization and fairness of the proposed method on unseen data, we conduct cross-dataset experiments. We train MASDT and AltFreezing on FF++, both without any fairness-cnhancing loss baseline and with the proposed loss. We then evaluate their performances on CelebDF. The results are presented in Table 4 where we observe that for deepfake detection with MASDT, our proposed method outperforms the baseline by nearly 3% in accuracy and over 2% in AUC, indicating that our loss can help improve generalization on unseen data while mitigating bias. Furthermore, the proposed loss achieves significantly lower FEO and GFER compared to the baseline, highlighting enhanced group fairness. Similar results are observed for the AltFreezing backbone, as presented in the table.
Finally, evaluate the computational overhead of our method on CelebDF. First observe that at test time, our method does not add any overhead given that the loss is only measured during training. Next, when evaluating the overhead during trainig, we observe that our method averages 167 seconds per epoch, compared to 187 seconds for (Hashimoto et al. 2018) and 142 seconds for (Deka & Sutherland 2022
Tradc-off analysis. To perform an analysis on faiuncss-accuracy trado-off, we use 1/ FBO following (Ju et al. 2024) to represent fairness performance for the intersoction of gender and skin tone to capture a holistic view of both sensitive attributes. Moreover, following (Wang al. 2023) select. AUC to represent the
10

---

Published in Transactions on Machine Learning Research (10/2025)
Table 7: Fairness metrics for gender sensitive attribute different scenarios: 1. Baseline: no fairness enhancing
method is applied 2. FairAlign with gender as the sensitive attribute 3. FairAlign with skin tone as the sensitive attribute.
Backbone
Gender
GFpR
FFPR
Fgo
Baseline
MASDT
1.38
1.38
19.71
AltFroozing
2.82
2.82
10.54
FairAlign (Targeting Gender)
MASDT
0.29
0.29
12.00
AlcFreezing
1.74
1.74
6.03
Fair FairAlign (Targeting Skin Tone)
MASDT
0.87
0.87
18.79
A Freexing
2,80
2.80
10.13
the deepfake detection performance of different methods. Using these metrics, we present Fairea and HM as discussed in Section 3 and present the results in Table G We observe that most of the time Alig outperforms other bias-mitigation methods across different backbones when considering both fairness and accuracy. For instance, FairAlign achieves the highest Fairea and HM scores on CelebDF for MASDT backbone. On the FF++ dataset, the best Fairea score is obtained by our method for the Efficient-Bi backbone. An important observation from this analysis is that while theoretically both metrics (Fairea and HM) are capable of quantifying the fairness-accuracy trade-off, Fairea seems to produce less discriminatory outcomes. In contrast, HM generates a wider range of values, offering more effective and discriminative means of capturing the trade-off.
Gaussian kernel effectiveness. To validate the effectiveness of employing a Gaussian kernel, we conduct an experiments. In this experiment, we compare our proposed FairAlign with Gaussian kernel against a variant using a standard linear kernel (Kiineur (24, xy) = x/x;). We use FF++ dataset, treating gender and skin tone the sensitive attributes. The results are presented in Table 5 The results demonstrate clear and consistent trend: the Gaussian kernel outperforms the linear kernel in reducing bias across both backbones and sensitive altribules. For instance, with the EfficientNet-B3 backbone and gender as the sensitive altribule, our proposed method with the Gaussian kernel achieves GFPR of 0.16, a nearly three-times improvement over the lincar kernel's score of 0.46.
Cross-Attie Attribute impact. Existing work show that mitigating bias with regards to one sensitive altribute may exacerbate another. To directly investigate this in the case of Fair we conduct an experiments analyze the cross-attribute impact of Fair.lign. Specifically, on the FF++ dataset with the MASDT and AltFreezing backbones, we trained three separate models: 1. Baseline with no fairesss-nhhancnn method. 2. A model where FairAlign is applied to mitigate gender bias directly. 3. A model where Alir is applied to mitigate skin tone bias. For all the models described, we report fairness metrics for gender as the sensitive attribute in Table 7 The results show that even when Fair directly targets only skin tone, the gender bias still reduced across all metrics compared to the baseline. For example, on Alt.Frezzing Fgo improves from 10.54 to 10.13, and on MASDT, GFPR improves from 1.38 to 0.87. This demonstrates that Aaig does not exacerbate the non-target bias beyond the baseline. Instead, yields a positive, albeit smaller, fairness improvement non-target attributes.
Impact of À on fairness. We further investigate the impact of the À hyperparameter in Eq. 9 on the bias metrics. Fig. 2 illustrates the relationship between FPPR (Intersection), AUC, and various values of X using the AltFreezing backbone on the CelebDF dataset. The figure demonstrates a significant reduction in bias, especially between = 0.50 and À 1.0, and an incrcase in the AUC. We illustrate the impact of for the remainder of the bias metrics the Appendix. According to these experiments, we set 1.0 throughout all the experiments.
11

---

Published in Transactions on Machine Learning Research (10/2025)
6 Conclusion
We introduced FairAlign, novel loss term aimed at enhancing group fairness in deepfake detection by aligning conditional embedding distributions in a high-dimensional kernel space. Additionally, we consider skin tone as an important factor toward bias in deepfake detection for the first time. Our method shows state-of-the-art or competitive performance in mitigating bias, while maintaining a strong balance when considering the fairness-aceuracy trade-off.
Acknowledgments
This work was partially funded by Irdeto Canada Corporation and the Natural Sciences and Engineering Research Council of Canada (NSERC), and was enabled in part by the support provided by the Digital Research Alliance of Canada.
References
R. Bhatia, Tanvi Jain, and Y. Lim. On the bures wasssswwsserstein distance between positive definite matrices. Erpositiones mathematicae, 2017.
J. Cao, C. Ma, T. Yao, S. Chen, S. Ding, and X. Yang. End-to-end reconstuctio--caasicciiionn learning for face forgery detection. IEEE/CVF Conference Computer Vision and Pattern Recognition, 2022.
J. Chai, T. Jang, and X. Wang. Fairness without demographics through knowledge distillation. Advances in Neural Information Processing Systems, 2022.
Sheng Chen, Yang Liu, Xiang Gao, and Zhen Han. Mobilefacenets: Efficient cnns for accurate real-time face verification mobile devices. Chinese Conference Biometric Recognition, 2018.
D. A. Coccomini, N. Messina, C. Gennaro, and F. Falchi. Combining efficientnet and vision transformers for video deepfake detection. International Conference on Image Analysis and Processing, 2022.
Sayantan Das, Mojtaba Kolahdouzi, Levent Özparlak, Will Hickie, and Ali Etemad. Unmasking deepfakes:
Masked autoencoding spatiotemporal transformers for enhanced video forgery detection. IEFF International Joint Conference Biometrics, 2023.
Namrata Deka and Danica J. Sutherland. Mmd-b-fair: Learning fair representations with statistical testing.
International Conference on Artificial Intelligence and Statistics, 2022.
Jiankang Deng, Guo, and S. Zafeiriou. Areface: Additive angular margin loss for deep face recognition. Computer Vision and Pattern Recognition, 2018.
Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Insightface: 2d and 3d face analysis project. http:://ggthhbocooo//eeesssi//// www.://eeppiiaigghh///niiihheeee https://github..
2022.
B. Dolhansky, J. Bitton, B. Pflaum, J. Lu, R. Howes, M. Wang, and C. C. Ferrer. The deepfake detection challenge (DFDC) dataset. ar arXiv:2006.0739 2020.
Sanghamitra Dutta, Dennis Wei, Hazar Yueksel, Pin-Yu Chen, Sijia Liu, and Kush Varshney. Is there a trade-off between fairness and accuracy? A perspective using mismatched hypothesis testing. International Conference on Machine Learning, 2020.
Kenji Fukumizu, Francis R Bach, and Michael I Jordan. Kernel dimension reduction in regression. The Annals of Staliskics, 2009.
Arthur Gretton, Karsten M. Borgwardt, Malte Rasch, Bernhard Schölkopf, and Alexander Smola. kernel two-sample lest. Journal of Machine Learning Research, 2012.
Xiaotian Han, Jianfeng Chi, Yu Chen, Qifan Wang, Han Zhao, Na Zou, and Xia Hu. Fib: A fair fairness benchmark for mn-rrocssing group fairness methods. International Conference on Learning Representations, 2024.
12

---

Published in Transactions on Machine Learning Research (10/2025)
T. Hashimoto, M. Srivastava, H. Namkoong, and P. Liang. Fairness without demographics in repeated loss minimization. International Conference on Machine Learning. 2018.
Caner Hazirbas, Joanna Bitton, Brian Dolhansky, Jacqueline Pan, Albert Gordo, and Cristian Canton Ferrer. Towards measuring fairness in ai: The casual conversations dataset. IEEE Transactions on Biometrics, Behavior, and Identity Science, 2021.
Courtney M. Heldreth, Ellis P. Monk, Alan T. Clark, Candice Schumann, Xango Eyee, and Susanna Rieco. Which skin tone measures are the most inclusive? an investigation of skin tone measures for artificial intelligence. ACM Journal Responsible Computing, 2023.
Max Hort, Jie M Zhang, Federica Sarro, and Mark Harman. Fairea: A model behaviour mutation approach to benchmarking bias mitigation methods. ACM Joint Meeling on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 2021.
Neil Joshi and Phil Burlina. Ai fairness via domain adaptation. arxi:: $104.0110, 2021.
Y. Ju, S. Hu, S. Jia, G. H. Chen, and S. Lyu. Improving fairness in deopfake detection. IEEE/CVF Winter Conference on Applications of Computer Vision, 2024.
Newton M. Kinyanjui, Timothy Odonga, Celia Cintas, Noel C. F. Codella, Rameswar Panda, Prasanna Sattigeri, and Kush R. Varshney. Estimating skin tone and effects on classification performance in dermatology datasets. Advances in Neural Information Processing Systems Fair Machine Learning for Health Workshop, 2019.
Oleg Lesota, Stefan Brandl, Matthias Wenzel, Alessandro B Melchiorre, Elisabeth Lex, Navid Rekabsaz, and Markus Schedl. Exploring cross-group discrepancics in calibrated popularity for accuracy /fairness tradc-off optimization. RecSys, 2022.
J. Li, Y. Moskovitch, and H. V. Jagadish. Denouncer: detection unfairness classifiers. VLDB Endowment, 2021.
X. Li, P. Wu, and J. Su. Accurate fairness: Improving individual fairness without trading accuracy. AAAT Conference Artificial Intelligence, 2023.
Y. Li, X. Yang, P. Sun, H. Qi, and S. Lyu. Celeb-DF: A large-scale challenging dataset for deepfake forensics. IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020.
Camille Olivia Little, Michael Weylandt, and Genevera T Allen. To the fairness frontier and beyond: Identifying, quantifying, and optimizing the fairncss-aceuracy pareto frontier. NNN:22206.0071, 2022.
Loshchilov and F. Hutter. Decoupled weight decay regularization. International Conference on Learning Representations, 2017.
You- Wei Luo and Chuan-Xian Ren. Conditional bures metric for domain adaptation. IEEE/CVF Conference Computer Vision and Pattern Recognition, 2021.
Subha Maity, Debarghya Mukherjee, Mikhail Yurochkin, and Yuekai Sun. Does enforcing fairness mitigate biases caused by subpopulation shift? Advances in Neural Information Processing Systems, 2021.
M. Masood, M. Nawaz, K. M. Malik, A. Javed, A. Irtaza, and H. Malik. Deepfakes generation and detection: State-of-the-art, open challenges, countermeasures, and way forward. Applied Intelligence, 2022.
Mukherjee, F. Petersen, M. Yurochkin, and Y. Sun. Domain adaptation meets individual fairness. and they get along. Advances in Neural Information Processing Systems, 2022.
A. V. Nadimpalli and A. Rattani. GBDF: Gender balanced deepfake dataset towards fair deepfake detection. ar 2rXi::2207.100266 2022.
13

---

Published in Transactions on Machine Learning Research (10/2025)
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-ppeformance deep learning library. Advonces in Neural Information Processing Systems, 2019.
M. Pu, M. Y. Kuan, N. T. Lim, C. Y. Chong, and M. K. Lim. Fairness evaluation in deepfake detection models using metamorphie testing. International Workshop on Melamorphic Testing, 2022.
A. Rossler, D. Cozzolino, L. Verdoliva, C. Riess, J. Thies, and M. Nießner. FaceForensics+ Learning to detect manipulated facial images. IEEE/CVF International Conference Computer Vision, 2019.
Robert S. Schulman and Elliot M. Cramer. Comparison of the accuracies of some regression algorithms. Journal of Statistical Computation and Simulation, 1975.
Megha Srivastava, Hoda Heidari, and Andreas Krause, Mathematical notions vs. human perception of fairness: A descriptive approach to fairness for machine learning. Conference Knowledge Discovery and Data Mining, 2019.
Florian Steinke and Bernhard Schölkopf. Kernels, regularization and differential equations. Pattern Recognition, 2008.
M. Tan and Q. Le. EfficientNet: Rethinking model scaling for convolutional neural networks. International Conference on Machine Learning, 2019.
Suramya Tomar. Converting vidco formats with fimpeg. Linuz Journal, 2006.
L. Trinh and Y. Liu. An examination of fairness of ai models for deepfake detection. International Joint Conference on Artificial Intelligence, 2021.
Thanh-Dat Truong, Ngan Le, Bhiksha Raj, Jackson Cothren, and Khoa Lun. Fredom: Fairness domain adaplation approach to semantic scene understanding. IEEE/CVF Conference on Computer Vision and Pottern Recognition, 2023.
Jialu Wang, Xin Eric Wang, and Yang Liu. Understanding instance-level impact of fairness constraints. International Conference on Machine Learning, 2022.
Qianwen Wang, Zhenhua Xu, Zhutian Chen, Yong Wang, Shixia Liu, and Huamin Qu Visual analysis of discrimination in machine learning. IEFF Transactions on Visualization and Computer Graphies, 2021a.
Yuyan Wang, Xuezhi Wang, Alex Beutel, Flavien Prost, Jilin Chen, and Ed H. Chi. Understanding and improving fairness-accuracy trade-offs in multi-task learning. Conference on Knowledge Discovery and Data Mönving, 2021b.
Z. Wang, J. Bao, W. Zhou, W. Wang, and H. Li. AltFreezing for more general video face forgery detection. IEEE/CVF /VV Conference on Computer Vision and Pattern Recognition, 2023.
H. Weerts, M. Dudík, R. Edgar, A. Jalali, R. Lutz, and M. Madaio. Fairlcarn: Assessing and improving fairness of ai systems. arXiv:2303. 16626, 2023.
Michael L. Wick, Swetasudha Panda, and Jean- nan-Baptiste Tristan. Unlocking fairness: trade-off revisited. Neural Information Processing Systems, 2019.
Han Xu, Abhijit Sarkar, and A. Lymn Abbott. Color invariant skin segmentation. IEEE/CVF Conference Computer Vision and Pattern Recognition Workshops, 2022a.
Y. Xu, P. Terhörst, K. Raja, and M. Pedersen. A comprehensive analysis of AI biases in deepfake detection with massively annotated databases. arXiv:2208. 5,5554,, 2022b.
H. Zhang, N. Shahbazi, X. Chu, and A. Asudeh. Fairrover: explorative model building for fair and responsible machine learning. Workshop on Data Management for End- To-Fnd Machine Learning, 2021.
14

---

Published in Transactions on Machine Learning Research (10/2025)
Zhen Zhang, Mianzhi Wang, and Arye Nehorai. Optimal transport in reproducing kernel hilbert spaces: Theory and applications. JEEE Transactions on Pallern Analysis and Machine Intelligence, 2020.
Bojia Zi, Minghao Chang, Jingjing Chen, Xingjun Ma, and Yu-Gang Jiang. Wilddeepfake: A challenging real-world dataset for deepfake detection. In Proceedings of the 28th ACM International Conference Multimedia, 2020.
15

---

Published in Transactions on Machine Learning Research (10/2025)
A
Appendix
A.1 Skin Tone Scale with Representative Face Crops
We used Monk skin tone scale to compute the skin color of the cropped faces. To visually demonstrate the results, we add the panel of the 10-level Monk skin tone scale with representative face crops for each bin. Figure 3 shows the result. This figure shows that within each skin tone, there exist several ethnicity groups. This is line with the previous finding in the literature that skin tone and ethnicity are different concepts. We analyzed and mitigated the bias for both ethnicity and skin color sensitive altributes in this paper.
A.2 Fairness Metric Definition
Following (Weerts et. al. 2023); we use demographic parity difference (DPD) and demographic parily ratio (DPR) fairness metrics, formulated as:
DPD = max P(ý = 112 = =z;) -min P(P = 112 = 2i) #:2
(13)
xE2
DPR minzez P( 1| 2,)
(14)
ma p( 1| 24)
DPD measures the disparity in positive outcomes across different groups, with ideal value of 0 indicating disparity. Conversely, DPR assesses the relative disparity, with an ideal value of suggesting equal positive outcome rates across all groups.
A.3 Additional Results
A.3.1 Metrics
In addition to utilizing bias metrics GPPR, FFPR, and FEO our experiments, we also employ DPR and DPD metrics and present the results Tables g and 9 We observe that the findings align with those reported in the main paper. Averaging across different backbones,
method records the lowest DPD and highest DPR for all groups categorized by gender, skin tone, and interscctional attributes. Among the backbones, AllFreezing detector consistently ranks the best or second-best for gender metrics across both FF++ and CelebDF benchmarks. Similarly, EfficientNet-B3 frequently secures the top scores for metrics related to skin tonc and intersectional groups in both datasets.
A.3.2 Performance on WildDeepfake Dataset
We report the performance of our proposed method on the WildDeepFake dataset in Table 10 For the gender sensitive attribute and its intersection with skin tone, FairAlign outperforms both DAG-EDD and the Baseline
Others
Asians
Whites
Blacks
10
9
8
7
5
4
Figure 3: Visual example of Monk skin scale along with representative face images.
16

---

Published in Transactions on Machine Learning Research (10/2025)
Table DPR and DPD results on FF++. Best results each column are bold and second best results are underlined.
Bias Metrics
Methods
Backbones
Gender
Skin Tone
Intersection
DPD 4
DPR
DPD
DPR T
DPD
DPR 1
EfficientNet-Be B3
0.29
0.79
0.38
0.64
0.55
0.56
RECCE
0.39
0.77
0.39
0.67
0.49
0.72
Baseline
EfficientNet-DD
0.69
0.46
0.73
0.53
0.66
MASDT
0.47
0.77
0.54
0.68
0.42
0.52
AltFreezing
0.38
0.67
0.47
0.73
0.54
0.65
Average
0.40
0.74
0.45
0.69
0.51
0.62
EfficientNet-BB
0.29
0.82
0.29
0.85
0.40
0.87
RECCE
0.13
0.89
0.20
0.88
0.28
0.83
DROx²
EfficientNNe-BA
0.39
0.70
0.36
0.84
0.37
0.87
Hashimoto al. 2018
MASDT
0.17
0.88
0.34
0.79
0.35
0.63
AltFreezing
0.18
0.88
0.27
0.84
0.30
0.76
Average
0.23
0.83
0.29
0.84
0.34
0.79
EfficientNNetBB.
0.20
0.85
0.19
0.87
0.35
0.89
RECCE
0.16
0.90
0.31
0.89
0.26
0.86
MMD
EfficientNet-B.
0.40
0.61
0.47
0.85
0.44
0.89
Deka & Sutherland
2022
MASDT
0.90
0.48
0.27
AltFreczing
0.14
0.90
0.28
0.85
0.24
0.83
Average
0.24
0.83
0.35
0.85
0.31
0.84
NeficientNet-1333
0.20
0.82
0.22
0185
0.32
RECCE
0.22
0.80
0.41
0.89
0.36
0.85
DAG-FDD
EfficieutNet-BB
0.30
0.72
0.27
0.85
0.24
0.88
Ju al. 2024
MASDT
0.32
0.89
0.35
0.79
0.33
0.74
AltFreezing
0.89
0.48
0.84
0.43
0.56
Average
0.27
0.82
0.35
0.84
0.34
0.78
EeficientNeetB333
0.21
0.88
0..30
0.89
0.39
0.91
RECCE
0.84
0.41
0.20
0.87
DAW-FDI
EfficieutNet-DD
0.21
0.84
0.28
0.87
0.54
0.90
Ju al. 2024
MASDT
0.28
0.92
0.46
0.83
0.17
0.86
AltFreezing
0.21
0.92
0.39
0.87
0.37
0.79
Average
0.23
0.88
0.37
0.88
0.33
0.87
EfficieutNet-BD
0.17
0.90
0.11
0.97
0.23
0.93
RECCE
0.12
0.94
0.32
0.93
0.20
0.90
FainAlign
EfficientNet-Bi
0.12
0.76
0.29
0.87
0.39
(Ours)
MASDT
0.18
0.93
0.37
0.83
0.18
0.85
AltFreezing
0.11
0.94
0.29
0.89
0.15
0.89
Average
0.89
0.28
0.90
0.23
0.90
across all three metrics. For skin tono-sensitive attributes, FairAlign achicves a statc-of-the-art Grrn of 9.01, while the Baseline with the RECCE backbone and DAG-FDD with the same backbone attain slightly better FEPR and FEO, respectively. Importantly, FairAlign achieves state-of-the-art detection performance all scenarios, demonstrating that our method enhances fairness without compromising detection efficacy.
A.3.3 Distributions of Gender and Skin tone in FF+ and CelebDF Datasets
We present the distributions for gender and skin tone sensitive attributes within the FF++ and CelebDF datasots in Fig. In the FF++ dataset, noticcable imbalance is evident in the distribution of skin tones compared to gender. Notably, skin tone 8 (Black) is significantly underrepresented, whereas skin tone 3 (White) is the most prevalent. The resulting imbalance factor defined as the ratio of the highest to the scarcest catcgory-is calculated be 8.6. In contrast, the gender distribution exhibits a more modest. imbalance, with a malc-to-fenae ratio of 1.8. These findings suggest that biases associated with skin tone are likely to be severe than those related to gender. A same pattern is observed for CelebDF dataset.
17

---

Published in Transactions on Machine Learning Research (10/2025)
Table DPR and DPD results CelebDF. Best results each column are in bold and second best results are underlined.
Bias Metrics
Methods
Backbones
Gender
Skin Tone
Intersection
DPD 1
DPR
DPD
DPR 1
DPD
DPR 1
EffiiientNNe--Be B3
0.39
0.89
0.28
0.84
0.44
0.86
RECCE
0.36
0.77
0.39
0.67
0.45
0.62
Baseline
EfficientNNet-H
0.38
0.72
0.36
0.83
0.43
0.62
MASDT
0.37
0.63
0.44
0.78
0.42
0.62
AltFreezing
0.38
0.87
0.31
0.83
0.34
0.85
Average
0.38
0.78
0.36
0.79
0.42
0.71
EffiicientNet-
0.39
0.90
0.28
0.85
0.54
0.87
RECCE
0.31
0.78
0.40
0.68
0.45
0.63
DROx²
EfficienNNet-DD
0.35
0.73
0.36
0.84
0.43
0.57
Ilashmmoo al. 2018
MASDT
0.88
0.34
0.79
0.32
AltFreezing
0.18
0.88
0.35
0.84
0.34
0.86
Average
0.28
0.83
0.35
0.80
0.42
0.71
EfficientNee-BS
0.40
0.91
0.29
0.87
0.25
RECCE
0.19
0.87
0.41
0.69
0.56
0.66
MMD
EfficientNet-ID B4
0.29
0.75
0.37
0.85
0.24
0.79
Deka & Sutherland
2022
MASDT
0.10
0.90
0.30
0.81
0.22
0.64
AltFreezing
0.15
0.90
0.38
0.85
0.30
0.88
Average
0.23
0.87
0.35
0.81
0.31
0.77
NetccientNet-B3
0.40
0.92
0.28
0.85
0.45
0.88
RECCE
0.79
0.41
0.69
0.56
0.65
DAG-FDD
EfficientNet-BA
0.28
0.73
0.37
0.85
0.64
0.58
Ju al. 2024
MASDT
0.27
0.79
0.32
0.79
0.33
0.64
AltFrezzing
0.89
0.38
0.44
Average
0.29
0.82
0.35
0.80
0.48
0.72
EfficientNet-B33
0.41
0.94
0.30
0.89
0.66
0.91
RECCE
0.24
0.73
0.41
0.62
0.36
0.67
DAW-FDD
EfficicutNet-BB
0.29
0.77
0.38
0.87
0.34
0.60
Ju et al. 2024
MASDT
0.28
0.62
0.27
0.83
0.53
0.66
AltFreezing
0.25
0.92
0.39
0.87
0.25
0.89
Average
0.80
0.35
0.82
0.75
EfficientNet-BB
0.27
0.96
0.29
0.97
0.35
0.93
RECCE
0.20
0.88
0.42
0.63
0.37
0.67
FairAlign (Ours)
EfficiontNet-BD
0.77
0.29
0.20
MASDT
0.08
0.93
0.22
0.89
0.19
0.65
AltFreczing
0.10
0.94
0.39
0.89
0.25
0.89
Average
0.18
0.90
0.32
0.85
0.27
0.77
18

---

Published in Transactions on Machine Learning Research (10/2025)
Table 10: Results on WildDeepfake dataset, Best results in each column are bold.
Metries (%6]
(%)
Methods
Backbones
Gender
Skin
Tone
Intersection
Overall
GyFr
FEPR
FBo
GePR
FFPr
FBO
GyPR
FEPR
Feo
AUC1
FPR.
TPR1
ACC1
RECCE
5.39
14.63
13.54
64.31
20:25
83.21
Baseline
EfliceutNee-B44
3.45
3.45
5.17
14.86
16.14
21.32
17.04
38.71
66.42
67.39
21.19
87.24
89.67
Anerage
4.42
4.42
6.77
12.47
13.66
17.97
15.29
39.15
66.68
65.85
20.72
85.22
86.46
RECCE
5.31
5.31
8.18
15.32
14.12
12.12
12.12
37.12
62.11
65.34
20.33
87.35
88 89
DAG-EDD
NeficientNee-AA
4.32
22.45
24.21
69.98
65.46
22.00
85.33
Ju 2024
Aucrage
4.83
4.83
7.28
15.70
18.28
18.16
15.06
38.93
66.04
65:40
21.21
86.35
89.02
RECCE
4.27
4.27
6.83
9.01
12.22
14.07
10.55
29.16
60.95
67.11
21.78
86.31
86 22
Align
ElficieuNee-B4
1.15
1.15
3.13
12.83
17.11
19.29
15.01
35.78
63.49
69.42
19.11
88.09
90.11
(Ours)
Aserige
2.71
2.71
4.98
10.92
14.06
16.68
12.78
32.47
62.22
68.26
20.44
87.21
Table 11: Results on FF++ dataset with ethnicity as the sensitive attribute.
Backbone
Ethnicity
GypR
FEPR
Fgo
Faig
MASDT
3.73
4.60
12.36
AleFreezing
3.84
4.67
12.69
Baseline
MASDT
10.56
12.99
11.18
AltFreezing
8.11
7.21
21.48
A.3.4 Performance on Ethnicity Sensitive Attribute
We also apply FairAlign on FF++ dataset with ethnicity the sensitive attribute. We use MASDT and AltFrecing backbones. The results are shown in Table D The results clearly demonstrate that Align significantly reduces ethnicily-based bias for both backbones. This confirms our method's effectiveness across all tested sensitive attributes (gender, skin tone, their intersection, and ethnicity).
A.3.5 Impact of on Fairness
In addition to the the impact of À on bias (FFPR. Intersection) in the main paper, here explore the relationship between and the other seven metrics for the CelebDF dataset versus À in Fig. 5 From this figure, we observe that for five out of the seven plots, = 1 results in the least amount of bias, while for Feo for Intersection and GFPR for Skin Tone, = 0.25 and À = 0.75 are marginally better than For consistency, we throughout all the experiments in this work.
19

---

Published in Transactions on Machine Learning Research (10/2025)
Percentage (%)
FF++ Gender
FF++ Skin Tone
CelebDF Gender
CelebDF Skin Tone
50
50
20
25
25
Gender 2
3 4 5 6
8
90
0
4 Skin
6
910
Skin Tone
Gender
Tone
Figure 4: Distribution of genders and skin tones the FF++ and CelebDF datasets.
Feo (Skin Tone)
26
Fão (Gender))
FEO (Intersection)
50
-00
19
1.00
(a) FEO Skin Tone
(b) FEO Gender
(c) FEO Intersection
13.0
7.5
Fron (Skin Tone)
G (Skin Tone)
Opnn (Gender)
1.0
12.0
7.0
0.5
11.0
6.5
6.0
10.5
0.4
5.5
0.25
0.50
0.75
100
0.25
3.73
1.00
0.25
0.50
0.25
1.00
(d) EpPR Skin Tone
(e) GppR Skin Tone
(f) GpPR Gender
Gypa (Intersaction)
0.25
0.30
0.15
180
(g) Gyrn Intersoction
Figure 5: Effect of tuning the À hyperparameter on bias metrics for AltFreezing backbone trained CelehDF dataset.
20


Reviews:
[{"review_id": "U47ZptozAi", "reviewer": "Reviewer_hug7", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n- Survey of Fairness research is well conducted, and the fairness evaluation measure is used. \n- The performance of FairAlign is higher than that of the state-of-the-art method on fairness deep fake detection, DAW=FDD (Ju et al, 2024).  \n- Fairness accuracy -trade-off concerning skin tone is firstly improved in the research area of deepfake detection. \n- The algorithm of face skin tone categorization is demonstrated, and code is provided.  \n\nWeaknesses \n\n-  This paper applies existing methods for fairness in deep fake detection. Therefore, the novelty is limited to the aspect of machine learning methodologies. \nCKB is an existing method. The authors only apply this method for improving Group Fairness in deepfake detection. \nThough the authors introduce fairness-accuracy trade-off evaluation in deepfake detection, the introduced methods, Fairea and Harmonic Mean, are existing methods, which were used in different fairness problems.\n\n-  It is not clear what kernel function is used. In addition, $z$ seems to be a label of a sensitive attribute. How is it treated? \n\n-  P1. (1) of the second paragraph, the authors claim that ``the embeddings generated by these detectors contribute to retain information related to sensitive attributes, “but this fact is not supported. \n\n-  Evaluation of other sensitive attributes, such as race and nationality, is lacking.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "1.\tThis paper proposes FairAlign, a new loss term for promoting group fairness for deepfake detection. This method is the loss function that uses the Conditional Kernel Bure (CKB) metric proposed in the paper (Lu&Ren, 2021). This method improves the group fairness of the state-of-the-art on the datasets FF++, CelebDF, and WildDeepfake. \n2.\tThis paper also shows that FairAlign improves skin-tone fairness. The skin tones are classified by the MST scale (Heldreth et al, 2023b). This paper enhances fairness based on the intersection of gender and skin tone in the context of deep-fake detection for the first time. \n3.\tTwo unified metrics are used for the first time in the realm of fair deep-fake detection. Fairea (Hort et al, 2021) and Harmonic Mean (Lesota et al, 2022: Li et al, 2023)are used to combine both fairness and accuracy into unified indices.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Clarify kernel function and compare with linear kernel to show the effectiveness of kernel space. \n- Conduct feature embedding visualization, e.g., t-SNE, to support the claim of P1.(1). \n- Conduct experiments on other sensitive attributes. \n- P.4, the second paragraph of FairAlign, replace “Let’s” with “Let us”. \n- P.5 Eq.(6), the third-fifth rows should be written in one row. \n- (Heldreh 2023a) and (Heldresh 2023b) are the same article. They should be merged. \n- P6, DPD, and DPR should be moved to the Appendix since they are reported only in the Appendix.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "3QM180nxhi", "reviewer": "Reviewer_bZWE", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n- The paper focuses on an important and contemporary research problem.\n- The motivation and the differences from the existing work are presented clearly.\n- In the fairness literature, MMD variants have been used for many applications. The paper employs a novel distance metric for conditional distribution alignment (Conditional Kernel Bures (CKB) metric), which is empirically shown to outperform MMD for deepfake detection. \n\n**Weaknesses**\n- In general, the paper uses existing strategies/metrics for a specific application (deepfake detection). Thus, its contributions are limited.\n- Experimental settings can be improved (please see requested changes).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a novel bias mitigation objective for deepfake detection. Furthermore, it develops a rigorous bias evaluation scheme for skin tone and examines fairness/utility trade-off in the deepfake detection realm.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- It would be interesting to examine the impact of FairAlign for other sensitive attributes and the biases related to them, as existing works showed that reducing one kind of bias can exacerbate the others. For example, how the use of FairAlign for skin tone impact the gender-related biases? \n\n- For the sensitivity analysis related to $\\lambda$ (Figure 1), the utility measures should also be presented.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "JAMvylHGUg", "reviewer": "Reviewer_cYx4", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "On the **positive side**, the paper presents technical innovation, mathematical modeling, and broad evaluation (across multiple datasets and in comparison with recent, strong prior works).\n\nOn the **weakness side**:\n1.  **Motivation – Skin Tone**\n    - The paper emphasizes that it is one of  the first to address bias with respect to **skin tone** in deepfake detection. However, it fails to justify why skin tone is a relevant attribute for evaluating fairness in the context of ethical AI models.\n    - In my view, **skin tone is of limited relevance**. \"Ethnicity\" is more directly tied to social biases, but skin tone is not necessarily indicative of ethnicity. *Image recorded skin tone* results from a non-linear combination of factors including ethnicity, tanning, image exposure, and white balance. Addressing bias related to exposure or white balance contributes marginally to the broader goal of ethical AI.\n    - Furthermore, the paper quantizes skin tone into a 10-value discrete set (inferred from Figure 2 in the Appendix), but does not justify why these 10 categories are meaningful or ethically relevant. I argue that ranges 2 and 3, for instance, contain significant overlap in ethnicity, with differences likely due to exposure and white balance variations.\n    - Additionally, skin tone is inherently a *continuous variable*, and the paper does not explain the need for quantization. The mathematical framework could support a conditional probability density function (PDF) without requiring discretization.\n\n2.   **Method Clarity**\n    The mathematical derivation of FairAlign lacks clarity and contains questionable assumptions:\n      -  From the formalism, it appears that z represents the attribute. The paper restricts this to discrete categorical variables (e.g., gender: z ∈ {0,1}; skin tone: z ∈ {0,...,9}). Meanwhile, x represents the embedding—a continuous, high-dimensional representation of the input image. Given these differences, applying the same kernel function over z and x is unjustified.\n      -  The method defines kernel functions $k_X$ and $k_Z$​ on the embedding and attribute spaces, respectively. Initially (\" \" Let’s define kernel functions $k_X$ and $k_Z$ on the space of embeddings X and Z, respectively\".\"), this suggests a mistake (treating z as an embedding). Later derivations, however, use kernels to project attributes into another space while still aiming for matching conditional PDFs. This raises a conceptual question: What is the meaning of fairness computed as differences in accuracy conditioned on a non-linear projection?\n       - The paper does not specify what kind of kernel functions are used (e.g., RBF?), which is a critical detail.\n        - The notation $\\phi(x)=k_X(x,\\cdot)$ is problematic. Since $k_X(x,\\cdot)$ implies an inner product from a specific point $x_b$ to the key point, $x$​, using $\\phi_x(x_b)$ would be more appropriate. Without specifying $x_b$​, $\\phi(x)$ is not well-defined. Later notations with superscripts {a/b} are clearer, but then the introduction of both K and Φ/Ψ becomes redundant—they represent the same objects.\n        - Equation (2) introduces four layers of abstraction and compact notation with minimal explanation. Subsequent notation builds on this, and to me, the left-hand side and right-hand side of the equation seem to have incompatible dimensions.\n       -  Equation (7) is intuitive, but Equation (6) is not, due to unclear usage of kernels over categorical attributes and inconsistent notation.\n\n3. **Evaluation Ambiguity**\n    - A major concern is that results attributed to Ju et al. (2024) in this paper cannot be verified in the original work.\n    -  While the paper introduces skin tone as a new attribute, evaluations with respect to gender and detection metrics should be reproducible.\n    -  Ju et al. use an Xception backbone, while this paper uses different architectures. More worryingly, the results reported for Ju et al. and for the proposed method are significantly worse than those in the original paper.\n    -  Later tables report results for EfficientNet-B3, and one would expect similar performance in gender and detection metrics. However, this paper shows noticeably worse outcomes.\n    - The use of \"Average\" across selected models lacks justification. The selection of these particular models and the omission of the Xception backbone highlight inconsistencies and weaken the experimental design.\n\n4.  **Supplementary Material**\n   -  Although the main paper is limited to 12 pages, additional results can and have been included in the supplementary material.\n   -  The paper fails to show visual examples of individuals categorized under different skin tone bins. Such examples are important to justify whether this attribute is meaningful and consistently annotated: each quanta should be easy to be linked to a ethnicity such that \"comparable results\" across values make sense \n   -  It is also necessary to show multiple images within each skin tone bin to demonstrate variability and confirm that skin tone is a relevant factor in judging whether a deepfake detection model is biased.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper addresses the theme of fairness in deepfake detection. It proposes FairAlign, a method based on Maximum Mean Discrepancy (MMD) to quantify differences between the distributions of embeddings conditioned on different categories. This approach isolates bias and encourages the model to reduce it. The method is evaluated using the FF++, Celeb-DF, and WildDeepfake datasets.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Argue why skin tone in this quantized form is relevant for ethical AI (deepfake detection).\n\n2. Explain the mathematical model in a clear and unambiguous way.\n\n3. Explain why the results reported in Ju et al. are not replicable here.\n\n4. Visual examples of skin tone ranges and prediction decisions, with detailed explanation, should be added to the supplementary material.", "broader_impact_concerns": "No concerns. The paper addresses a theme focused on alleviating various ethical biases in AI models.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)