Do You Remember? Overcoming Catastrophic Forgetting for Fake Audio Detection
Authors: XiaoHui Zhang
Jiangyan Yi
Chenglong Wang
Chu Yuan Zhang
Jianhua Tao
Venue: Submitted to ICLR 2023 (2023)
Keywords: fake audio detection
regularized adaptive weight modification
catastrophic forgetting
continual learning

Abstract: Current fake audio detection algorithms achieve promising performances on most datasets. However, their performance may be significantly degraded when dealing with audio of a different dataset. The orthogonal weight modification to overcome catastrophic forgetting does not consider the similarity of some audio, including fake audio obtained by the same algorithm and genuine audio, on different datasets. To overcome this limitation, we propose a continual learning algorithm for fake audio detection to overcome catastrophic forgetting, called Regularized Adaptive Weight Modification (RAWM). Specifically, when fine-tuning a detection network, our approach adaptively computes the direction of weight modification according to the ratio of genuine utterances and fake utterances. The adaptive modification direction ensures the network can detect fake audio on the new dataset while preserving its knowledge of previous model, thus mitigating catastrophic forgetting. In addition, orthogonal weight modification of fake audios in the new dataset will skew the distribution of inferences on audio in the previous dataset with similar acoustic characteristics, so we introduce a regularization constraint to force the network to remember this distribution. We evaluate our approach across multiple datasets and obtain a significant performance improvement on cross-dataset experiments.

Under review as a conference paper at ICLR 2023
Do YOU REMEMBER? OVERCOMING CATASTROPHIC FORGETTING FOR FAKE AUDIO DETECTION
Anonymous authors Paper under double-blind review
ABSTRACT
Current fake audio detection algorithms achieve promising performances on most datasets. However, their performance may be significantly degraded when dealing with audio of a different dataset. The orthogonal weight modification to over- come catastrophic forgetting does not consider the similarity of some audio, in- cluding fake audio obtained by the same algorithm and genuine audio, on different datasets. To overcome this limitation, we propose continual learning algorithm for fake audio detection to overcome catastrophic forgetting, called Regularized Adaptive Weight Modification (RAWM). Specifically, when fine-tuning a detec- tion network, our approach adaptively computes the direction of weight modifica- tion according to the ratio of genuine utterances and fake utterances. The adap- tive modification direction ensures the network can detect fake audio on the new dataset while preserving its knowledge of previous model, thus mitigating catas- trophic forgetting. In addition, orthogonal weight modification of fake audios the new dataset will skew the distribution of inferences on audio the previ- ous dataset with similar acoustic characteristics, so we introduce a regularization constraint to force the network remember this distribution. We evaluate our ap- proach across multiple datasets and obtain a significant performance improvement on drtssetttaeet experiments.
INTRODUCTION
Currently, fake audio detection has attracted increasing attention since the organization of series of challenges, such as the VSosp challenge (Wu et al. [215: Kinnunen et al.) [017;; Todisco et al.)
2019 Yamagishi et al. 2021) and the Audio Deep Synthesis Detection challenge (ADD) (Yi et al, 2022 In these competitions, deep neural networks have achieved great success. Currently, large- scale pre-trained models have gradually been applied to fake audio detection and achieved state- of-the-art results several public fake audio detection datasets (Tak et al.) 2022 Martin-Doñar &
Álvarez 2022 Lv et al. 2022 Wang & Yamagishi 2021). Although fake audio detection achieves promising performance, it may be significantly degraded when dealing with audio of another dataset.
The diversity of audio proposes significant challenge fake audio detection across datasets (Zhang
ct al. 2021b
Some approaches have been proposed to improve detection performance across datasets. Monteiro let al. (2020) proposed an ensemble learning method to improve the detection ability of the model for unseen audio. WWang al.) (2020) designed a dual-adversarial domain adaptive network to learn more generalized Teatures for different datasets. Both methods require some audio from the old dataset, but in some practical situations, it almost impossible to obtain them. For instance, pre- trained model proposed by company has been released to the public. It is unfeasible for the public to fine-tune using the data belonging to the original company. /Zhang al.) (202Tb) proposed data augmentation method to extract more robust features for detection across datasets, which is only suitable for the datasets with similar feature distribution. Ma et al. (2021) proposed the first continual learning method for fake audio detection, called Detecting Fake Without Forgetting (DFWF) inspired by Learning without Forgetting (LwF) (Li & Hoiem| 2017). The DFWF improves the detection performance by fine-tuning by introducing regularization. Although the above methods are evaluated as viable options, there are still some insufficient places, like the acquisition of previous data in the first two and deteriorating
the new dataset and overcomes catastrophic forgetting

---

Under review as a conference paper at ICLR 2023
Old
New
Modifictiio OWNM Projector introdaed
Agg
R
R
P
Q
OWM
SGD
Orzure
(a)
(h)
Schematic of SGD, OWM, and RAWM. (a). With RAWM, the optimization process searches for configurations that lead to great performance on both old (blue area) and new (green area) datasets. successful optimized configuration Oraum stops inside the overlapping subspace. However, the configuration agd obtained by SGD is optimized without considering forgetting, and the configuration fown obtained by orthogonal weight modification can not reach the overlapping region. (b), the RAWM adaptively modifies weight direction by introducing projector that is or- thogonal to the projector P proposed by OWM.
Figure 1:
learning performance in the DFWE. This paper, however, aims to overcome catastrophic forgetting while exerting positive influence on acquiring new knowledge without any previous samples.
As for fake audio detection, we have observed that most datasets are under clean conditions. Re- garding these datasets, the genuine audio has more similar feature distribution than the fake audio. Specifically, the variance of the feature distribution of genuine audio is smaller than that fake collected under noisy conditions (Müller jet al. 2022), which makes a great difference in their feature distributions of genuine audio (Ma al. 2022j. In this regard, if we modify the model weights the orthogonal weight modifica-
audio (Ma al.) 2021). D. A few datasets, however,
tion (OWM) method (Zeng et al. (019) which introduces a new weight direction orthogonal to all previous data, most genuine audio can not be trained efficiently. The reason is that new data is supposed by the OWM to damage learned knowledge for its different feature distribution but it is unreasonable for fake audio detection. It more efficient for most genuine audio to be trained with the same direction modification because of their similar feature distributions. To address these is- sues, we propose a continual learning approach, named Regularized Adaptive Weight Modification (RAWM). Because genuine audio has more similar feature distribution, it is reasonable to modify model weights the same direction as the old one. Specifically, if the proportion fake audio is larger, the modified direction is closer to the orthogonal projector of the subspace spanned by all previous input; the proportion vious input subspace. However, when the feature distributions of old and new genuine audio quite different, the effect of the above method is not obvious. We address this issue by introducing regularization constraint. This constraint forces the model to remember the feature distribution without requiring prior knowledge. In addition, compared with the spperience-reply-baaee in continuous learning, RAWM does not require previous data, which makes this method suitable in most situations. Finally, the optimization process of RAWM is compared with that of the Stochastic Gradient Descent search (SGD) and OWM in Figure Ta
genuine audio is larger, the modification closer to the pre-
method
We propose regularized adaptive weight modification algorithm to overcome catastrophic forgetting for fake audio detection. There are two essential modules in our method: adaptive weight modification (AWM) and regularization. The former AWM is proposed for contin- ual learning in most situations where genuine audio has similar feature distribution and the latter regularization is introduced to ease the problem that genuine audio may have different feature distri- bution in a few cases. The experimental results show that our proposed method outperforms several continual learning methods in acquiring new knowledge and overcoming forgetting, including Elas- tic Weight Consolidation (EWC) (Kirkpatrick et al. 2017), LwF, OWM, and DFWF. The code will be publicly available in the foreseeable future.
Contributions:
2

---

Under review as a conference paper at ICLR 2023
2 RELATED WORK
In continual learning, overcoming catastrophic forgetting methods can be divided into the following categories. The regularization methods perform regularization on the objection function or regulate important weights that are essential for previous tasks (Kinnunen et al. 2017; Zenke et al. 2017b; Aljundi et al. 2018 2019 Mallya Lazebnik 2018 Serra al. 2018). The dynamic architecture methods reserve their previous knowledge by introducing additional layers or nodes and grow model architecture (Rusu et al. 2016 Schwarz et al., 2018); (Yoon et al. (2017). The memory-based
methods remember their previous data prevent gradient updates from damage
their learned
knowledge, (Lopez-Paz & Ranzato 2017; Castro et al. 2018; Wu et al., 2019; Lee et al. 2019).
The natural gradient descent methods approximate the Fisher information matrix EWC using the generalized
method to fast gradient descent (T'eran et al. 2018 Chen et al.
2019
Although the mainstream continual learning methods, such as the EWC, LwF and OWM, have achieved great success in many fields including image classification (Zeng et al. 2019 Kirkpatrick
al. 2017), object detection (Perez-Rua al. 2020), semantic segmentation (Cermelli al. 2020),
lifelong language learning (de Masson D Autume et al.] 2019) and sentence representation (Liu
al. 2019). However, the approximation of regularization methods will produce error accumulation
in continual learning (Zenke et al. [2017a] Huszár 2017, Ma et al. 2021). In contrast, our proposed method only needs the current inputs and some hyperparameters of the last task, which leads to better performance our method achieves than others in error accumulation. Compared with the DFWF, we relax its regularized constraint and introduce direction modification.
3 BACKGROUND
3.1 ORTHOGONAL WEIGHT MODIFICATION
The OWM algorithm overcomes catastrophic forgetting by modifying the direction of weights on the new task. The modified direction P. which is a square matrix, is orthogonal to the subspace spanned by all inputs of the previous task. The orthogonal projector is constructed by an iterative method similar to the RLS algorithm (Shah et al. 1992). which hardly requires any previous samples.
We consider a feed-forward network consisting of L layers, indexed by | 0, 1. L with the same activation function g(-). The X (i,j) E IR' represents the output of the Ith layer in response to the mean of the ith batch inputs on jth dataset, and the xi( is the transpose matrix of the xi (i,j). The modified direction P can be calculated as:
P(i.)) (i 1,j) (( (i..)) (1(1-1)) 1,9)
P. P(i-L,,)
(t.j,) a + (1.) P(i 1.)
where a is hyperparameter decaying with the number of tasks. This iterative algorithm only needs the current inputs and orthogonal projector for the last task, thus avoiding loading data from the previous task.
3.2 LEARNING WITHOUT FORGETTING
The LwF algorithm is inspired by the idea of model distillation, where old knowledge is viewed as a penalty term to regulate the new model representation similar to the old. Specifically, the model trained on old datasets is replicated into two models with the same parameters. The two models are named teacher and student models in the LwF, In process of training on new datasets, the parameters of the teacher model are frozen produce its features as "soft labels". The student model is trained by the loss function as:
Ltw do Lotayay 8o) + (yn, Fn)
where Åo is ratio coefficient representing the importance of learned knowledge; yo is the "soft label" produced by the teacher model and ya is the ground truth of new data; Both ÿ. and yn are the softmax output of the student model. Both Lold and Lnew encourages predictions ÿ. to be consistent with the ground truth yn and the last regulates the output probabilities y. be close to the recorded output yo from the teacher model.
cross-entropy loss. The first
3

---

Under review as a conference paper at ICLR 2023
4 PROPOSED METHOD
As for fake audio detection, we have observed some distinctive features in this regard. On most fake audio detection datasets, under the same acoustic conditions, feature distributions of genuine audio are relatively more concentrated and unified than the fake, which means the feature distribution of genuine audio has smaller variance than that of fake audio (Ma et al.) 2021 Yan et al. (022). Apart from that, there are also few datasets whose genuine audio has quite different feature distributions from others (Ma al. 2022 Müller et al. 2022). For instance, genuine audio collected from noisy conditions may extremely skew their acoustic feature distribution. Although it seems not norm in this field, we still are supposed to pay attention to these datasets since it was common in reality.
Based on this inference, we propose a continual learning method, named Regularized Adaptive Weight Modification (RAWM), to overcome catastrophic forgetting for fake audio detection. There are two essential parts in our method: adaptive direction modification (AWM) and regularization. The AWM, which is described in Sec. 4.1 is proposed for most situations where the feature distribu- tions of genuine audio are very similar. By introducing an extra projector, which is square matrix orthogonal to the projector proposed by the OWM, our method could adaptively modify weight di- rection forcing it closer to the previous imputs subspace. As for those genuine audio collected from noisy conditions, it is detrimental for learned knowledge to modify weight according to the rule we mentioned above, because their feature distribution is distinct from others. To address this issue, we introduce a regularization term to force the new distribution of inference to be similar to the old one, which is described in Sec. 4.2 Apart from that, we also present how our method regulates models by modifying weight direction under the restriction of regularization in Sec. 4.3 and show the process of our algorithm in Algorithm |
On top of that, our method does not require any prior knowledge of both old and new datasets and replay of previous samples.
4.1 ADAPTIVE WEIGHT MODIFICATION
We start by introducing an adaptive modification of weight direction according to the ratio of genuine and fake audio in batch data, which is essential for sequence training on multi-datasets. We first consider a feed-forward network like that described in Sec. S.c.3.1| Then, we introduce a square matrix Q as a projector that is orthogonal to the P proposed by the OWM algorithm. This orthogonal projector can be written as Eq
Q= P= P)-
(3)
where the projector P, which is orthogonal to the subspace spanned by all previous inputs, can be calculated as Eq [ and I is an identity matrix. The construction of the orthogonal projector Q is mathematically sound (Haykin) 2002. Ben-Israel & Greville 2003 Bengio & LeCun 2007). To
verify the modification direction according to the ratio of genuine audio and fake audio, we introduce the 3 defined as:
N +1
(4)
in which Ng and N₂ represent the number of genuine and fake audios in a batch, respectively. By adding one to both the numerator and denominator, 3 can be calculated when all the batch audios are genuine. As illustrated in 3 the norm of projector Q is proportional to the ratio 3. Our approach defines the modified direction R of weights as:
R- P=Px+mQ2
(5)
P
P
Q QN= QN I-
(6)
|PT
where is a constant to constrain the norm of projector Q to prevent gradient explosion or gradient vanishing in the backward process; PN and QN are identity matrices normalized by P and Q, respectively. Normalization is to prevent the case that the change of has little effect on the modified direction because of the large norm gap between P and Q. In the baok-proe (BP) process, the direction of network weights is modified as:
Wi(i. = Wi +7..) (i.
when j =
-
(i,j) when j > 1
where Wi().) R represents the connection weights between the lth layer and the (1+1)
layer; y represents the learning rate of this network; A AW.P (i.j) represents the standard BP gradi-
ent; R. represents the modification projector in our method. In Eq 7 we can easily observe that
4

---

Py Under review as a conference paper at ICLR 2023
Algorithm Regularized Adaptive Weight Modification
Require: Training data from different datasets, "Y (learning rate), m (constant hyperparameter), Treg (constant
hyperparameter).
1: for every dataset do
repeat
3:
for every batch i do repeat
4:
5:
if j 1 then
6:
Wi(t, = ()
AWW gradient by standard BP method
8:
k(i.)) = - P(0--- (1
9:
(1 = (i(i-1,j) -(. P1(1-1,))
b xx output of the mean of inputs
3 Ng +1
10:
Ng. Nr: genuine, fake audio number
N
Q B[I P(P" P)
P
12:
PI
Q
13:
QN [IP((PT)) P
14:
R PN + mQN
(1)
16:
(n(1)
17:
A AW.P BP
lreg -V(F-(1) log (1)
18:
Wi.t,) Wi(i-1,5) + (1.)) + (AWee (i.5)
19:
end if
20:
until loss plateaus
21:
end for
22:
end
modify weight direction adaptively by multiplying the BP gradient AW,P"(i)) with our projector R whose direction is varied according the ratio of genuine and fake audio.
4.2 REGULARIZATION
There are few datasets where genuine audio is collected from noisy conditions. In this case, is unreasonable to use the above method directly. As for these utterances, we introduce an extra regularization forcing the model to remember the previous inference distribution.
We first replicate the pre- praitrained model into two models with the same parameters, one is the teacher model and the other one is the student model. The parameter the teacher model is frozen in the process of training on the new dataset and the parameter of the student model is fine-tuned. Like the operation in the LwF, we view the softmax output y. from the teacher model as "soft labels" and use the loss function to slash the distinction between the "soft labels" y. and the softmax output yn of the student model, thus forcing the student model to remember the learned knowledge. The loss function, which is modified cross-entropy loss, can be written
Lieg (F..) ) - - log
(8)
1/Treg
1/Treg
yo
0o
In
99)
Σy. 1/Treu
Σ yn
1/Trez
where Trey is a constant hyperparameter. The yo. yn are softmax outputs of teacher and student models, respectively; The ý is a normalized form of the y: The 9 and y are one item of y and y, respectively. The weight modification of this regularization AWBP can be written Eq 10 frep
AW free
V
(10)
4.3 REGULARIZED ADAPTIVE WEIGHT MODIFICATION
5

---

Under review as a conference paper at ICLR 2023
In brief, our continual learning method RAWM is proposed for fake audio detection by modifying weight direction according the ratio of genuine and fake audio in a batch and eases the problem that a few utterances corrupted by noise may interfere with the direction modification by introducing a regularized restriction. Considering a continual learning situation, the BP process of regularized adaptive weight modification can be written as Eq
RP (i.j) - ((L.)--W(LL----(-(-(------------ (1.1)
W((i.) (((.))=
when
(11)
( (ee (i.5)) when j > 1
Compared with the Eq our method introduces regularization constraint to the adaptive weight modification. The importance of the regularization depends on the hyperparameter which is a coefficient measuring the attention degree of the old task.
EXPERIMENTS
5.1 DATASETS
We conduct our experiments four fake audio datasets, including the ASVspoof2019LA (S), ASVspoof2015 (T1), VCC2020 (T2), and In-the-Wild (Ta). The models are firstly trained us- ing the training set of the ASVspoof2019 and are fine-tuned the training sets of the other three datasets. All of the experiments are evaluated using two or four evaluation sets in these datasets.
ASVspoof 2019 LA Dataset (Toiisco et al., 2019) is the sub-challenge dataset (30 males and 37 females) containing three subsets: training. development, and evaluation. The training set and de- velopment share the same attack including four TTS and two VC algorithms. The bonafide audio attacks.
collected from the VCTK corpus (Veaux et al., 2017). The evaluation set contains totally different
ASV ASVspoof2015 dataset eWu al. 2013) is an open-source standard dataset of genuine and synthetic speech the ASosoof2001 challenge. The genuine speech was recorded from 106 speakers (45 males and 61 females) with no significant channel or background noise effects. The spoofing speech is generated using a variety of speech synthesis and voice conversion algorithms.
VCC2020 dataset (Zhao et al., 2020) is collected from Voice Conversion Challenge 2020. This dataset contains two subsets: a set of genuine audio provided by organizers and a set of fake audio provided by participating teams. Different from the previous three datasets, VCC2020 is multilin- gual fake audio dataset, including English, Finnish, German and Mandarin.
In-the-Ve Wild dataset (uüller et al., 2022) contains set of deep fake audio (and corresponding real audio) of 58 politicians and other public figures collected from publicly available sources, such as social networks and video streaming platforms. In total, 20.8 hours of genuine audio and 17.2 hours of fake audio were collected. On average, each speaker had 23 minutes of genuine audio and 18 minutes of fake audio.
We divide the genuine and fake audios of the VCC2020 dataset into four subsets. A quarter is used to build the evaluation set, quarter to build the development set, and the to be used the training set. The in-the-Wild dataset is divided in the same way as the VCC2020. The ASVspoof2015 is the most similar to the ASVspoof20199. conversion algorithms. The audio of the In-the Will dataset is collected from the real world and the audio of the VCC2020 dataset is multilingual. The detailed statisties of the datasets are presented in Table The Equal Error Rate (EER), which is widely used for fake audio detection and speaker verification, is applied to evaluate the experimental performance.
for their audios are collected from the same datasets or
5.2 EXPERIMENTAL SETUP
Fake audio detection Model: We use the pre-trained model Wav2vec 2.0 (Baevski et al. 2020) as the feature extractor and the self-attention convolutional neural network (S-CNN) as the classifier. The parameters of Wav2vee 2.0 is loaded from the pre-train model XLSR-53 (Conneau et al. 2020). The classifier S-CNN contains three ID-Convolution layers, one self-attention layer, and two full connection layers, according to the forward process. The input dimension of the first convolution layer is 256 and the hidden dimension of all convolution layers is 80. The kerne size and stride

---

Under review as a conference paper at ICLR 2023
Table I: Statistics of experimental datasets.
ASVSpoo12019
ASVSpoo12015
VCC2020
In-the-Wii
Dataset
#Real
#Fake
#Real
#Fake
#Real
#Fakee
#Real
#Fake
Train
2,580
22,800
3,750
12,625
1,330
3,060
9,431
5,908
Dev
2,548
22,296
3,497
49,875
665
1,530
4,715
2,954
Eval
7,355
63,882
9,404
184,000
665
1,530
4,717
2,954
Table The EER(%) of our baseline on multiple evaluation sets.
Model
S
T₁
T2
Ts
Baseline
0.258
24.532
46.503
91.473
set to 5 and I, respectively. The hidden dimension of all full connection layers is 80 and the output dimension of the last is 2.
We fine-tune the model weights including the pre-trained model XLSR-53 and the classifier S-CNN. All of the parameters are trained by the Adam optimizer with a batch size of and learning rate of 0.0001. The constant and Treg in RAWM are set to 0.1 and 2, respectively. The a is initialized to 0.00001 for convolution layers, 0.0001 for the self-attention layer, and 0.1 for full connection layers. The norm in normalization of projector P and Q is the L norm. In addition, we present the results of training all datasets (Tain-on-All that considered to be the lower bound to all continual learning methods
Training Details:
mentioned (Parisi et al. 2019). All results are (re)produced by us and averaged over 7 runs with standard deviations.
5.3 BASELINE
We first train our model on the training set of the ASVspoof2019LA dataset. Table shows the detection performance of our baseline on multiple evaluation sets which is very close to the state- of-the-art result in the same dataset (Nautsch et al. (02T). Although the model achieves promis- ing performance on the ASVspoof20T9LA, its detection accuracy degrades significantly on other datasets. addition, our baseline achieves the lowest cross-datasets EER on the ASVspoo/201 dataset among three unseen datasets, which verifies that the fake audio generated by the same al- gorithms has similar feature distribution, while the feature distribution of fake audio generated by different algorithms is quite different. Apart from that, the results with different training steps are presented in Table 7 in the appendix.
5.4 THE EFFECTIVENESS OF THE 1) FOR OUR METHOD
Sequence training between two datasets: We start by performing some experiments evaluate the effectiveness of 1 in R, which represents the attention degree to learned knowledge. In Table 3 we can easily observe that the RAWM achieves great performance on both old and new datasets, especially in the experiment on S - T,. By comparing the results of three cross-datasets, we observe that when the new and old datasets have similar feature distribution (Table 3a), there is an improvement in the performance of both acquiring new knowledge and overcoming forgetting with the increasing of (n < 1); When the feature distribution of the new and previous datasets differ- ent (Table 3b Table 3c it is the model when y = 0.50 that achieves the best result, which shows that regularization is also of benefit to performance on both learning and overcoming forgetting.
Sequence training on four datasets:
We also present the results on multiple evaluation sets about different y in Table 4a It can observed that our method slashes performance degradation when training across datasets. The RAWM achieves the lowest EER among the results when 7 0.50, which demonstrates that the same attention degree to both old and new datasets is the best choice for learning and overcoming forgetting. In addition, the results of dataset T3 show that smaller is more beneficial for models to acquire knowledge. Apart from that, the results of S, T, and T show that the model with larger y more effective in overcoming forgetting.

---

Under review as a conference paper at ICLR 2023
Table The EER(%) on evaluation sets of our method with different 7) All experiments are trained using the training set in order to S - TK and are evaluated using the evaluation set on S and Tk
(b)
(c)
?
Ti
5
T2
?)
S
Baseline
0.258
24.532
Baseline
0.258
46.503
Baseline
0.258
91.473
0.00
1.643
0.256
0.00
1.413
3.845
0.00
4.126
1.457
0.20
1.424
0.431
0.20
1.334
4.288
0.20
3.490
1.848
0.25
1.175
0.311
0.25
1.275
3.994
0.25
2.975
1.593
0.50
0.878
0.257
0.50
1.237
3.721
0.50
2.038
1.425
0.75
0.666
0.247
0.75
1.262
4.571
0.75
2.482
2,271
1.00
3.123
0.343
1.00
4.234
4.566
2.453
2.598
Table 4: The EER(%) on four evaluation sets. All experiments are trained using training set in order to S + T1 + T2 + T3 and are evaluated using evaluation sets.
(a) The EER(%) of the RAWM with different 1j.
(b) The EER(%) () of our method compared with others.
S
Ti
T2
Ts
Method
T₁
T2
Tg
Baseline
0.258
24.532
46.503
91.473
Baseline
0.258
24.532
16.503
91-173
Train-on-All
1.324
0.561
3.579
0.00
1.845
1.127
3,916
1.410
0.20
1.724
1.003
4.120
1.367
Fine
5.674
2.543
EWC
5.589
3.444
4.510
2.129
0,25
1.699
0.945
4.017
1.529
OWM
4.083
2.167
4.480
2.472
0.50
1.508
0.641
3.850
LwF
2.714
1.621
3.875
2.325
0.75
1.636
0.873
3.975
2.454
DFWF
3.476
3.735
7.345
3.114
1.00
2.714
1.621
3,875
2.325
RAWM(Ours)
1.508
0.641
3.850
1.163
5.5
ABLATION STUDIES FOR OUR METHOD
Sequence training between two datasets: In this section, we compare our proposed method with adaptive weight modification without regularization (-REG) and orthogonal weight modification without regularization (-AWM), Table 5 presents their EER on three evaluation sets. We observe that RAWM achieves similar EER to REG on the new dataset, both them are superior sig- nificantly to AAWM. which shows that the adaptive weight modification has significant positive impact on acquiring knowledge, while regularization impacts little. As for overcoming forgetting, when the feature distribution of the and old datasets is similar (Table 5a the EER of the REG on the old datasets is much lower than that of the AWW and higher than that of the RAWM,, which shows that the adaptive weight modification and regularization can significantly reduce the forget- ting in this case. When the languages of the new and old datasets are different (TableBb), the EER of RAWM the old datasets similar that of the REG and much lower than that of the AWM, which also proves that the adaptive weight modification has significant positive impact on over- coming forgetting. When the feature distribution of the new and old datasets quite different (Table 5c). the EER of the REEG is similar to that of the - AWM and much higher than that of the RAWM, which shows that in this case, regularization is of great benefit to overcoming forgetting, while the effect of adaptive weight modification is not obvious.
Sequence training on four datasets:
this section, we present the results of the ablation study on four evaluation sets in Table 5d We observe that the EER of -REG to AAWM degrades more obviously than that of RAWM to - REG on all evaluation sets, which indicates that adaptive weight obvious benefit in learning and overcoming forgetting than regularization
modification has for sequence training on multiple datasets.
5.6 COMPARISON OF OUR METHOD WITH OTHER METHODS
Sequence training between two datasets: We compare our method with several methods in Table 6 The EWC, LwI, and OWM as three mainstream continual learning methods achieve great success in many fields. The DFWF is the first continual learning method to overcome forgetting for fake audio detection. The results demonstrate that fine-tuning without modification (Fine-tune) forgets previous knowledge obviously. The forgetting of RAWM is one-tenth that of Fine-tune on Table ba and the EER the dataset of RAWM is also half that of Fine-tune. We also observe that the
8

---

Under review as a conference paper at ICLR 2023
Table 5: The EER(%) on evaluation sets of the ablation studies. (a), (b) and (c) are trained using the training set in order to S - Tk and are evaluated using the evaluation set on S and Tr: (d) is trained in order T1 + T2 + T3 and are evaluated using evaluation sets.
(a)
(b)
Method
S
T₁
Method
T2
RAWM
0.666
0.247
RAWM
1.237
3.721
REG
1.643
0.256
REG
1.413
3.845
AWM
2.448
0.500
AWM
3.086
5.432
Method
To
Method
S
T₁
T2
Ts
RAWM
2.038
1.425
RAWM
1.508
0.641
3.850
1.163
REG
4.126
1.457
-REG
1.845
1.127
3.916
1.410
AWM
4.857
2.663
-AWM
4.083
2.167
4.480
2.472
Table 6: The EER(%) of our method compared with various methods. All experiments are trained using the training set in order to S - Tk and are evaluated using the evaluation set on S and Tk
(a)
(b)
Method
Method
T2
Method
S
Baseline
0.258
24.532
Baseline
0.258
46.503
Bascline
0,258
91-173
Train-on-All
0.406
0.201
Train-on-All
0.965
Train-on-All
1,740
0.800
Fine-tune
7.324
0.510
Fine-tune
8.755
5.047
Fine-tune
20.976
2.679
EWC
3.494
EWC
5.039
OWM
2.448
0.500
OWM
3.086
5.432
OWM
4.857
2.683
DEWF
1.849
0.689
DFWF
1.874
7.355
DFWF
5.324
3.275
(ArM(Ours)
Ours)Ours
RAwM(Ours)
2.038
Fine-tune, EWC and OWM achieve similar performance in three experiments and the performance of LwF outperforms theirs the new dataset. In addition, the overcoming forgetting of LwF is also superior to that of the three methods when the feature distributions of new and old datasets are quite different (Table (c). The DFWF is more effective in overcoming forgetting than the above methods, but its performance on the new dataset inferior to others. Compared with others, our method achieves lower EER both old and new datasets of all experiments, which demonstrates that both overcoming forgetting and learning could definitely benefit from our method when training across datasets, regardless of whether the datasets have similar feature distributions (Table 6al Table Gb) or same languages (Table (c).
Sequence training on four datasets: Finally, We compare our method with several methods for sequence training on four datasets in Table 4b The results show that the accuracy of DFWF is inferior to others on the last three datasets. which proves that the DFWF is not effective as others in acquiring knowledge. Apart from that, most methods achieve lower EERs than fine- fine-tuning, and the best result for overcoming forgetting and learning is our proposed method, which indicates that the RAWM is superior to others for sequence training on both two and multiple datasets.
CONCLUSION
In this work, we propose continual learning algorithm for fake audio detection, called RAWM. that could adaptively modify the weight direction in process of training on new datasets. Our method overcomes catastrophic forgetting by updating weights according to the adaptive modified direc- tion under the restriction of regularization. The experimental results demonstrate that our method performs better than four continual learning methods in learning and overcoming forgetting. The results also prove that our method is effective in overcoming catastrophic forgetting in scenarios of sequence training on both two and multiple datasets. In addition, our method does not require pre- vious data; thus it can be applied to most classification networks. Despite results, we have yet to study the impact of different acoustic conditions and noise interference on forgetting, and exploring these questions will form the focus of our future studies.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Rahaf Aljundi, Francesca Babiloni. Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In Proceedings of the European Confer- ence on Computer Vision (ECCV), pp. 139-154, 2018.
Rahaf Aljundi, Klaas Kelchtermans, and Tinne Tuytelaars. Task-free continual learning. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11254 11263, 2019.
Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. wav2vec 2.0: A framework for self-supervised learning of speech representations. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Floring Balcan, and Hsuan-Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020. URL hhttps://proceediiiig neurips. cc//paper/222/haat. 92dleeeebccds1ffa32227 TTA.EE//..22211100000100--------
Adi Ben-Israel and Thomas NE Greville. Generalized inverses: theory and applications, volume 15. Springer Science Business Media, 2003.
Yoshua Bengio and Yann LeCun. Scaling learning algorithms towards AI. In Large Scale Kernel Machines. MIT Press, 2007.
Francisco M Castro, Manuel Marfn-Jiménez, Nicolás Guil, Cordelia Schmid, and Karteek Alahari. End-to-end incremental learning. In Proceedings of the European conference on computer vision (ECCV), pp. 233-248, 2018.
Fabio Cermelli, Massimiliano Mancini, Samuel Rota Bulo, Elisa Ricci, and Barhara Caputo. Mod- cling the background for incremental learning in semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9233-9242, 2020.
Yu Chen, Tom Diethe, and Neil Lawrence. Facilitating bayesian continual learning by natural gra- dients and stein gradients. arXiv preprint VrXiv:1904.10644 2019.
Alexis Conneau, Alexei Baevski, Ronan Collobert, Abdelrahman Mohamed, and Michael Auli. Unsupervised cross-lingual representation learning for speech recognition. arXiv preprint 1cXi1:2006.13999 2020.
Cyprien de Masson D' A'tumm, Sebastian Ruder, Lingpeng Kong, and Dani Yogatama. Episodic memory in lifelong language learning. Advances in Neural Information Processing Systems, 32, 2019.
Simon Haykin. Adaptive filter theory. Pearson Education India, 2002.
Ferenc Huszár. On quadratic penalties in elastic weight consolidation. CoRR, 1b/11//2...3384 2017. URL
abs 03847
Tomi Kinnunen, Md. Sahidullah, Héctor Delgado, Massimiliano Todisco, Nicholas W. D. Evans, Junichi Yamagishi, and Kong-Aik Lee. The asvspoof 2017 challenge: Assessing the limits of replay spoofing attack detection. In Francisco Lacerda (ed.), Interspeech 2017, 18th Anmual Conference of the International Speech Communication Association, Stockholm, Sweden, August 20-24, 2017, pp. 2-6. ISCA, 2017. URL http:/ www. isca-sspechheeeeceecceceeccccece weseeech.org//. arrg/rrchiel. Interspeech
htm
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska BaraaBBaaa et al. Overcom- ing catastrophic forgetting neural networks. Proceedings of the national academy of sciences, 2017.
Kibok Lee, Kimin Lee, Jinwoo Shin, and Honglak Lee. Overcoming catastrophic forgetting with un- labeled data in the wild. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 312-321, 2019.
10

---

Under review as a conference paper at ICLR 2023
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, 40(12):2935- 29477 2017.
Tianlin Liu, Lyle Ungar, and Joao Sedoc. Continual learning for sentence representations using conceptors. arXiv preprint arXiv::904.09111 2019.
David Lopez-Paz and Mare' Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30, 2017.
Zhiqiang Lv, Shanshan Zhang, Kai Tang, and Pengfei Hu. Fake audio detection based on unsuper- vised pretraining models. In ICASSP 2022-2022 IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pp. 9231-9235. IEEE, 2022.
Haoxin Ma, Jiangyan Yi, Jianhua Tao, Ye Bai, Zhengkun Tian, and Chenglong Wang. Continual learning for fake audio detection. arXiv preprint arXiv:2 I04.222,0..7266, 2021.
Haoxin Ma, Jiangyan Yi, Chenglong Wang, Xinrui Yan, Jianhua Tao, Tao Wang, Shiming Wang, Le Xu, and Ruibo Fu. FAD: A chinese dataset for fake audio detection. CoRR, abs/2207, 12308 2022. doi: 10.48550/axxvv22207.1 12308.
URL
2207 12308
Arun Mallya and Svetlana Lazebnik. Packnet: Adding multiple tasks a single network by iterative pruning. In Proceedings of the IEEE conference Computer Vision and Pattern Recognition, pp. 7765-7773, 2018.
Juan M Martín-Doñas and Aitor Álvarez. The vicomtech audio deepfake detection system based on wav2vec2 for the 2022 add challenge. ICASSP 2022-2022 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 9241-9245. IEEE, 2022.
João Monteiro, Md. Jahangir Alam, and Tiago H. Falk. An ensemble based approach for gener- alized detection of spoofing attacks to automatic speaker recognizers. 2020 IEEE Interna- tional Conference on Acoustics, Speech and Signal Processing, ICASSP 2020, Barcelona, Spain, May 4-8, 2020, pp. 6599-6609 6603. IEEE, 2020. doi: 10.1 00.1000CAASP447776220000000
URL
https: doi
//te::///ooo.o//0// org/10 110111191160 ICASSP 40776 2020 9054558
Nicolas M Müller, Pavel Czempin, Franziska Dieckmann, Adam Froghyar, and Konstantin Böttinger. Does audio deepfake detection generalize? arXiv preprint arXiv:2203. 16263, 2022.
Andreas Nautsch, Xin Wang, Nicholas W. D. Evans, Tomi H. Kinnunen, Ville Vestman, Mas- similiano Todisco, Héctor Delgado, Md. Sahidullah, Junichi Yamagishi, and Kong Aik Lee. Asvspoof 2019: Spoofing countermeasures for the detection of synthesized, converted and re- played speech. IEEE Trans. Biom. Behav, Identity Sci., 3(2): 265, 2021. doi: 10.1 3059479
1B15542001.3059479. URLttttss: doi. www.:.to.......... www.:///...110//1111 TBIOM. 2 2021 2021.205947.7
German Ignacio Parisi, Ronald Kemker, Jose L. Part, Christopher Kanan, and Stefan Wermter. Con- tinual lifelong learning with neural networks: A review. Newral Nenworks, 113:54-71, 2019. doi: 10.101 wet.://..eenee.211.....0. 01.012
URL /ttt::://6d1.oogg/1.01011.. htw.:/............eeeee......... 2019.
Perez-Rua, Xiatian Zhu, Timothy M Hospedales, and Tao Xiang. Incremental few-shot object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13846-13855 2020.
Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint ToXiv16606.066771 2016.
Jonathan Schwarz, Wojeiech Czarnecki, Jelena Luketina, Agnieszka Grabska- Barwinska, Yee Whye Teh, Razvan Pascanu, and Raia Hadsell. Progress & compress: A scalable framework for contin- ual learning. In International Conference Machine Learning, pp. 4528-4533 PMLR, 2018.
Joan Serra, Didac Suris. Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard attention to the task. In International Conference on Machine Learning, pp. 4548 4557. PMLR, 2018
11

---

Under review as a conference paper at ICLR 2023
Samir Shah, Francesco Palmieri, and Michael Datum. Optimal filtering algorithms for fast learning in feedforward neural networks. Neural networks, 5(5):779-7877 787, 1992.
Mayank Sharma. Multi-lingual multi-task speech emotion recognition using wav2vec 2.0. In ICASSP 2022 2022 IEEE International Conference on Accuutics, Speech and Signal Processing (ICASSP), pp. 6907-6911, 2022.
doi: 10.1199NCAAS44492220027777777
Hemlata Tak, Massimiliano Todisco, Xin Wang, Jee-weon Jung, Junichi Yamagishi, and Nicholas Evans. Automatic speaker verification spoofing and deepfake detection using wav2vee 2.0 and data augmentation. arXiv preprint arXiv:2202.122333 2022.
Massimiliano Todisco, Xin Wang, Ville Vestman, Md Sahidullah, Héctor Delgado, Andreas Nautsch, Junichi Yamagishi, Nicholas Evans, Tomi Kinnunen, and Kong Aik Lee. Asvspoof 2019 Future horizons in spoofed and fake audio detection. arXiv preprint arXiv: 1904.05441, 2019.
Hanna Tseran, Mohammad Emtiyaz Khan, Tatsuya Harada, and Thang D Bui. Natural variational continual learning. In Continual Learning Workshop@ NeurIPS, volume 2, 2018.
Christophe Veaux, Junichi Yamagishi, Kirsten MacDonald, al. Cstr vetk corpus: English multi- speaker corpus for estr voice cloning toolkit. University of Edinburgh. The Centre for Speech Technology Research (CSTR), 2017.
Hongji Wang, Heinrich Dinkel, Shuai Wang, Yanmin Qian. and Kai Yu. Dual-adversarial domain adaptation for generalized replay attack detection. In Helen Meng, Bo Xu, and Thomas Fang Zheng (eds.), Interspeech 2020, 21st Annual Conference of the International Speech Commu- nication Association, Virtual Event, Shanghai, China, 25-29 October 2020, pp. 1086-1090. ISCA, 2020. doi: 10.21437/mmerrspecch.200 2020 1255. URL https: /ww. org. 214377 Interspeech. 2020- 1255
Xin Wang and Junichi Yamagishi. Investigating self-supervised front ends for speech spoofing countermeasures. arXiv preprint arX1v:2111.07725, 2021.
Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 374-382, 2019.
Zhizheng Wu, Tomi Kinnunen, Nicholas Evans, Junichi Yamagishi, Cemal Hanilçi, Md Sahidullah, and Aleksandr Sizov. Asvspoof 2015: the first automatic speaker verification spoofing and coun- termeasures challenge. In Sixteenth annual conference of the international speech communication association, 2015.
Juniehi Yamagishi, Xin Wang, Massimiliano Todisco, Md Sahidullah, Jose Patino, Andreas Nautsch, Xuechen Liu. Kong Aik Lee, Tomi Kinnunen, Nicholas Evans, et al. Asvspoof 2021: accelerating progress spoofed and deepfake speech detection. arXiv preprint arXiv:2109.00533 2021.
Xinrui Yan, Jiangyan Yi, Jianhua Tao, Chenglong Wang, Haoxin Ma, Tao Wang, Shiming Wang, and Ruibo Fu. An initial investigation for detecting vocoder fingerprints of fake audio. CoRR, doi: 11..5550aaXxv.22088096....
abs/2208. 2022.
URL https: //doi. org
48550/arXiv 22686
Jangyan Yi, Ruibo Fu, Jianhua Tao, Shuai Nie, Haoxin Ma. Chenglong Wang, Tao Wang, Zhengkun Tian, Ye Bai, Cunhang Fan, et al. Add 2022: the first audio deep synthesis detection challenge. In ICASSP 2022- 2022 IEEE International Conference on Acousties, Speech and Signal Processing (ICASSP), pp. 9216-9226 IEEE, 2022.
Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically expandable networks. arXiv preprint arXiv: 1:Xivv/708.015444 2017.
Guanxiong Zeng, Yang Chen, Bo Cui, and Shan Yu. Continual learning of context- -eeeex--dependent pro- cessing in neural networks. Nature Machine Intelligence, (8):364-372. 2019.
12

---

Under review as a conference paper at ICLR 2023
Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence.
In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference Machine Learning. ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pp. 3987-3995. PMLR, 2017a. URL http://
proceedings. ml www.mml.r........mmmmmm.mm..
www.prrsssvvvoom v0/ zenke 7a html
Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In International Conference on Machine Learning, pp. 3987-3995, PMLR, 2017b.
You Zhang, Fei Jiang, and Zhiyao Duan. One-class learning towards synthetic voice spoofing de- tection. IEEE Signal Processing Letters, 28:937-941, 2021a.
You Zhang, Ge Zhu, Fei Jiang, and Zhiyao Duan. An empirical study on channel effects for synthetic voice spoofing countermeasure systems. arXiv preprint arXiv:2/04.013200 2021b.
Yi Zhao, Wen-Chin Huang, Xiaohai Tian, Junichi Yamagishi, Rohan Kumar Das, Tomi Kinnunen, Zhenhua Ling, and Tomoki Toda. Voice conversion challenge 2020: Intra-lingual semi-paralle, and cross-lingual voice conversion. arXiv preprint arXiv:2008. /2527, 2020.
13

---

Under review as a conference paper at ICLR 2023
A APPENDIX
A.1 RESULTS OF OUR BASELINE
Table 7: The EER(%) on multiple evaluation sets. Model-1 to Model-6 are the models trained using the ASVspoof2019L. training set with increasing training steps.
Evaluation Sets
Model
8
T1
T2
Tg
Model-1
3.751
6.316
7.670
75. 198
Model-2
2.975
8.517
10.000
78.477
Model-3
1.794
9.988
26.165
85.436
Model-4
0.258
24.532
46.503
91.473
Model-5
0.259
25.698
44.741
91.824
Model-6
0.262
27.872
49.726
92.113
A.2 RESULTS OF CONTINUAL LEARNING ON FEW-SHOT
We also present some results of our pre-trained model continually learned on a few samples. In experiments, only 100 samples randomly selected from new datasets were used for fine- tuning or continual learning. Table shows the results of few-shot continual learning from the ASVspoof2019 dataset the ASV spoof2011 dataset. Most models trained on the new dataset within five steps. From the results, we can observe that our method RAWM also achieves the best performance on both old and new datasets. By comparing the results in Table 6a and Table 8, we can easily find that reducing the number of samples has only a little damage to our method.
A.3 CONTINUAL LEARNING FOR SPEECH EMOTION RECOGNITION
Our method can be easily used in other continual learning fields such as image recognition, object detection, and emotion recognition. In the body of the paper, we only demonstrate the application of fake audio detection, which our current research field.
The key of the RAWM method for other applications is the designation of B. The most intuitive designation is
Ni + No + N N& +1 8 N +k++1-+ N,k 2) + N m) +1
(12)
Where the N₂ represents the number of samples in class i. Samples of class 1 class & are those that have similar feature distributions on old and new datasets (That means, the variance of their feature distributions is small different datasets). For fake audio detection, the class in the numerator represents genuine audio where is 1. Apart from that, samples of class + 1 class k + m are those that have great difference feature distributions. For fake audio detection, the class in the denominator represents fake audio.
For speech emotion recognition, the previous result shows that neutral emotion achieved the highest recognition accuracy across thirteen emotion datasets (Sharma 2022). So we infer that neutral speech has more similar feature distribution than that of happy, sad, and angry, thus the ratio hyperparameter 3 of our method can be written as follows.
Nneutral
3 N Nhappy N Nangry + Nead +
(13)
Based on this inference, we also conduct some experiments for speech emotion recognition. We choose four emotional classes, including neutral, happy, angry, and sad. The results have been added to Table It could be easily observed that our method still achieves the highest accuracy on both datasets compared with other continual learning methods.
14

---

Under review as a conference paper at ICLR 2023
Table 8: The (ER(%) of few-shot experiments. All experiments are first trained using the training set of Aoospof2009 and then trained on the subset of the training set of ASVspoof2015 The subset only includes 100 samples randomly chosen from the training set of ASVspoof2015. All experiments are evaluated using the evaluation set on ASVspoof2019 and ASVspoo22155.
Method
ASVspoof2019
ASVspoof2015
Baseline
0.258
24.532
Fine-tune
7.951
0.617
EWC
2.972
0.619
OWM
2.683
0.617
LWF
3.198
0.542
DFWF
1.975
0.733
(AuM(Ours)
0.923
0.312
Table 9: The Acc(%) of various continual learning methods for 4-classes speech emotion recog-
nition. All experiments are trained using the training in order MSP Podcast IEMOCAP and are evaluated using the evaluation set on MSP Podcast and IEMOCAP
Method
MSP
Podcast
IEMOCAP
Baseline
54.416
30.043
Fine-tune
24.094
50.379
OWM
32.267
50.162
LwF
38.800
44.034
RAWM(Ours)
41.995
54.229
15


Reviews:
[{"review_id": "iYWlaz9Cd5", "reviewer": "Reviewer_d2it", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors address the problem of \"catastrophic forgetting\" in the context of fake audio detection. When a network trained on one dataset (D1) and is fine tuned on another dataset (D2), the fine-tuned network loses its original performance on the D1. This is referred to as catastrophic forgetting for the considered task of fake audio detection. There are several approaches addressing this problem. One of the recent approaches is using the orthogonal weight modification (OWM) technique. This is based on the viewpoint that modifying the weights in a direction orthogonal to the Weights subspace corresponding to D1 would ensure that the modified network's performance on D1 would not deteriorate. Thus, the idea while fine-tuning on D2 is to modify the weight update equation by modifying the weights in a direction orthogonal to that corresponding to weights subspace representing D1. Such a weight modification has a regularization effect on the network to retain its performance on D1. This however presents a significant drawback: The Performance of this regularized network on D2 suffers compared to without regularization. \n\nIn this paper the authors address the above two issues in OWM by proposing Regularized Adaptive Weight Modification (RAWM) where the above two problems are addressed respectively by a nice observation that blindly updating the weights in an orthogonal direction to the subspace corresponding to D1 might be a bad idea as there could be several genuine audios in D1 which are similar to D2 and for those samples - the regular update might not harm network performance on D2 while improving its performance on D2. This is accounted in the weight update by modifying the direction of update to be based on the number of genuine vs fake examples in each batch and using this to adapt the orthogonal weight update. In the degenrate case of all fake examples this proposed approach would boil down to the OWM method.   \n\nOverall the paper is well written with clear problem definition, experimental results and useful comparisons. ", "summary_of_the_review": "Overall the proposed solution of adapting the weight modification by accounting for the genuine and fake samples in each batch seems interesting and shows promising results. The methodology is clearly explained with detailed insightful evaluation. ", "strength_and_weaknesses": "Strengths:\n1. Paper is well written and the solution proposed addresses the problem quite well. \n2. The experimental validation is sufficiently elaborate and shows good improvement over the baselines considered. \n\nWeakness:\n1. There are a few parts of the paper which are not particularly well written and makes it a difficult read. For instance:\n a) In Section 4.2 the notations and equations 9, 10 are not in sync.\n b) Symbols and notations are not clearly explained on several occasions. \n c) Out of nowhere in Section 5.3, authors start talking about a \"Model-4\n2. How does the proposed method compare to training on all datasets ?\n3. It is not clear why the assumption that genuine audio across different datasets is more likely to be similar than fake audios from different datasets is necessary ? Can the authors not think about exploiting fake audio similarity as well in their technique ?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "YZUmPft_Rkz", "reviewer": "Reviewer_VFqH", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a continual learning algorithm, called Regularized Adaptive Weight Modification (RAWM) to overcome catastrophic forgetting for fake audio detection. RAWM is based on the previously published orthogonal weight modification (OWM). Because OWM does not consider the similarity of some audio, including fake audio obtained by the same algorithm and real audio, on different datasets. To solve this limitation, adaptive modification direction and a regularization constraint are proposed. Experimental results show a good performance improvement compared to baselines.", "summary_of_the_review": "This paper is based on the previously published OWM paper, and two modifications (adaptive modification direction and a regularization constraint) are proposed. As pointed out earlier, the reason to apply Q in adaptive modification direction is unclear, and the novelty of using a teacher model for regularization constraint is somewhat limited.", "strength_and_weaknesses": "Strength:\nThe proposed continual learning algorithm is designed for the application of fake audio detection. The authors observe some problems when using conventional continual learning algorithms and propose two methods (adaptive modification direction and a regularization constraint) to overcome them. \n\nWeaknesses:\nTo avoid using training data from the previous tasks, the proposed regularization constraint is based on a teacher model, which is not a very novel idea. On the other hand, the way to apply an adaptive modification direction is not very clear to me. In page 4, the authors claim that “In the training process on the new dataset, they should be trained without any modification.” However, the authors propose a new projector Q which is orthogonal to the projector P from OWM. My question is: why not just use the original gradient from SGD, ∆W^BP in this case? \nOther comments:\n1. Is P a square matrix? Otherwise, how to compute Q using e.q. (4)?\n\n2. In the original paper of ‘OWM’ (Continual Learning of Context-dependent Processing in Neural Networks), the authors claim that ” Recursive Least Square (RLS) algorithm which can be used to train feedforward and recurrent neural networks to achieve fast convergence’ However in this paper, the proposed fake audio detection model is based on Wav2vec (which includes convolutional encoder) and a CNN classifier. I’m not sure how OWM performs on convolutional layers. My concern is that I think OWM assumes the calculation between input and model weights is through matrix multiplication instead of convolutional operation.\n\n3. In Fig. 1. of the original paper of ‘OWM’, they claim that OWM can reach the position inside the overlapping subspace between two tasks. However, in Fig. 1. of your paper, you mention that OWM can not reach the region of dataset 2. Please explain why there is such a difference.\n\n4. In Table 6, why even Fine-tune cannot obtain the best performance on the new task compared to other methods?\n\n5. It would be great to also report the ‘experience-replay-based’ method in the experiment.\n\nTypo:\n1. Page 1, fae audio detection -> fake audio detection \n\n2. Page 3, yo and yo are the old and new ground truth -> yo and yn are the old and new ground truth\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "O0V3eRIJez", "reviewer": "Reviewer_a37q", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper considers the problem of continual learning in the context of fake audio detection. One of the main challenges of continual learning is catastrophic forgetting. The paper proposes a new algorithm called Regularized Adaptive Weight Modification (RAWM). The motivation is genuine audios are more similar than fake audios in different datasets. The proposed approach can adaptively modify the direction of weights according to the ratio of genuine audio and fake audio of each batch in the process of fine-tuning. The paper evaluated the proposed approach across multiple fake audio detection datasets and showed promising results in the continual learning setting. \n", "summary_of_the_review": "The paper proposes a novel method (RAWM) for continual learning in the context of fake audio detection. Experiments demonstrate that the proposed method obtains strong performance compared to the prior methods. The paper can be made stronger if the proposed method can be extended to more applications or to general recognition tasks.\n\n", "strength_and_weaknesses": "Strength \n\nThe paper proposes a novel method called Regularized Adaptive Weight Modification (RAWM) to address the problem of catastrophic forgetting in the context of fake audio detection. The proposed method builds on prior work orthogonal weight modification (OWM).\n\nThe approach is well-motivated (genuine audios are more similar than fake audios in different datasets. The proposed method obtains a better tradeoff between learning from a new dataset while not forgetting the past knowledge.)\n\nThe paper provides a satisfactory literature review.\n\nThe paper evaluates the proposed approach across multiple fake audio detection datasets and shows promising results in the continual learning setting. \n\nWeaknesses\n\nThe proposed method seems to be general for continual learning. However, experiments only demonstrate its value for the task of fake audio detection. Can authors comment on whether the proposed method can benefit any other tasks other than fake audio detection? The paper can be made stronger if the proposed method can be extended to more applications or to general recognition tasks (e.g., image/video/audio recognition).\n\nDoes the proposed method also work for multiclass classification tasks? If so, what are the necessary changes?\n\nPresentation of section 3 can be improved due to lack of context. It will be useful to include a paragraph to describe the problem setting (e.g., notation of model layers, notation for each dataset, notation for gradient etc.). A lot of the context is only introduced in section 4 (“We consider a feed-forward network consisting of L + 1 layers..”).\n\n\nMinor comments:\n\nIn the second equation of EQ1,  the numerator is P_l(i-1,j) \\bar{X}(l-1). Should \\bar{X}(l-1) be indexed with i,j ?\n\nSection 3.2, “y_o and y_o are the old and new ground truth” (typos?).\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "EbzvcSEfuA", "reviewer": "Reviewer_jUDi", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a new method for continual learning which leverages structure found in the particular task of fake audio detection. Specifically, the proposed method augments continual learning algorithms by leveraging structural differences found in fake audio detection between two individual distribution shifts latent in the source and target datasets: source fake -> target fake, and source real -> target real. Compared to other continual learning algorithms, the proposed algorithm leads to large improvements in performance on both source and target datasets.", "summary_of_the_review": "Overall, this paper presents some interesting ideas and compelling results, but the current version struggles w/ (1) motivating the setting of interest (continual learning for fake audio detection) including lack of comparison to methods which can access the source data, and (2) clarity of explanation.", "strength_and_weaknesses": "The primary strength of this paper is that of the results: the proposed algorithm clearly outperforms other comparable continual learning algorithms for standard fake audio detection tasks. The primary weakness is a lack of motivation for the problem formulation explored here. Specifically, this paper explores continual learning for fake audio detection, for which there are two key issues:\n\n(1) I would argue that the biggest practical challenge in fake audio detection is not _adapting_ to new domains w/ many examples (as is explored in continual learning) but rather improving _zero/few-shot generalization_ to new domains. The threat model in fake audio detection should emphasize the persistent development of new methods for generating fake audio. Instead, the exploration of continual learning suggests a perpetual cat-and-mouse game where detection models must be manually adapted to new domains.\n\n(2) The motivation for the _continual_ learning setup (where source data is inaccessible) is unclear. The introduction says in one sentence that “in some practical situations, it is almost impossible to obtain the old data”, but does not suggest any particular situations where that would be the case. Moreover, all of the experiments _contrive_ such situations (since the source dataset is always accessible), and there are no comparisons to “oracle” performance w/ access to source data.\n\nAdditionally, this paper struggles w/ clarity (see next section).", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject