Title: DataStealing: Steal Data from Diffusion Models in Federated Learning with Multiple Trojans
Venue: NeurIPS 2024
Accepted: yes
Keywords: Federated Learning

Diffusion Models

DataStealing

Multiple Trojans

Adaptive Scale
Abstract: Federated Learning (FL) is commonly used to collaboratively train models with privacy preservation. In this paper, we found out that the popular diffusion models have introduced a new vulnerability to FL, which brings serious privacy threats. Despite stringent data management measures, attackers can steal massive private data from local clients through multiple Trojans, which control generative behaviors with multiple triggers. We refer to the new task as ${\bf\textit{DataStealing}}$ and demonstrate that the attacker can achieve the purpose based on our proposed Combinatorial Triggers (ComboTs) in a vanilla FL system. However, advanced distance-based FL defenses are still effective in filtering the malicious update according to the distances between each local update. Hence, we propose an Adaptive Scale Critical Parameters (AdaSCP) attack to circumvent the defenses and seamlessly incorporate malicious updates into the global model. Specifically, AdaSCP evaluates the importance of parameters with the gradients in dominant timesteps of the diffusion model. Subsequently, it adaptively seeks the optimal scale factor and magnifies critical parameter updates before uploading to the server. As a result, the malicious update becomes similar to the benign update, making it difficult for distance-based defenses to identify. Extensive experiments reveal the risk of leaking thousands of images in training diffusion models with FL. Moreover, these experiments demonstrate the effectiveness of AdaSCP in defeating advanced distance-based defenses. We hope this work will attract more attention from the FL community to the critical privacy security issues of Diffusion Models. Code: https://github.com/yuangan/DataStealing.

DataStealing: Steal Data from Diffusion Models in Federated Learning with Multiple Trojans

Abstract

Federated Learning (FL) is commonly used to collaboratively train models with privacy preservation. In this paper, we found out that the popular diffusion models have introduced a new vulnerability to FL, which brings serious privacy threats. Despite stringent data management measures, attackers can steal massive private data from local clients through multiple Trojans, which control generative behaviors with multiple triggers. We refer to the new task as DataStealing and demonstrate that the attacker can achieve the purpose based on our proposed Combinatorial Triggers (ComboTs) in a vanilla FL system. However, advanced distance-based FL defenses are still effective in filtering the malicious update according to the distances between each local update. Hence, we propose an Adaptive Scale Critical Parameters (AdaSCP) attack to circumvent the defenses and seamlessly incorporate malicious updates into the global model. Specifically, AdaSCP evaluates the importance of parameters with the gradients in dominant timesteps of the diffusion model. Subsequently, it adaptively seeks the optimal scale factor and magnifies critical parameter updates before uploading to the server. As a result, the malicious update becomes similar to the benign update, making it difficult for distance-based defenses to identify. Extensive experiments reveal the risk of leaking thousands of images in training diffusion models with FL. Moreover, these experiments demonstrate the effectiveness of AdaSCP in defeating advanced distance-based defenses. We hope this work will attract more attention from the FL community to the critical privacy security issues of Diffusion Models. Code: https://github.com/yuangan/DataStealing.

1 Introduction

Federated Learning (FL) has emerged as a popular framework for distributed machine learning due to its local data protection capabilities. Meanwhile, the issue of privacy data leakage in federated learning has attracted significant attention. Most previous FL works focused on discriminative models, and several studies [19, 61, 68] have demonstrated that FL may leak small amounts of arbitrary local data through gradient inversion. Recently, FL has begun to be utilized to train diffusion models [27, 51, 25], one of the advanced generative methods [22, 48, 45, 58, 11]. In this paper, we found that diffusion models have brought new privacy vulnerabilities to FL. Attackers can steal thousands of specified high-quality local data through diffusion models trained with FL, which exposes more severe privacy security issues compared with discriminative models.

Specifically, to safeguard data privacy, using FL to train diffusion models is considered a definitive choice. According to GDPR [52], strict data privacy management mechanisms should be adopted to prevent attackers from directly accessing or transmitting data from local clients, including banning

ComboTs

Broadcast

global model

Upload

local model

with Defenses

Release

Generated Data

0.0061

0.0020

0.0036

MSE

USB drives, restricting network access, and disabling copy-paste. Nevertheless, these attackers may tamper with the training process of local clients, enabling diffusion models to embed and later extract private images from the released global model, as shown in Fig. 1. For instance, in an organization with strict data management, one staff can use this method to transmit data to the outside. We refer to this task as DataStealing . The recently emerging Trojan attacks on diffusion models [5, 9, 10] make it possible to steal an image with a specific trigger. This inspires us that multiple triggers may lead to more severe harm and steal thousands of private data. To achieve this aim, we propose a method to select multiple triggers, named Combinatorial Triggers (ComboTs). Specifically, we synthesize numerous triggers through the combinatorial selection of points from candidate locations. Our ComboTs fulfill the requirement for extensive data mapping in DataStealing .

Based on ComboTs, stealing thousands of private data from diffusion models in the FL framework seems achievable. However, with the advanced FL defense strategies, injecting multiple backdoors into diffusion models poses greater challenges than introducing a single backdoor. We observe that data poisoning [6, 20, 33] faces difficulties in effectively injecting multiple backdoors into diffusion models under FedAvg [37]. While model poisoning [1] can rapidly inject multiple backdoors by scaling malicious updates, it generally fails to overcome the advanced FL defenses [50, 2, 17, 42, 44, 24]. Although these defenses are designed for classification tasks, distance-based defense methods [2, 17, 44, 24] remain effective in rejecting the malicious update according to the distances between each local update, thereby intensifying the challenge of implanting multiple Trojans into diffusion models.

To address the above challenge, we propose an Adaptive Scale Critical Parameters attack (AdaSCP) to effectively circumvent various distance-based defenses [37, 2, 17, 44, 24]. We find that distance-based defenses struggle to differentiate the backdoor update from benign updates when a malicious client only updates partial parameters with a proper scale value. Specifically, we estimate the importance of parameters in the diffusion model in a Taylor expansion framework [39] over dominant timesteps. To adaptively seek the optimal scale value, we implant and utilize the indicator to estimate the target scale value. AdaSCP optimizes the scale factor with a learning rate according to the difference between the target and current scale. Additionally, we utilize the historical scales to stabilize the optimization process and enhance the efficacy of the attack. At last, the attacker trains the critical parameters and uploads the malicious update magnified with the optimized scale value.

Overall, our main contributions are summarized as follows:

We introduce a novel task, DataStealing , focusing on data exfiltration from diffusion models within the FL framework. Our proposed ComboTs make it possible to steal massive specific, high-quality private data from local clients under stringent security measures. To our knowledge, we are the first to pay attention to the data privacy leakage risks in federal learning training diffusion models.

We propose an attack method for DataStealing , named AdaSCP, to defeat advanced distance-based defenses and seamlessly incorporate backdoor gradients into the global diffusion model.

Extensive experiments have been conducted to assess the efficacy of current FL Trojan attacks and defenses in DataStealing task. Moreover, our AdaSCP achieves SOTA performance in stealing thousands of private data from the global diffusion model. Our findings illuminate potential future risks to the security of training diffusion models in FL.

0

(c) AdaSCP

(b) Trojan Diffusion Process

𝐶 𝑝, 𝑞

(a) ComboTs

Private Data

Restore Data

Benign Update

Benign Update

Benign Update

Poisoned Update

Combinatorial Triggers

Target Images

Candidate Positions

Broadcast

global model

Upload

local model

𝑡𝑎𝑟 𝑠

Distance-based

Defenses

Scale:

Server

𝛿𝑖

1

2 Background

Denoising Diffusion Probabilistic Models (DDPMs). DDPMs [22] have garnered considerable interest due to their capacity to generate high-fidelity samples. At the core of DDPMs lies the fundamental principle of modeling the distribution of data through a diffusion process. In this progression, the initial data x 0 undergoes incremental corruption until it eventually converges into a state of complete noise x t with t ( 1 ≤ t ≤ T ) steps.

where ϵ ∼ N (0 , I ) is a Gaussian noise, and ¯ α t = ∏ t i =1 α i is a hyperparameter that gradually increases the noise level in the forward process. Subsequently, a reverse process works as a denoising operation to retrieve the original data from the noise. This process is defined by a neural network, which is trained to predict the noise ϵ at timestep t . Then, the trained model can gradually denoise random noise to photorealistic images.

Diffusion Models in Federated Learning. FL [37] involves a central server and multiple distributed N c clients to train a global model without sharing the data. Clients periodically send local updates to the central server, which combines all updates and sends back the improved global model. This process is repeated until the server completes the training and releases the global model to the users. Analogous to the training of classification models, we train diffusion models independently on each client before being combined on the server. Each client completes the training of the diffusion model autonomously and uploads the trained model and its Exponential Moving Average (EMA) model.

3 DataStealing: Task and Algorithms

3.1 Threat Model and Attack Scenario

Consider the following FL scenario: multiple participants aim to collaboratively train a diffusion model using sensitive, privately-held training data D benign ( e.g. , medical images) securely stored on their respective clients. Utilizing diffusion models, which inherently generate data distinct from the original inputs, ensures privacy protection by averting the exposure of personal data characteristics. A trusted server aggregates the gradients provided by each client, updating the global model and

redistributing to each client. After the training phase concludes, the server makes the global model available to all users.

Attacker's Goal. Attacker partition client data into two subsets: { D benign , D backdoor } . The attacker's goal is to extract a large number of training images, D backdoor , from the compromised client via the released global model, as illustrated in Figure 1. Additionally, the attacker should minimally affect the original model's functionality to preserve the stealth of the backdoor attack.

Attacker's Abilities. Under strict data privacy protection, attackers can not upload or download images from the infiltrated client. To achieve the goal of embedding the target data D backdoor into the global diffusion model, the attacker is permitted to enact arbitrary alterations to the training procedure of the diffusion model and the uploaded gradients of the infiltrated client. To streamline the task and facilitate analysis, we focus on scenarios with a single attacker and exclude those with multiple distributed attackers. Considering the difficulty of training diffusion models and the complexity of our task, we assume that the attacker has prior knowledge of the total number of participating clients.

3.2 Combinatorial Triggers

Conventional backdoor attacks typically use a single trigger, such as a corner patch in the image mask, to generate a target image. However, a fixed trigger is impractical to generate plenty of target images. To tackle this issue, we introduce Combinatorial Triggers (ComboTs) to fulfill the need of mapping multiple target images. Firstly, we choose p candidate points in a grid pattern on the image mask to form the candidate positions. Subsequently, we select q points from the candidate positions in a combinatorial manner. The total number of potential triggers can be determined using the straightforward combination formula: C ( p, q ) . For a 32 × 32 image mask with a pixel spacing of 1, the value of p is 225 when border pixels are excluded. When q is set to 2, the number of ComboTs is 25,200, which fully satisfies our requirements. Furthermore, the ComboTs demonstrate significant scalability: with q set to 3, the potential number of ComboTs can exceed 2 million.

Once ComboTs are obtained, they will be utilized as backdoor triggers for training. As shown in Fig. 2 (a), each trigger δ i corresponds to a target image x 0 ,i . Then we extend the Trojan attack method [5] on the diffusion model from one trigger-target pair to multiple pairs. According to Eq. 1 the forward diffusion process with trigger δ i can be represented as:

where γ is the blend weight. Trojan noise ( γϵ +(1 -γ ) δ i ) is a blended image between Gaussian noise ϵ and trigger δ i . Fig. 2 (b) shows the forward and backward diffusion process with one trigger. The sampling process with Trojan noise has been introduced in the literature [5]. The sampling process is extended from one trigger to multiple triggers according to our ComboTs.

3.3 Adaptive Scale Critical Parameters Attack

To achieve an effective attack, model poison [1] is treated as our baseline, which magnifies the updates with a scale value. As Table 1 shows, model poison (scale is N c ) with ComboTs succeed in stealing thousands of data from the FedAvg framework. However, such a brute-force method is ineffective in advanced FL with defense protocol. Two reasons make the attack invalid: 1) The difficulty of training diffusion models with ComboTs requires scaling the malicious updates, which will be easily detected by distance-based defenses [2, 17, 44, 24]. 2) The training of the diffusion model will collapse with a high scale value due to its high sensitivity to the gradients.

To evade the advanced distance-based defenses, we introduce an Adaptive Scale Critical Parameters attack algorithm (AdaSCP). To alleviate the effect of scaling, AdaSCP estimates and trains critical parameters [14] instead of the whole network (Sec. 3.3.1). As for the second issue, AdaSCP adopts an adaptive scaling policy with an implanted indicator [31] (Sec. 3.3.2 and Sec. 3.3.3). The details of AdaSCP are shown in Algorithm 1 and Fig. 2 (c). Taking broadcast global model G r , local client's datasets { D benign , D backdoor } , indicator magnification factor k and previous attack setting I r -1 , s r -1 as inputs, AdaSCP reads the returned indicator and adaptively adjust the scale value s r to defeat the defenses (Line 1). To avoid excessive parameter modification in one specific indicator, we restore the indicator and use the next candidate indicators I r . Then, if the candidate indicators I r are used up, the attacker will estimate the importance score of w G with benign dataset D benign to acquire the

critical parameter mask M G and then find new candidate indicators (Lines 2-5). After that, AdaSCP trains the critical parameters with D benign and D backdoor and scale the critical updates with s r to acquire the scaled malicious model w r (Line 6). Finally, we implant the new indicator to get the indicator-implanted backdoor model w ∗ r , which will be uploaded to the server (Lines 7 and 8).

3.3.1 Diffusion Importance Estimation

To defeat the advanced defenses with critical parameters in diffusion models, we first adopt the diffusion importance estimation used in the structural pruning task of diffusion model [14]. They claim that numerous noisy and redundant timesteps make minimal contributions to the overall generation in the diffusion process as t approaches T [36, 14]. Hence, our importance scores are estimated with accumulated gradients in dominant timesteps under the Taylor expansion framework [39]. To locate the critical parameters, we sort estimated importance scores and choose a proportion τ to filter important parameters of the diffusion model. The mask M G of critical parameters is then obtained for subsequent training. In the meantime, preparing for finding indicators, we record the absolute value of accumulated gradients | ˆ w G | and the corresponding Hessian matrix | H G | , which represents the changing direction of gradient updates [67]. More details can be found in Appendix C.1.

3.3.2 Find Candidate Indicators

Algorithm 2 illustrates the process of finding candidate indicators. To prevent the magnification of indicators affecting the performance of the diffusion model, we find candidate indicators in the uncritical parameters according to M G (Line 1). Inspired by previous work [31], we consider two necessary conditions to choose the indicator (lines 2 and 3). Firstly, the neuron's gradient update should be smaller than other neurons. Secondly, the neuron's Hessian value, the second derivative of diffusion loss, should be close to zero. This means the change of this neuron contributes less to the performance [29, 63]. Besides, to mitigate the influence of the backdoor dataset and align indicators with other clients, we only use the benign dataset D benign to calculate gradients and curvatures. To improve the efficiency of finding

Algorithm 2 Find Candidate Indicators and Implant New Indicator

Input: Global model G r , accumulated gradients | ˆ w G | , Hessian matrix | H G | , indicator magnification factor k , current scale s r , mask M G

Output: Indicator-implanted model w ∗ r , candidate indicators I r

1: I ′ ←{ I ′ i | M G ( I ′ i ) = False } 2: ˆ I = { ˆ I 1 , ˆ I 2 , ..., ˆ I j } ← arg min ˆ I i ∈ I ′ θ | ˆ w G | , ˆ I i 3: I r = { I 1 , I 2 , ..., I m } ← arg min I i ∈ ˆ I θ | H G | , ˆ I i 4: for each I i ∈ I r do M G ( I i ) ← True 5: w r ← TrainCriticalParam ( ..., M G ) 6: ▷ Implant the first indicator to get w ∗ r 7: θ ′ w r -G r ,I 1 ← k s r · θ w r -G r ,I 1 8: return w ∗ r , I r

candidate indicators, we only compute | H G | in the last several layers of the diffusion model as the whole parameters bring a heavy burden on calculating the second derivative. Moreover, to avoid the computation of | ˆ w G | and | H G | in every round, we select m indicators at one time to form a candidate indicator set I r = { I 1 , I 2 , ..., I m } for subsequent use (Line 3).

To implant the indicator, AdaSCP treats the indicators as critical parameters and magnifies its updates (Lines 4 - 7). After training and scaling the critical parameters, we can get the scaled backdoor model w r . Then AdaSCP chooses the first indicator I 1 in candidate queue I r and re-scale the corresponding

Algorithm 3 Adaptive Scale with Indicator

Input: Indicator magnification factor k , previous attack setting s r -1 , I r -1 , uploaded indicator update θ ′ w r -1 -G r -1 ,I 1 , returned indicator update θ G r -G r -1 ,I 1 , learning setting his acc , his rej , η , d Output: Current scale s r , candidate indicators I r

update θ w r -G r ,I 1 with the magnification factor k and current scale value s r . The rescaled update θ ′ w r -G r ,I 1 will be merged to form the indicator-implanted backdoor model w ∗ r of this round (Line 7).

3.3.3 Adaptive Scale with Indicator

Although the implanted indicator can help us judge whether the malicious model is accepted by the FL server or not, the potential of the indicator has not yet been fully tapped. We observe that although the black-box server can ensure that a malicious client is unaware of its weight in the aggregation process, the returned indicator update implies this information . This inspires us to control the weighted average results by adjusting the scale value s r . The optimal scaling factor enables the attacker to circumvent distance-based defenses and seamlessly incorporate backdoor gradients into the global model, bolstering the efficiency and effectiveness of attacking with multiple Trojans. Hence, we design an adaptive algorithm to optimize the scaling factor s r .

Algorithm 3 illustrates the procedure of our adaptive scale algorithm. According to the global model G r , we can get the returned indicator update θ G r -G r -1 ,I 1 in the ( r -1 )-th round. At first, we need to define the target scale value tar s according to the returned indicator update. Given n c local updates participating in averaging and the weights λ i for each client, the optimal value of tar s should be ∑ n c i =1 λ i /λ adv for poisoning the global updates with the adversary updates. When λ i = 1 , tar s equals the number of selected clients n c , which is the optimal scale value used in model poison [1]. But when the indicator-implanted model is rejected by the server, the scale value should approach zero to defeat the defenses. As shown in Line 1 and 2, we can derive tar s with the uploaded and returned indicator updates. We leave the derivation in Appendix D.

Anatural way is to replace s r with tar s when ' Accepted " by the server. However, this is not practical. The derivation of tar s is based on the assumption that the trained updates at the position of indicator θ w i -G r ,I 1 are similar across all clients due to the very small gradients. This assumption is not always valid, especially when training with non-IID or backdoor datasets. To reduce the impact of deviation caused by training, we treat ( tar s -s s -1 ) as an optimization direction toward the optimal scale value. Hence, we define a learning rate η to optimize the scale value s r . The adaptive scale can defeat some defenses and implant multiple backdoors stealthily. However, some defenses [2] are effective in detecting the scaled malicious updates or diluting them by averaging. This will lead to an unstable training process, where s r fluctuates between 0 and tar s . To stabilize s r and improve the attack effectiveness, we record history scale values in his acc and his rej . They further help determine the optimal value of tar s and s r (Line 3). Additionally, we design a weight decay d for decreasing the learning rate η according to the accepted state in the previous and current rounds. Finally, we restore the indicator value and dequeue the first index I 1 in candidate indicators to prepare for the subsequent operation in the current round (Line 4). More details are provided in Appendix C.3.

4 Experiments

Datasets and Diffusion Models. We conduct our experiments on three widely-used datasets in generation tasks: CIFAR10 ( 32 × 32 ) [30], CelebA ( 64 × 64) ) [34] and LSUN Bedroom ( 256 × 256 ) [62]. We concentrate on Denoising Diffusion Probability Models (DDPMs) [22]. We follow the model replacement attack [1], injecting backdoor data as training approaches convergence. To simulate the attack process efficiently, we train the diffusion models under the FL framework for CIFAR10 and CelebA. Then we use the pre-trained model for subsequent backdoor experiments. The

Dataset

Attacks

Defenses

FedAvg [37]

Krum [2]

Multi-Krum [2]

Foolsgold [17]

RFA [44]

Multi-metrics [24]

Mean

FID

↓

/ MSE

↓

FID

↓

/ MSE

↓

FID

↓

/ MSE

↓

FID

↓

/ MSE

↓

FID

↓

/ MSE

↓

FID

↓

/ MSE

↓

CIFAR10

Data Poison [20]

6.87/0.1226

10.09/0.1480

6.20/0.1427

7.70/

0.1238

6.72/0.1241

7.09/

0.1213

7.45/0.1304

Model Poison [1]

12.86/

0.0069

8.29/0.1454

6.23/0.1426

459.64/0.3124

6.12/

0.1194

70.98/0.1685

94.02/0.1492

PGD Poison [53]

6.86/0.1232

19.98/

0.1239

6.93/

0.1221

7.45/0.1243

6.85/

0.1231

6.78/0.1228

9.14/

0.1232

BC Layer Sub. [69]

5.75/0.1382

132.02/0.1719

6.03/0.1433

6.67/0.1388

5.64/0.1488

6.69/0.1233

27.13/0.1441

AdaSCP (Ours)

12.93/

0.0117

30.68/

0.0861

8.23/

0.1271

24.21/

0.0129

8.22/0.1233

15.04/

0.0328

16.55/

0.0657

CelebA

Data Poison [20]

5.91/0.1304

7.64/0.1520

6.13/0.1506

6.22/0.1441

5.74/0.1212

6.65/

0.0922

6.38/0.1317

Model Poison [1]

16.05/

0.0465

7.95/0.1524

6.16/0.1504

446.81/0.3161

5.49/

0.0858

N/A

96.49/0.1502

∗

PGD Poison [53]

8.16/0.1516

7.01/

0.0462

8.04/0.1435

6.49/0.1636

8.02/0.1263

7.44/0.1362

7.53/0.1279

BC Layer Sub. [69]

12.29/0.1328

76.49/0.0536

15.63/

0.1204

10.40/

0.1417

18.36/0.1159

17.08/0.1177

25.04/

0.1137

AdaSCP (Ours)

7.00/

0.0082

13.66/

0.0367

4.55/

0.1312

7.36/

0.0103

6.20/

0.1029

7.62/

0.0104

7.73/

0.0499

LSUN

Bedroom

Data Poison [20]

23.50/0.0969

12.28/0.2512

25.31/

0.1169

23.47/0.1321

23.45/

0.0947

22.44/

0.0862

21.74/

0.1297

Model Poison [1]

33.20/

0.0723

11.97/0.2557

13.31/0.2539

404.92/0.2529

21.80/

0.0894

174.83/0.3135

110.00/0.2063

PGD Poison [53]

23.49/0.0976

11.95/0.2546

39.93/0.1476

16.31/

0.1282

23.68/0.0959

21.27/0.0966

22.77/0.1368

BC Layer Sub. [69]

10.84/0.1392

45.77/

0.1157

12.29/0.1391

15.41/0.1361

13.90/0.1313

13.05/0.1354

18.54/0.1328

AdaSCP (Ours)

22.30/

0.0544

51.15/

0.1634

25.81/

0.1131

28.50/

0.0554

24.36/0.1162

22.28/

0.0623

29.07/

0.0941

AdaSCP

Data Poison

BC Layer Sub.

Target

N/A

AdaSCP

Data Poison

BC Layer Sub.

Target

FedAvg

Krum

Multi-Krum

Multi-metrics

FedAvg

Krum

Multi-Krum

RFA

Multi-metrics

(b) CelebA

(a) CIFAR10

Figure 3: Qualitative Comparison . The generated image with the lowest MSE in one trigger-target pair is presented under every attack and defense method. More visual results are in Appendix A.10.

pre-training round number is 2,000 for CIFAR10 and 1,200 for CelebA. As for LSUN Bedroom, we use the pre-trained weight from DDPMs [22].

Backdoor Attacks. Due to the considerable cost of training diffusion models, we simulated five local clients and selected one as the adversarial client. Based on the pre-trained model, we fine-tune 300 rounds for CIFAR10 and 150 rounds for CelebA and LSUN Bedroom in every client. We train 10,000 images per round per client for CIFAR-10 and CelebA, and 1,000 images for LSUN Bedroom. The patch size of ComboTs is 3 × 3 for CIFAR10, 5 × 5 for CelebA, and 25 × 25 for LSUN Bedroom. The backdoor and benign datasets are mixed in one batch with a 50 / 50 ratio. For the attack algorithm, we select the common and latest attack methods as our baselines: Data Poison [20], Model Poison [1], PGD Poison [53], and Backdoor-Critical (BC) layer substitution [69]. We set the scale value of model poison as the total number of clients ( N c = 5 ). We train about 80% parameters in our AdaSCP and layers in the BC layer substation for fair comparison. More details can be found in Appendix B.

Defenses. To verify the effectiveness of DataStealing methods under defense strategies, we conduct experiments with various distance-based defense methods: FedAvg [37], Krum [2], Multi-Krum [2], Foolsgold [17], RFA [44], Multi-metrics [24]. We observe that clipping and adding noise, commonly used in the differential privacy method [13, 42, 50], will undermine the training of diffusion models. Therefore, we do not consider these defenses in our experiments.

Evaluation. We leverage two metrics for evaluating the performance of diffusion models and the efficacy of DataStealing: 1) the Frechet Inception Distance (FID) [21]. 2) Mean Square Error (MSE) between the generated backdoor images and the ground truth images. A lower FID score indicates that

the generated images better match the distribution of real images, implying higher quality. A lower MSE means the restored images are more similar to the private images, indicating better performance in DataStealing . To calculate FID, we generate 50,000 images for every model with CIFAR10 and CelebA and 10,000 images for LSUN Bedroom. As for MSE, we synthesize 20 images with different noise inputs for every combinatorial trigger. All images are generated with 50 -timestep DDIM [48].

4.1 DataStealing under Advanced FL Defenses

To evaluate AdaSCP and attack baselines in DataStealing , we conduct experiments on CIFAR10 [30], CelebA [34] and LSUN Bedroom [62]. The target image count is 1,000 for the CIFAR10 dataset, 500 for the CelebA dataset, and 50 for LSUN Bedroom.

Comparisons with Non-IID Datasets. We first consider the challenging non-IID distribution. Both datasets are sampled with Dirichlet distribution of hyperparameter 0 . 7 . As shown in Table 1, our AdaSCP achieves the best attack performance according to MSE. The critical parameters and adaptive scale factor enable AdaSCP to evade the detection of most distance-based defenses. Additionally, the adaptive scale prevents the inclusion of backdoor updates from devastating the overall training of the diffusion models. Although Model Poison [1] can achieve the aim of DataStealing under FedAvg (12.86/0.0069), a large and fixed scale value makes it hard to defeat the advanced defenses or causes the training of diffusion model

14

0.14

12

0.12

10

0.10

8

0.08

MSI

E

6

0.06

-FID_AdaSCP-FedAvg

4

-FID_AdaSCP-MultiKrum

0.04

MSE_AdaSCP-FedAvg

MSE_AdaSCP-MultiKrum

0.02

0

0.00

10

50

100

500

1000

2500

5000

Trigger Number

collapse, as shown in Fig. 3. Our adaptive scale solves these problems with an acceptable and proper scale value. PGD Poison [53] can overcome all defenses in CIFAR10 by constraining the updates within a limited range. However, the efficiency of this attack remains a significant concern. AdaSCP achieves the balance between stealthiness and efficiency. The results of BC Layer Substitution [69] show that it is not a good choice for poisoning the diffusion model. The layer substitution process leads to training collapse in the diffusion model, especially under the defense of Krum (132.02/0.1719). This demonstrates that training critical parameters is more effective than layer substitution in attacking the diffusion model.

Compared to CIFAR10, AdaSCP is more effective than other attacks in CelebA. We observe that the optimal tar s is 3 . 60 in CelebA under FedAvg, which is smaller than the corresponding value of 4 . 54 in CIFAR10 and the total number of local clients. The difference means the data proportion in the infiltrated client is more unbalanced in CelebA than in CIFAR10, which can affect the performance of attacking methods. Moreover, experiments on the LSUN Bedroom dataset demonstrate that AdaSCP can achieve the best attack performance on high-resolution images ( 256 × 256 ). This indicates that AdaSCP is more robust and effective with imbalanced and diverse data.

In summary, Data Poison and PGD Poison encounter difficulties in effectively implanting multiple backdoors. Model Poison fails to bypass advanced FL defenses due to improper scale values. BC Layer Substitution is unsuitable for training diffusion models and may lead to training collapse. AdaSCP outperforms other methods by utilizing critical parameters with adaptive scale factors, balancing stealth and efficiency while preventing training collapse in diffusion models.

Comparisons with IID Dataset. Table 2 shows the results of DataStealing on CIFAR10 in IID distribution. In this scenario, the attacker is easier to defeat the defenses. Model Poison [1] success in circumventing the defense of Multi-metrics [24] (10.55/0.0047) and exceed the PGD Poison on average. Our AdaSCP is still superior in attacking on average (8.86/0.0699). It is because the adaptive scale can bypass defenses that other attack methods fail.

4.2 Ablation Study

Ablation on Each Component of AdaSCP. To verify our proposed AdaSCP, including the critical parameter training and adaptive scaling, we conduct an ablation study on each component. We adopt the model poison as our baseline and set the initial scale value as N c . As Table. 3 shows, training with critical parameters helps the model defeat the Multi-metrics defense (30.59/0.0422) with a sacrifice of attack performance in FedAvg and RFA. It is reasonable as only about 80% parameters are included in the training. Furthermore, based on training critical parameters, we evaluate the effect of tar s by replacing s r with tar s directly. It shows that the estimated target scale tar s plays a critical role in defeating many defenses, which proves the effectiveness of our design. However, the deviation of training affects the accuracy of tar s and results in a decline in FedAvg (12.97/0.0157). Then we use tar s to optimize s r with a learning rate η that can alleviate the problem and can achieve slightly better performance. With the stabilization of history scales, we make progress in defeating Multi-Krum (8.23/0.1271) and achieve the best performance on average. Moreover, the decrease in MSE for Multi-Krum and RFA implies that they can be attacked by AdaSCP in a longer training.

Ablation on Different Trigger Number. To assess the impact of trigger quantity on DataStealing , we undertook ablation studies with incremental numbers of triggers with AdaSCP. Fig. 4 shows that reducing the number of target images increases the success of attacking. And it is increasingly hard to steal more data. Following a successful theft, the FID tends to remain at a relatively high level. This represents one of the limitations of DataStealing . More experiments are listed in Appendix A.

5 Discussion on Defense Mechanisms

To safeguard the training of diffusion models in FL, this section discusses potential defensive mechanisms for future research. First, Multi-Krum has shown strong performance in detecting and diluting malicious updates, preventing attacks from quickly degrading the global model's accuracy. However, as shown in Appendix A.1, its effectiveness diminishes over long-term training, suggesting future enhancements like adaptive thresholds [26] or combining it with other methods [55]. Second, differential privacy methods can invalidate indicators by adding noise or norm clipping while sacrificing generative performance, as discussed in Appendix F. Since AdaSCP requires specific indicators with a large magnification factor, identifying candidate indicators and filtering outliers through comparison could enhance efficiency and effectiveness in defending against DataStealing . Third, the durability experiment in Appendix A.2 demonstrates that backdoors diminish after 100 rounds of continued training with clean data, suggesting a potential mitigation strategy for future work [65, 12]. Finally, as the triggers are embedded in the input noise, releasing only the generated outputs without exposing the global model could serve as an additional defensive strategy.

6 Related Work

Trojan Attacks on Diffusion Model. Trojan attacks [5, 9, 10, 49] on diffusion models have been proposed recently. BadDiffusion [9] and TrojDiff [5] attack DDPMs [22] with an additional trigger

term in the forward/backward process. Struppek et al. [49] attacks the text-to-image diffusion models by adding triggers to the text inputs. VillanDiffusion [10] attacks various diffusion models and samplers with image or caption triggers in a unified backdoor framework. In this paper, we show that diffusion models in FL can also be subject to multiple Trojans with combinatorial triggers.

More methods [57, 35, 64, 12, 31] focus on backdoor poison with multiple distributed attackers or optimizing the triggers. 3DFed [31] proposes three modules: constraint loss, noise masks, and decoy models. They adjust hyper-parameters of these modules with the acceptance status (" Accepted ", " Clipped " or " Rejected ") based on the feedback of indicators. However, their indicators and decoy models require collaboration among multiple attackers, which does not align with our task that involves only a single malicious client. We leave DataStealing with multiple distributed attackers for future work.

Trojan Attacks on Federated Learning. Since Bagdasaryan et al. [1] point out that the widelyused FL algorithm FedAvg [37] is vulnerable to model replacement attack, plenty of backdoor attack methods [57, 35, 53, 65, 12, 31, 41, 15, 7, 64, 69] have subsequently emerged. Data poison [6, 20, 33, 9] attacks the global model by replacing the local training data with a mixture of benign and backdoor datasets. Model poison [1] replaces the global updates with scaled malicious updates. PGD poison [53] projects the local gradients into a sphere surrounding the global model. Backdoor-critical layer substitution [69] inserts the critical malicious layers to the benign models to balance the attack effectiveness and stealthiness. The above attack methods are applied to the classification task. We are the pioneers in investigating the issue of stealing massive private data under stringent security measures in FL. Furthermore, we propose AdaSCP to defeat the advanced distance-based defenses.

More Related Works. Various studies are related to our work. A more detailed discussion of related research is provided in the Appendix E.

7 Conclusion

In this paper, we explored the vulnerabilities of diffusion models within the FL framework, highlighting new avenues for privacy threats through our proposed DataStealing task. Even under rigorous local data protection, substantial private data can be exfiltrated from local clients by employing our ComboTs. Furthermore, we introduce AdaSCP, an adaptive attack method, designed to circumvent advanced distance-based defenses. Our method adaptively seeks the optimal scale value with the indicators. It then trains and magnifies critical parameters, allowing for the seamless integration of backdoor gradients into the global model. Our extensive experiments not only confirm the efficacy of existing FL Trojan attacks and defenses but also highlight the superior performance of AdaSCP in circumventing advanced distance-based defenses. This work serves as a critical reminder of the vulnerabilities in FL and underscores the necessity for continuous advancement in defensive strategies to safeguard against evolving threats in the realm of generative models. We hope our findings will catalyze further research and prompt the FL community to prioritize the development of robust mechanisms to protect privacy in the increasingly complex landscape of machine learning models [38, 59].

8 Limitations and Ethical Statements

Due to the limited page number, we discuss them in Appendix F

9 Acknowledgment

This work is supported by the National Key R&D Program of China (Grant No.2022ZD0160101), Fundamental Research Funds for the Zhejiang Provincial Universities (Grant No.226-2024-00208), National Key R&D Program of China (Grant No.2021ZD0112801), National Natural Science Foundation of China (Grant No.62306273) and Fundamental Research Funds for the Central Universities, Sun Yat-sen University (Grant No.24qnpy154).

References

Eugene Bagdasaryan, Andreas Veit, Yiqing Hua, Deborah Estrin, and Vitaly Shmatikov. How to backdoor federated learning. In International conference on artificial intelligence and statistics , pages 2938-2948. PMLR, 2020.

Peva Blanchard, El Mahdi El Mhamdi, Rachid Guerraoui, and Julien Stainer. Machine learning with adversaries: Byzantine tolerant gradient descent. Advances in neural information processing systems , 30, 2017.

Franziska Boenisch, Adam Dziedzic, Roei Schuster, Ali Shahin Shamsabadi, Ilia Shumailov, and Nicolas Papernot. When the curious abandon honesty: Federated learning is not private. In 2023 IEEE 8th European Symposium on Security and Privacy (EuroS&P) , pages 175-199. IEEE, 2023.

Nicolas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tramer, Borja Balle, Daphne Ippolito, and Eric Wallace. Extracting training data from diffusion models. In 32nd USENIX Security Symposium (USENIX Security 23) , pages 5253-5270, 2023.

Weixin Chen, Dawn Song, and Bo Li. Trojdiff: Trojan attacks on diffusion models with diverse targets. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4035-4044, 2023.

Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv:1712.05526 , 2017.

Siyuan Cheng, Guanhong Tao, Yingqi Liu, Guangyu Shen, Shengwei An, Shiwei Feng, Xiangzhe Xu, Kaiyuan Zhang, Shiqing Ma, and Xiangyu Zhang. Lotus: Evasive and resilient backdoor attacks through sub-partitioning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 24798-24809, 2024.

Christopher A Choquette-Choo, Florian Tramer, Nicholas Carlini, and Nicolas Papernot. Label-only membership inference attacks. In International conference on machine learning , pages 1964-1974. PMLR, 2021.

Sheng-Yen Chou, Pin-Yu Chen, and Tsung-Yi Ho. How to backdoor diffusion models? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4015-4024, 2023.

Sheng-Yen Chou, Pin-Yu Chen, and Tsung-Yi Ho. Villandiffusion: A unified backdoor attack framework for diffusion models. Advances in Neural Information Processing Systems , 36, 2024.

Florinel-Alin Croitoru, Vlad Hondru, Radu Tudor Ionescu, and Mubarak Shah. Diffusion models in vision: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2023.

Yanbo Dai and Songze Li. Chameleon: Adapting to peer images for planting durable backdoors in federated learning. In International Conference on Machine Learning , pages 6712-6725. PMLR, 2023.

Cynthia Dwork, Aaron Roth, et al. The algorithmic foundations of differential privacy. Foundations and Trends® in Theoretical Computer Science , 9(3-4):211-407, 2014.

Gongfan Fang, Xinyin Ma, and Xinchao Wang. Structural pruning for diffusion models. Advances in neural information processing systems , 36, 2024.

Pei Fang and Jinghui Chen. On the vulnerability of backdoor defenses for federated learning. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 37, pages 11800-11808, 2023.

Liam Fowl, Jonas Geiping, Wojtek Czaja, Micah Goldblum, and Tom Goldstein. Robbing the fed: Directly obtaining private data in federated learning with modified models. arXiv preprint arXiv:2110.13057 , 2021.

Clement Fung, Chris JM Yoon, and Ivan Beschastnikh. The limitations of federated learning in sybil settings. In 23rd International Symposium on Research in Attacks, Intrusions and Defenses (RAID 2020) , pages 301-316, 2020.

Yuan Gan, Zongxin Yang, Xihang Yue, Lingyun Sun, and Yi Yang. Efficient emotional adaptation for audio-driven talking-head generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 22634-22645, 2023.

Jonas Geiping, Hartmut Bauermeister, Hannah Dröge, and Michael Moeller. Inverting gradients-how easy is it to break privacy in federated learning? Advances in neural information processing systems , 33:16937-16947, 2020.

Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg. Badnets: Identifying vulnerabilities in the machine learning model supply chain. arXiv preprint arXiv:1708.06733 , 2017.

Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems , 30, 2017.

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems , 33:6840-6851, 2020.

Jiyue Huang, Chi Hong, Lydia Y Chen, and Stefanie Roos. Gradient inversion of federated diffusion models. arXiv preprint arXiv:2405.20380 , 2024.

Siquan Huang, Yijiang Li, Chong Chen, Leyu Shi, and Ying Gao. Multi-metrics adaptively identifies backdoors in federated learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 4652-4662, 2023.

Xumin Huang, Peichun Li, Hongyang Du, Jiawen Kang, Dusit Niyato, Dong In Kim, and Yuan Wu. Federated learning-empowered ai-generated content in wireless networks. IEEE Network , 2024.

Jinyuan Jia, Zhuowen Yuan, Dinuka Sahabandu, Luyao Niu, Arezoo Rajabi, Bhaskar Ramasubramanian, Bo Li, and Radha Poovendran. Fedgame: a game-theoretic defense against backdoor attacks in federated learning. Advances in Neural Information Processing Systems , 36, 2024.

Fiona Victoria Stanley Jothiraj and Afra Mashhadi. Phoenix: A federated generative diffusion model. arXiv preprint arXiv:2306.04098 , 2023.

Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 4401-4410, 2019.

James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences , 114(13):3521-3526, 2017.

Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.

Haoyang Li, Qingqing Ye, Haibo Hu, Jin Li, Leixia Wang, Chengfang Fang, and Jie Shi. 3dfed: Adaptive and extensible framework for covert backdoor attack in federated learning. In 2023 IEEE Symposium on Security and Privacy (SP) , pages 1893-1907. IEEE, 2023.

Chao Liang, Fan Ma, Linchao Zhu, Yingying Deng, and Yi Yang. Caphuman: Capture your moments in parallel universes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6400-6409, 2024.

Yingqi Liu, Shiqing Ma, Yousra Aafer, Wen-Chuan Lee, Juan Zhai, Weihang Wang, and Xiangyu Zhang. Trojaning attack on neural networks. In 25th Annual Network And Distributed System Security Symposium (NDSS 2018) . Internet Soc, 2018.

Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV) , December 2015.

Xiaoting Lyu, Yufei Han, Wei Wang, Jingkai Liu, Bin Wang, Jiqiang Liu, and Xiangliang Zhang. Poisoning with cerberus: Stealthy and colluded backdoor attack against federated learning. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 37, pages 9020-9028, 2023.

Zhaoyang Lyu, Xudong Xu, Ceyuan Yang, Dahua Lin, and Bo Dai. Accelerating diffusion models via early stop of the diffusion process. arXiv preprint arXiv:2205.12524 , 2022.

Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics , pages 1273-1282. PMLR, 2017.

Jiaxu Miao, Zongxin Yang, Leilei Fan, and Yi Yang. Fedseg: Class-heterogeneous federated learning for semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8042-8052, 2023.

Pavlo Molchanov, Stephen Tyree, Tero Karras, Timo Aila, and Jan Kautz. Pruning convolutional neural networks for resource efficient inference. arXiv preprint arXiv:1611.06440 , 2016.

Milad Nasr, Reza Shokri, and Amir Houmansadr. Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning. In 2019 IEEE symposium on security and privacy (SP) , pages 739-753. IEEE, 2019.

Dung Thuy Nguyen, Tuan Minh Nguyen, Anh Tuan Tran, Khoa D Doan, and KOK SENG WONG. Iba: Towards irreversible backdoor attacks in federated learning. In Thirty-seventh Conference on Neural Information Processing Systems , 2023.

Thien Duc Nguyen, Phillip Rieger, Roberta De Viti, Huili Chen, Björn B Brandenburg, Hossein Yalame, Helen Möllering, Hossein Fereidooni, Samuel Marchal, Markus Miettinen, et al. { FLAME } : Taming backdoors in federated learning. In 31st USENIX Security Symposium (USENIX Security 22) , pages 1415-1432, 2022.

Mustafa Safa Ozdayi, Murat Kantarcioglu, and Yulia R Gel. Defending against backdoors in federated learning with robust learning rate. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 9268-9276, 2021.

Krishna Pillutla, Sham M Kakade, and Zaid Harchaoui. Robust aggregation for federated learning. IEEE Transactions on Signal Processing , 70:1142-1154, 2022.

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 10684-10695, 2022.

Victor Sanh, Thomas Wolf, and Alexander Rush. Movement pruning: Adaptive sparsity by fine-tuning. Advances in neural information processing systems , 33:20378-20389, 2020.

Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP) , pages 3-18. IEEE, 2017.

Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502 , 2020.

Lukas Struppek, Dominik Hintersdorf, and Kristian Kersting. Rickrolling the artist: Injecting invisible backdoors into text-guided image generation models. arXiv preprint arXiv:2211.02408 , 6(7), 2022.

Ziteng Sun, Peter Kairouz, Ananda Theertha Suresh, and H Brendan McMahan. Can you really backdoor federated learning? arXiv preprint arXiv:1911.07963 , 2019.

Ye Lin Tun, Chu Myaet Thwal, Ji Su Yoon, Sun Moo Kang, Chaoning Zhang, and Choong Seon Hong. Federated learning with diffusion models for privacy-sensitive vision tasks. In 2023 International Conference on Advanced Technologies for Communications (ATC) , pages 305-310. IEEE, 2023.

Paul Voigt and Axel Von dem Bussche. The eu general data protection regulation (gdpr). A Practical Guide, 1st Ed., Cham: Springer International Publishing , 10(3152676):10-5555, 2017.

Hongyi Wang, Kartik Sreenivasan, Shashank Rajput, Harit Vishwakarma, Saurabh Agarwal, Jy-yong Sohn, Kangwook Lee, and Dimitris Papailiopoulos. Attack of the tails: Yes, you really can backdoor federated learning. Advances in Neural Information Processing Systems , 33:16070-16084, 2020.

Pengyu Wang, Yuan Gan, Panpan Shui, Fenggen Yu, Yan Zhang, Songle Chen, and Zhengxing Sun. 3d shape segmentation via shape fully convolutional networks. Computers & Graphics , 76:182-192, 2018.

Shuaiqi Wang, Jonathan Hayase, Giulia Fanti, and Sewoong Oh. Towards a defense against federated backdoor attacks under continuous training. arXiv preprint arXiv:2205.11736 , 2022.

Ryan Webster. A reproducible extraction of training images from diffusion models. arXiv preprint arXiv:2305.08694 , 2023.

Chulin Xie, Keli Huang, Pin-Yu Chen, and Bo Li. Dba: Distributed backdoor attacks against federated learning. In International conference on learning representations , 2019.

Ling Yang, Zhilong Zhang, Yang Song, Shenda Hong, Runsheng Xu, Yue Zhao, Wentao Zhang, Bin Cui, and Ming-Hsuan Yang. Diffusion models: A comprehensive survey of methods and applications. ACM Computing Surveys , 56(4):1-39, 2023.

Yi Yang, Yueting Zhuang, and Yunhe Pan. Multiple knowledge representation for big data artificial intelligence: framework, applications, and case studies. Frontiers of Information Technology & Electronic Engineering , 22(12):1551-1558, 2021.

Dong Yin, Yudong Chen, Ramchandran Kannan, and Peter Bartlett. Byzantine-robust distributed learning: Towards optimal statistical rates. In International Conference on Machine Learning , pages 5650-5659. Pmlr, 2018.

Hongxu Yin, Arun Mallya, Arash Vahdat, Jose M Alvarez, Jan Kautz, and Pavlo Molchanov. See through gradients: Image batch recovery via gradinversion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 16337-16346, 2021.

Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365 , 2015.

Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In International conference on machine learning , pages 3987-3995. PMLR, 2017.

Hangfan Zhang, Jinyuan Jia, Jinghui Chen, Lu Lin, and Dinghao Wu. A3fl: Adversarially adaptive backdoor attacks to federated learning. In Thirty-seventh Conference on Neural Information Processing Systems , 2023.

Zhengming Zhang, Ashwinee Panda, Linyue Song, Yaoqing Yang, Michael Mahoney, Prateek Mittal, Ramchandran Kannan, and Joseph Gonzalez. Neurotoxin: Durable backdoors in federated learning. In International Conference on Machine Learning , pages 26429-26446. PMLR, 2022.

Joshua Christian Zhao, Atul Sharma, Ahmed Roushdy Elkordy, Yahya H Ezzeldin, Salman Avestimehr, and Saurabh Bagchi. Loki: Large-scale data reconstruction attack against federated learning through model manipulation. In 2024 IEEE Symposium on Security and Privacy (SP) , pages 30-30. IEEE Computer Society, 2023.

Xingchen Zhou, Ming Xu, Yiming Wu, and Ning Zheng. Deep model poisoning attack on federated learning. Future Internet , 13(3):73, 2021.

Ligeng Zhu, Zhijian Liu, and Song Han. Deep leakage from gradients. Advances in neural information processing systems , 32, 2019.

Haomin Zhuang, Mingxian Yu, Hao Wang, Yang Hua, Jian Li, and Xu Yuan. Backdoor federated learning by poisoning backdoor-critical layers. International Conference on Learning Representations , 2024.

A Additional Experiments

We conduct more experiments on CIFAR10 to evaluate the performance of AdaSCP with different settings and hyperparameters.

A.1 Attacking in Extended Training

We conducted an experiment to demonstrate that existing defense mechanisms struggle to prevent successful attacks by adversaries over extended training periods. While AdaSCP fails to succeed within 300 rounds on CIFAR-10 under the defense of Multi-Krum in Table 1, the outcome changes when the duration is extended to 1500 rounds. As shown in Fig. 5, AdaSCP can successfully defeat Multi-Krum defenses given more training time. Furthermore, a longer stable attack is more stealthy and has fewer side effects on the performance of the diffusion model, which is 8.55/0.0088 in round 1,500 compared to the similar result of attacking Foolsgold 24.21/0.0129 in Table 1.

Target image:

16

0.16

14

0.14

12

0.12

10

0.10

D

E

8

0.08

M

6

0.06

4

FIDAdaSCP-MultiKrum

0.04

2

MSE_AdaSCP-MultiKrum

0.02

0.00

0

100

200

300

400

500

600

700

800

900

100011001200

）1300

1400

1500

Round

A.2 Durability of AdaSCP Attack

The durability of a backdoor measures the time span an inserted backdoor continues to be effective after the attacker withdraws. To assess the durability of our AdaSCP attack, we continue training an attacked model (AdaSCP-FedAvg) with benign data and FedAvg protocol. As shown in Fig. 6, the backdoors implanted by AdaSCP remain effective for approximately 100 rounds. After 100 rounds, the backdoors become almost ineffective as the MSE is higher than 0.1. Moreover, the FID decreases with further training. This suggests a defense strategy for DataStealing : continuing to train with benign data after the initial training can effectively mitigate the impact of multiple backdoors.

14

0.14

12

0.12

10

0.10

D

8

0.08

MSE

6

0.06

FID_FedAvg

0.04

MSE_FedAvg

0.02

0.00

0

50

100

150

200

250

300

Round

A.3 Analysis on Defending against Malicious Updates

To better understand the defense process, we analyze the distance between malicious and mean benign updates under Krum and Multi-Krum defense across different training rounds. Compared to other defense algorithms, Krum and Multi-Krum display more pronounced variations in distance. The results in Fig. 7 show that the initial scale value is substantially higher than the optimal value. This discrepancy causes the initial distance of the malicious updates to be approximately 6.7 times greater than that of the benign updates. As AdaSCP progressively optimizes the scale value, malicious updates gradually align with those benign updates, approaching the optimal relative distance necessary for the attack to bypass the defenses.

/s32

/s48

/s49

/s48

/s50

/s48

/s51

/s48

/s52

/s48

/s53

/s48

/s54

/s48

/s55

/s48

/s56

/s48

/s57

/s48

/s49

/s48

/s48

/s48

/s50

/s52

/s54

/s65

/s100

/s97

/s83

/s67

/s80

/s45

/s77

/s117

/s108

/s116

/s105

/s75

/s114

/s117

/s109

/s65

/s100

/s97

/s83

/s67

/s80

/s45

/s75

/s114

/s117

/s109

/s82

/s111

/s117

/s110

/s100

/s82

/s101

/s108

/s97

/s116

/s105

/s118

/s101

/s68

/s105

/s115

/s116

/s97

/s110

/s99

/s101

/s32

/s32

A.4 AdaSCP Stability Assessment

To evaluate the stability of AdaSCP, we conducted experiments with two additional seeds, resulting in three distinct Non-IID data distributions. Our results are presented as mean ± standard deviation , as shown in Table 4. The values in the 'Mean' row represent the overall mean and standard deviation of the averaged values obtained from the three different seeds. The 'Mean' values in Table 4 show a tolerable deviation from the previously reported results, indicating the robustness of our method to variations in Non-IID data distributions.

A.5 Ablation on Proportion of Critical Parameters

To assess the impact of the proportion τ in selecting critical parameters, we conducted an ablation study presented in Table 5. The results indicate that more critical parameters lead to better attack performance. However, the performance with τ = 0 . 99 is as effective as with τ = 0 . 8 , suggesting that some parameters are redundant in attacking. Consequently, we selected τ = 0 . 4 in other experiments to achieve a trade-off between performance and the critical parameter count, which contains approximately 80% of the parameters. To ensure a fair comparison, we maintain the same percentage of parameters in the BC Layer substitution.

A.6 Ablation on Learning Rate in Optimizing Scale

To investigate the impact of different learning rates η on the performance of AdaSCP under the defense of FedAvg and Multi-Krum, we conducted an ablation study in Table 6. It shows that at relatively lower learning rates, the training process is more stable and effective. When η = 0 . 2 , AdaSCP achieves better performance while minimizing side effects on the FID of generated data. Hence, we adopt η = 0 . 2 in our paper.

A.7 Ablation on Patch Size of ComboTs.

To verify the impact of different patch sizes in ComboTs in DataStealing , we conducted an ablation study with various patch sizes. Table 7 shows that for CIFAR10 images ( 32 × 32 ), a 3 × 3 patch is the minimum size capable of successfully executing the attack. Although larger patches can also succeed in attacking, they would harm both FID and MSE. Smaller patches are insufficient to distinguish the triggers from the noise, while larger patches obscure the image and degrade performance. Therefore, we set the patch size to 3 × 3 in CIFAR10. For CelebA, we increased the patch size to 5 × 5 because the images in CelebA are preprocessed to ( 64 × 64 ), which is twice the resolution of CIFAR-10. As for LSUN Bedroom, we choose 25 × 25 patch size for attacking ( 256 × 256 ) images.

A.8 Ablation on Total Number of Clients.

To verify the effectiveness of our method across varying numbers of clients, we conducted experiments by increasing the total number of clients from 5 to 10. The results in Table 8 demonstrate that AdaSCP remains effective in executing attacks across different client numbers, confirming its robustness.

A.9 Complexity and Computational Analysis

The complexity of ComboTs depends on the time required to select triggers from potential positions, as defined in Section 3.2. The efficiency of AdaSCP is influenced by the batch size and a hyperparam-

eter T (threshold), demonstrated in Appendix C.1 and Algorithm 4. The primary source of complexity is the process of searching for critical parameters and identifying candidate indicators. The runtime of AdaSCP varies based on different hyperparameters and model complexity. For example, when finetuning with the LSUN bedroom dataset at a resolution of 256 × 256 , with a batch size of 3 and a hyperparameter T set to 0.05, the runtime for finding critical parameters and candidate indicators is 2 minutes and 29 seconds. This process is performed after all candidate indicators are exhausted, as demonstrated in Algorithm 1. In our experiments, we set the number of candidate indicators to 10. Increasing the number of candidate indicators can enhance the efficiency of our method.

A.10 More Visual Comparison

Further visual comparisons are presented in Fig. 8 and Fig. 10. The diversity of content in these targets suggests that our method is applicable across various image types and can achieve superior performance. Moreover, the results indicate that the training of the diffusion model within the FL system reveals greater privacy security vulnerabilities than those observed in discriminative models.

B Backdoor Attacks Implementation

Dataset and Implementation Details. We implement all experiments on four 48G NVIDIA A40 GPUs. When estimating the importance score, we adopt a batch size of 64 for CIFAR10 and CelebA, and 3 for LSUN Bedroom. The Non-IID distribution of CIFAR10 and CelebA used in Table 1 are shown in Table 9. The first client is chosen for attack. The FID/MSE of the pretrained model is 6.24/0.1428 in CIFAR10, 6.00/0.1507 in CelebA, and 9.84/0.2594 in LSUN Bedroom. It needs to be noted that we preprocess the CelebA with the pre-processing code from StyleGAN [28] repository 2 . The batch size is 128 for CIFAR10, 64 for CelebA and 8 for LSUN Bedroom.

PGD Poison. Projected Gradient Descent (PGD) attack [53] guarantees that the malicious model sent by the attacker would not be rejected by the norm-based defenses when the norm of updates is less than a radius. We note that the original radius is too small to attack the diffusion model. Hence, we set the radius as 8 to achieve a tradeoff between the effectiveness and stealthiness of the PGD attack.

BC Layer Substitution. To defeat the advanced defenses in classification tasks, critical layers [69] are effective. However, DataStealing poses two challenges in estimating the critical layers: 1) There is no such efficient metric as Backdoor Success Rate (BSR) to justify the importance of every layer. The low sampling efficiency of diffusion models makes such a method impractical. 2) The diffusion model exhibits a substantially increased layer number relative to previously employed architectures like ResNet18, which further increases the difficulty in finding critical layers. Hence, for the BC layer substitution attack, we adopt the momentum-based importance score [46, 15] to identify the critical layers.

AdaSCP

Data Poison

BC Layer Sub.

Target

AdaSCP

Data Poison

BC Layer Sub.

Target

AdaSCP

Data Poison

BC Layer Sub.

Target

AdaSCP

Data Poison

BC Layer Sub.

Target

AdaSCP

Data Poison

BC Layer Sub.

Target

AdaSCP

Data Poison

BC Layer Sub.

Target

FedAvg

Krum

Multi-Krum

RFA

Multi-metrics

FedAvg

Krum

Multi-Krum

Multi-metrics

FedAvg

Krum

Multi-Krum

RFA

Multi-metrics

FedAvg

Krum

Multi-Krum

RFA

Multi-metrics

FedAvg

Krum

Multi-Krum

RFA

Multi-metrics

FedAvg

Krum

Multi-Krum

RFA

Multi-metrics

AdaSCP

Data Poison

BC Layer Sub.

Target

N/A

AdaSCP

Data Poison

BC Layer Sub.

Target

N/A

AdaSCP

Data Poison

BC Layer Sub.

Target

N/A

AdaSCP

Data Poison

BC Layer Sub.

Target

N/A

AdaSCP

Data Poison

BC Layer Sub.

Target

N/A

AdaSCP

Data Poison

BC Layer Sub.

Target

N/A

FedAvg

Krum

Multi-Krum

Multi-metrics

FedAvg

Krum

Multi-Krum

Multi-metrics

FedAvg

Krum

Multi-Krum

RFA

Multi-metrics

FedAvg

Krum

Multi-Krum

RFA

Multi-metrics

FedAvg

Krum

Multi-Krum

Multi-metrics

FedAvg

Krum

Multi-Krum

RFA

Multi-metrics

AdaSCP

Data Poison

BC Layer Sub.

Target

AdaSCP

Data Poison

BC Layer Sub.

Target

FedAvg

Krum

Multi-Krum

RFA

Multi-metric

FedAvg

Krum

Multi-Krum

RFA

Multi-metrics

Targets

Sampled Images

Sampled Images

C More Algorithms of AdaSCP

C.1 Diffusion Importance Estimation

Algorithm 4 demonstrates the process of estimating the parameter importance with accumulated gradients in dominant timesteps under the Taylor expansion framework, as described in the literature [14]. The timesteps used to estimate the importance score have higher loss values than the others. Hence, we refer to these timesteps as dominant timesteps (Line 7). After estimating the importance score of the diffusion model, we create a mask of critical parameters M G with the top τ structural elements from all scores (Line 15). While estimating the importance score, we record the accumulated gradients | ˆ w G | and their Hessian matrix | H G | to subsequently find indicators (Lines 11-12 and 16-18).

Algorithm 4 Diffusion Importance Estimation

Input: Indicator-restored global model w G and its parameter θ , parameters for finding indicators ˆ θ , benign datasets D , a threshold T , a proportion τ

Algorithm 5 Train Critical Parameters

C.2 Train Critical Parameters

Algorithm 5 illustrates the process of training the critical parameters with benign dataset and backdoor dataset. We train benign and backdoor data in one batch (Line 2-7). With Eq. 1 and Eq. 2, we add benign noise and Trojan noise into the corresponding data. After training the diffusion model, we only update the critical parameters with the mask M G (Line 8 and 9). Finally, we scale the accumulated malicious updates with s r and return the scaled model w r .

C.3 Optimize Scale

Algorithm 6 illustrates the process of optimizing the scale value according to the estimated target scale value and history scale values. According to the purpose of tar s , demonstrated in Eq. 5, the scale value is better to be close to tar s . Hence, when ' Accepted " by the server, the previous scale value s r -1 can defeat the defense and should move toward tar s to achieve better attacking performance (Line 1-5). To prevent tar s from greatly exceeding the previously rejected scale, the 20th percentile value of the sorted his rej is used as the upper boundary for tar s , as illustrated in Algorithm 7. Then we optimize the scale value with the current optimal target tar ∗ s (Line 4) and set the flag of the previous accepted state Accept pre to True.

When ' Rejected " by the server, tar s will be set to 0 and the scale value should be decreased to defeat the defenses (Line 7-11). To avoid the optimized scale s ′ r being significantly lower than the accepted scales, we find the optimal scale with Algorithm 8. Specifically, the medium value of the sorted his acc is adopted as the lower bound for s r . If a malicious update was previously ' Accepted ' but is now ' Rejected ', we reduce the learning rate by multiplying it with a weight decay factor. This adjustment is made based on the value of the flag Accept pre . Finally, we set the flag of the flag Accept pre to False.

Algorithm 6 Optimize Scale

Input: Target scale tar s , previous scale s r -1 , scale learning rate η , scale weight decay d Output: Current scale s

Algorithm 7

Input: Target value tar s , history rejected scale values his rej Output: Current optimal target value tar ∗ s 1: if len ( his rej ) > 10 then 2: med rej ← his rej [ len ( his rej ) // 5] 3: tar ∗ s ← min ( tar s , med rej ) 4: else 5: tar ∗ s ← tar s 6: end if 7: return tar ∗ s

Algorithm 8 Find Optimal Scale Value

Input: Optimised scale value s ′ r , history accepted scale values his acc Output: Current optimal scale value s r 1: if len ( his acc ) > 10 then 2: med acc ← his acc [ len ( his acc ) // 2] 3: s r ← max ( s ′ r , med acc ) 4: else 5: s r ← s ′ r 6: end if 7: return s r

C.4 Restore and Dequeue Candidate Indicators

Algorithm 9 illustrates the process of restoring and dequeuing the candidate indicators. Specifically, we set the indicator update into the value before magnifying. To alleviate the impact of magnifying the indicator update, we remove the used indicator and use the other indicators. Hence, we dequeue the front element in the queue of candidate indicators and prepare for implanting the new indicator in the round.

Algorithm 9 Restore and Dequeue Candidate Indicators

Input: Indicator magnification factor k , uploaded indicator θ ′ w r -1 -G r -1 ,I 1 , previous candidate indicators I r -1

Output: Current candidate indicators I r , indicator-restored global model w G

1: θ w r -1 -G r -1 ,I 1 ← 1 k · θ ′ w r -1 -G r -1 ,I 1

2: I r ← I r -1 . dequeue()

3: return w G , I r

D Mathematical Derivation of the Optimal Scale

Theorem 1: ∑ n c i =1 λ i /λ adv is the optimal scale value of the adversary client to replace the updates with the malicious update.

Proof of Theorem 1: According to the aggregation process of FL [37, 50], the server updates the global model with the updates from clients. To simplify the analysis, we assume the learning rate of the server is 1 . Given n c clients participating in the aggregation, we can get the server's updating equation:

where W i represents the trained local model in the i -th client, and λ i is the weight for averaging. To simplify the notation, let ∆ θ i denote the update of the local model θ W i,r -1 -G r -1 . Then the updating of the server can be represented as:

To proof that ∑ n c i =1 λ i /λ adv is the optimal scale value, we scale the adversary updates to ∑ nc i =1 λ i λ adv ∆ θ adv . Then we can get:

When training converges, ∆ θ i is approaching zero and the global updates can be replaced by the malicious updates ∆ θ adv .

Theorem 2: Since the gradient and curvature are approaching zero, we assume that the training update at indicator I 1 is similar in every client. Under the assumption, the target scale value ∑ n c i =1 λ i /λ adv = k -1 f -1 , where f = θ Gr -Gr -1 ,I 1 1 k · θ ′ wr -1 -Gr -1 ,I 1 , when the indicator-implanted malicious model is 'Accepted" by the server.

Proof of Theorem 2: We will derive the target scale value ∑ n c i =1 λ i /λ adv according to the implanted indicator. Given the indicator I 1 , we can get the returned indicator update θ G r -G r -1 ,I 1 . Let ∆ θ G,I 1 represent θ G r -G r -1 ,I 1 and ∆ θ ′ adv,I 1 denote the uploaded indicator update θ ′ w r -1 -G r -1 ,I 1 . We can get the following equation:

▷ Restore the indicator to get w G ▷ Remove the front element

where ∆ θ ′ adv,I 1 = k ∆ θ adv,I 1 as the training update at I 1 is magnified by k to form the implanted indicator. Since the gradient and curvature are approaching zero at position I 1 , we assume the training update ∆ θ i,I 1 between each client is similar. Then Eq. 7 can be re-written as:

By rearranging Eq. 8, the target scale value ∑ n c i =1 λ i /λ adv can be derived:

where f = ∆ θ G,I 1 θ adv,I 1 = θ Gr -Gr -1 ,I 1 1 k · θ ′ wr -1 -Gr -1 ,I 1 , which is used in the Algorithm 3 Line 1. In our experiments, k is set to 50 . When ' Rejected ' by the server, ∆ θ G,I 1 is nearly equal to θ i,I 1 and f is close to 1 under our assumption. Considering the deviation introduced by training, the result of Eq. 11 is either a negative or a very large number. Hence, we set the ' Accepted " range to (0 , 2 · N c ] . Then the equation of target scale value tar s is:

The equation is used in the Algorithm 3 Line 2. As illustrated in Sec. 3.3, we will optimize the scale value according to the results of tar s .

E More Related Works

Defenses Against Trojan Attacks. Recently, the defense methods against Trojan attacks have achieved great progress. These methods can be broadly categorized into two types according to their algorithms. The first type is distance-based defenses [2, 60, 17, 44, 24, 26]. They try to distinguish the malicious updates from uploaded gradients according to distance or scores and then aggregate updates that have undergone filtering. The second type of defenses [43, 42, 50] is based on differential privacy algorithm [13], which use clipping and adding noise to defense backdoor attacks. However, we notice that these techniques will lead to the diffusion model collapsing or failing to converge. Hence, we utilize several advanced distance-based methods [2, 17, 44, 24] as the defense strategies in this paper.

Data Reconstruction Attack. Despite only sending updates in FL, prior works have demonstrated that attackers can still obtain information about private training data through various techniques [8, 40, 47, 56, 4, 19, 61, 68, 3, 16, 66]. Membership inference attacks [8, 40, 47, 56, 4] involve determining whether a specific data sample was part of the model's training dataset by analyzing the model's outputs. However, they require a large amount of model output data for training and the original data for analysis. A more common and potentially more dangerous methods are the gradient-based reconstruction approaches. Gradient Inversion attacks [19, 61, 68, 23] in FL involve reconstructing the original training data from the gradient updates shared between clients and the server. Linear Layer Leakage (LLL) attacks [3, 16, 66] exploit the gradients of fully connected layers to infer

and reconstruct the input data. Gradient-based attack methods, which rely entirely on the gradient information, are explored on deterministic models and suffer from low data recovery quality. Unlike these attack methods, our work focuses on diffusion models and can accurately and efficiently reconstruct large amounts of specified training data through numerous Trojans.

F Limitations and Ethical Statements

Although AdaSCP can defeat many distance-based defenses, there are still some limitations. After a successful multiple backdoor attack, the FID slightly increases, which is +10.31 in CIFAR10 and +1.73 in CelebA. Designing better triggers may reduce the impact of the multi-backdoor attack on the FID, making the attack more covert. Some defense mechanisms, such as norm clipping or differential privacy methods, can render our indicator-based attacks ineffective. These methods compromise the indicators' effectiveness. However, such approaches are not suitable for training diffusion models, as they can easily lead to non-convergence or model collapse. When pre-training the diffusion model with norm clipping in FL, the FID of the pre-trained model is 80.82, which is 6.24 without the norm clipping. We leave these limitations for future research.

While our framework aims to enhance the robustness of diffusion models in FL, we recognize the potential for misuse of our findings in more computer vision tasks [18, 32, 54, 38, 59]. Nonetheless, raising awareness about this issue can lead to the adoption of stricter security standards in the design and implementation of training diffusion models with FL, thereby minimizing the risks of data breaches and misuse. We hope our red-teaming effort can draw more attention from the federated learning (FL) community to the critical privacy security issues associated with diffusion models.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: We included the paper's contributions in the abstract and introduction.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discussed the limitations in Appendix F.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We proof the Theorem 1 and 2 in Appendix D.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We included the details of our algorithms in Sec. 3.3 and Appendix C. The code will be released upon the publication of our paper.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: The code is released in: https://github.com/yuangan/DataStealing.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Please refer to Sec. 4 and Appendix B for experimental details.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We conducted repeated experiments with various data distributions, and the error bars are provided in Appendix A.4.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Please refer to Appendix B for the GPU type used in our experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We acknowledge the Code of Ethics and obey them in our paper.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: The ethical statements can be found in Appendix F.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: Our paper is based on publicly available datasets and models.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We cited the datasets and codes used in our paper. All datasets and codes used in our paper are publicly available.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: We do not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper identifies a pioneering privacy threat in Federated Learning (FL) introduced by popular diffusion models, called DataStealing. The researchers show that an attacker can use Combinatorial Triggers (ComboTs) to steal private data from a client in vanilla FL, even with strict data management protocols in place. For advanced FL systems, they propose an attack method (AdaSCP) to bypass distance-based defenses by adaptively scaling critical parameter updates with indicators. Extensive experiments demonstrate that AdaSCP not only circumvents existing defenses but also successfully steal thousands of images from the system. The paper calls for increased attention to privacy issues in diffusion models within the FL community.\n\nSTRENGTHS:\n1. They propose a new benchmark (DataStealing) to demonstrate the security vulnerability of training federated diffusion models, which can be utilized to circumvent the strict data protection measures of some organizations. They show that multiple Trojans (ComboTs) are effective in leaking thousands of data. \n2. The proposed AdaSCP is the first to update the critical parameters and utilize indicators to optimize the scale value according to the theoretical target value. The authors provide a simple proof for calculating the optimal scaling value with the returned indicator updates. \n3. The paper offers valuable insights into the critical privacy and security concerns associated with federated diffusion models. It proposes a crucial security issue and an attack method for the FL community.\n4. Well-written paper with clear motivation, technical approach, and clear figures. Extensive experiments support their claim well.\n\nWEAKNESSES:\n1. The reason that Model Poisoning cannot bypass the defenses is only demonstrated in Section 3.3. More quantitative analysis, including details of defending against malicious updates with different attack methods, can enhance understanding. A figure is also acceptable.\n2. The error bars are not reported. Although it will require significant computational resources, I recommend conducting repeat experiments with different seeds, at least for the proposed AdaSCP. This will enhance the academic quality of this paper.\n3. The recent studies [1, 2, 3] should be discussed in the related work.\n4. Some minor typos:\n- “..”->”.” in line 200.\n\t- “front” should be “first” to keep consistent in Algorithm 2 line 6.\n[1] Jia J, Yuan Z, Sahabandu D, et al. FedGame: a game-theoretic defense against backdoor attacks in federated learning[J]. Advances in Neural Information Processing Systems, 2024, 36.\n[2] Cheng S, Tao G, Liu Y, et al. Lotus: Evasive and resilient backdoor attacks through sub-partitioning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024: 24798-24809.\n[3] Huang J, Hong C, Chen L Y, et al. Gradient Inversion of Federated Diffusion Models[J]. arXiv preprint arXiv:2405.20380, 2024.\n\nQUESTIONS:\nPlease see the weakness part.\n\nLIMITATIONS:\nPlease see the weakness part.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a novel attack method named DataStealing, which exploits vulnerabilities in diffusion models trained in Federated Learning (FL) systems. The authors highlight that diffusion models, despite their advanced capabilities in data generation, present new privacy threats when integrated with FL. They propose a method called Combinatorial Triggers (ComboTs) to steal images by embedding multiple backdoors in the diffusion models. To counteract distance-based FL defenses, they introduce an Adaptive Scale Critical Parameters (AdaSCP) attack that evaluates the importance of parameters and adaptively scales malicious updates to evade detection. Extensive experiments validate the effectiveness of AdaSCP in stealing private data from diffusion models.\n\nSTRENGTHS:\nStrengths\n\n1. Novelty and Relevance: The paper proposes a new task, which performs data stealing from diffusion models trained in FL. Due to the popularity of diffusion models and FL systems, it points out an emerging threat in this critical area.\n\n2. Clarity: The paper is well-organized and easy to follow. It considers scenarios where defenses are available and proposes an adaptive method to evade the defenses.\n\n3. Experiments: The authors conduct experiments on two image datasets, comparing their methods against various state-of-the-art attacks and defenses, providing an empirical foundation for their claims.\n\nWEAKNESSES:\nWeaknesses:\n\n1. Defense Mechanisms: A discussion on potential defense mechanisms or mitigations would make the work have better impacts in this area and further highlight the significance of the threat.\n\n2. Assumptions: In the experiments, the proportion of malicious clients is 1/5, which might not be realistic in real practice.\n\n3. More Experiments: The attack is evaluated on two datasets (CIFAR10 and CelebA). Expanding the evaluation to include more diverse datasets could further support the work.\n\nQUESTIONS:\nNA\n\nLIMITATIONS:\nThe authors discussed the limitations and potential negative societal impact of the work in Appendix.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper titled \"DataStealing: Steal Data from Diffusion Models in Federated Learning with Multiple Trojans\" investigates a novel privacy vulnerability in federated learning (FL) when training diffusion models. The authors introduce a new attack methodology named DataStealing, which leverages multiple Trojans to exfiltrate private data from local clients. The attack utilizes Combinatorial Triggers (ComboTs) to map extensive data and proposes the Adaptive Scale Critical Parameters (AdaSCP) attack to bypass advanced FL defenses.\n\n**Key Contributions:**\n* 1\\. **Identification of Vulnerability**:\n\n    - The paper identifies a new privacy threat in FL where diffusion models can leak substantial private data. Despite stringent privacy measures, attackers can exploit these models to steal high-quality local data.\n\n* 2\\. **Combinatorial Triggers (ComboTs)**:\n\n    - A method to select multiple triggers for backdoor attacks, significantly increasing the capability to map and steal large amounts of private data.\n\n* 3\\. **Adaptive Scale Critical Parameters (AdaSCP)**:\n\n    - An attack strategy that evaluates and scales critical parameter updates, making the malicious updates indistinguishable from benign ones. This circumvents distance-based defenses effectively.\n\n* 4\\. **Experimental Validation**:\n\n    - Extensive experiments demonstrate the ability of the proposed methods to leak thousands of images from training diffusion models in an FL setup. AdaSCP is shown to be highly effective against advanced FL defenses.\n\n**Conclusion**:\n\nThe paper highlights the severe privacy risks associated with training diffusion models in FL and calls for more robust defensive measures to protect against such vulnerabilities. The findings emphasize the need for continuous advancements in FL security to safeguard private data.\n\nSTRENGTHS:\n**Strengths of the Paper**\n**Originality**\n- **New Vulnerability Identification**: The paper identifies a previously unrecognized vulnerability in federated learning (FL) when using diffusion models, termed as DataStealing. This identification highlights a novel attack surface in the intersection of generative models and FL.\n- **Innovative Attack Methodologies**: The introduction of Combinatorial Triggers (ComboTs) and the Adaptive Scale Critical Parameters (AdaSCP) attack represents significant originality. ComboTs utilize multiple triggers to enhance the attack’s capability, while AdaSCP circumvents advanced defenses by adaptively scaling critical parameter updates.\n- **Creative Combination of Techniques**: The paper creatively combines the concepts of Trojans in generative models with adaptive scaling techniques to formulate an effective attack strategy against FL systems.\n\n**Quality**\n\n- **Rigorous Experimental Validation**: The paper presents extensive experiments to validate the effectiveness of the proposed attack methodologies. The results are comprehensive, demonstrating the ability of AdaSCP to defeat various advanced FL defenses.\n\n- **Detailed Methodological Explanation**: The methodologies are described in detail, with clear explanations of the underlying principles and the implementation of the attack strategies. This thorough presentation ensures that the proposed methods are reproducible and verifiable.\n\n- **Robustness of Results**: The experimental results are robust, showing consistent performance across different datasets (CIFAR10 and CelebA) and FL settings (IID and non-IID distributions).\n\n**Clarity**\n\n- **Well-Organized Structure**: The paper is well-organized, guiding the reader logically from the identification of the problem to the proposed solutions and experimental validation.\n\n- **Clear Writing Style**: The writing is clear and concise, making complex technical content accessible. Each section is well-articulated, with smooth transitions that maintain the reader’s engagement.\n\n- **Effective Use of Visual Aids**: Figures and tables are used effectively to illustrate key points and results. These visual aids enhance the reader’s understanding of the methodologies and the significance of the findings.\n\n**Significance**\n\n- **High Relevance to Privacy and Security**: The paper addresses a critical issue in the domain of FL, highlighting significant privacy risks associated with training diffusion models. The identified vulnerabilities and proposed solutions are highly relevant to the ongoing discussions about data privacy and security in machine learning.\n\n- **Impact on Future Research**: The findings of this paper are likely to stimulate further research in the field, prompting the development of more robust defense mechanisms in FL. By uncovering new attack strategies, the paper sets the stage for advancements in both attack and defense techniques.\n\n- **Broad Applicability**: The proposed methodologies and findings are not limited to a specific application but are broadly applicable to various domains where FL and generative models are used. This broad applicability enhances the overall significance of the paper.\n\n**Conclusion**\n\nThe paper \"DataStealing: Steal Data from Diffusion Models in Federated Learning with Multiple Trojans\" excels in originality, quality, clarity, and significance. It introduces novel vulnerabilities and attack methodologies in FL, backed by rigorous experimental validation and clear presentation. The contributions of this paper are highly relevant to the research community and hold significant potential for future advancements in privacy and security in machine learning.\n\nWEAKNESSES:\n**Weaknesses of the Paper**\n\n**Limited Discussion on Defense Mechanisms**\n\n- **Lack of Proactive Defense Strategies**: While the paper thoroughly explores the vulnerabilities and proposes robust attack methodologies, it falls short in discussing proactive defense mechanisms. It would significantly benefit from suggesting or exploring potential defense strategies to mitigate the identified vulnerabilities. Including a section on how to strengthen FL systems against such attacks would provide a more balanced view.\n\n**Depth of Contextualization**\n- **Comparative Analysis with Prior Work**: Although the paper positions its contributions within the broader context of federated learning and privacy security, a deeper comparative analysis with specific prior studies would enhance its impact. For example, more detailed comparisons with existing backdoor and Trojan attack methodologies would help in highlighting the advancements made by this work.\n- **Exploration of Related Work**: The related work section could be expanded to include a more comprehensive review of similar vulnerabilities in federated learning and generative models. This would help in better contextualizing the novelty and significance of the proposed methods.\n\n**Experimental Scope**\n- **Scalability and Generalizability**: While the experiments are robust, they are limited to specific datasets (CIFAR10 and CelebA) and certain configurations (IID and non-IID distributions). Exploring the scalability and generalizability of the proposed attack methodologies to other datasets and real-world scenarios would strengthen the empirical validation. For instance, including more diverse datasets or different FL frameworks could provide insights into the broader applicability of the findings.\n\n- **Extended Analysis of Parameters**: The paper briefly touches upon the impact of various parameters, such as the proportion of critical parameters and patch sizes in ComboTs. A more detailed and systematic analysis of how these parameters influence the attack's effectiveness across different settings would be beneficial. Including ablation studies on a wider range of parameters and configurations would offer a deeper understanding of the methods' robustness.\n\n**Presentation Improvements**\n\n- **Simplifying Technical Explanations**: Some sections of the paper, particularly those explaining the technical details of the methodologies, could be simplified for better readability. This would make the paper more accessible to a broader audience, including those who may not have a deep technical background in the specific area.\n\n- **Enhanced Flow and Readability**: Improving the flow and readability of certain sections, particularly the methodology and experiment results, would enhance the overall presentation. Ensuring smoother transitions between sections and sub-sections would maintain the reader’s engagement and comprehension.\n\n**Specific Suggestions for Improvement**\n\n* 1\\. **Incorporate Proactive Defense Mechanisms**: Add a section discussing potential defense strategies to mitigate the vulnerabilities identified. This could include theoretical approaches, proposed defense mechanisms, or an exploration of existing techniques that could be adapted.\n* 2\\. **Expand Comparative Analysis**: Deepen the comparative analysis with prior work, providing detailed discussions on how the proposed methods advance the state-of-the-art in backdoor and Trojan attacks in FL.\n* 3\\. **Broaden Experimental Validation**: Expand the experimental scope to include more diverse datasets and configurations. Conduct a systematic analysis of various parameters influencing the attack's effectiveness, with detailed ablation studies.\n* 4\\. **Simplify and Enhance Readability**: Simplify technical explanations where possible, and improve the overall flow and readability of the paper. This could involve rephrasing complex sections and ensuring smooth transitions between different parts of the paper.\n\nBy addressing these weaknesses, the paper can further solidify its contributions and provide a more comprehensive and impactful addition to the research area.\n\nQUESTIONS:\n**Questions and Suggestions for the Authors**\n\n**Questions**\n\n* 1\\. **Defense Mechanisms**:\n\n    - What proactive defense strategies do you suggest to mitigate the vulnerabilities identified in the paper? Have you considered evaluating existing defense mechanisms or proposing new ones specifically tailored to counter the DataStealing attack?\n\n* 2\\. **Comparative Analysis**:\n\n    - Can you provide a more detailed comparative analysis with specific prior studies on backdoor and Trojan attacks in federated learning? How do your proposed methods significantly advance the state-of-the-art compared to these existing approaches?\n\n* 3\\. **Scalability and Generalizability**:\n\n    - How do you anticipate your proposed attack methodologies (ComboTs and AdaSCP) will perform on more diverse datasets and in real-world scenarios? Have you considered evaluating your methods on other datasets or federated learning frameworks to validate their scalability and generalizability?\n\n* 4\\. **Parameter Sensitivity**:\n\n    - Can you provide a more systematic analysis of how different parameters (e.g., proportion of critical parameters, patch sizes in ComboTs) influence the effectiveness of the attacks? Detailed ablation studies on these parameters would be beneficial for understanding the robustness of your methods.\n\n* 5\\. **Complexity and Computation**:\n\n    - What are the computational costs and complexities associated with implementing your proposed methods (ComboTs and AdaSCP)? How feasible is it to execute these attacks in practical settings, considering the required computational resources and time?\n\n**Suggestions**\n\n* 1\\. **Proactive Defense Strategies**:\n\n    - Include a dedicated section discussing potential defense strategies to mitigate the vulnerabilities identified. This could involve theoretical approaches, proposed defense mechanisms, or an exploration of existing techniques that could be adapted to counter DataStealing.\n\n* 2\\. **Enhanced Comparative Analysis**:\n\n    - Deepen the comparative analysis with prior work, providing detailed discussions on how your proposed methods advance the state-of-the-art in backdoor and Trojan attacks in federated learning. This can help in better contextualizing the novelty and significance of your contributions.\n\n* 3\\. **Broaden Experimental Validation**:\n\n    - Expand the experimental scope to include more diverse datasets and configurations. Conduct a systematic analysis of various parameters influencing the attack's effectiveness, with detailed ablation studies. This would strengthen the empirical validation and demonstrate the broader applicability of your findings.\n\n* 4\\. **Simplify Technical Explanations**:\n\n    - Simplify the technical explanations where possible to make the paper more accessible to a broader audience, including those without a deep technical background in the specific area. This would enhance the overall readability and impact of the paper.\n\n* 5\\. **Detailed Discussion on Limitations**:\n\n    - Include a more detailed discussion on the limitations of your proposed methods. Address potential challenges and constraints in implementing the attacks, and suggest future research directions to overcome these limitations.\n\nBy addressing these questions and suggestions, the authors can provide a more comprehensive and impactful contribution to the research area, enhancing the clarity, robustness, and significance of their work.\n\nLIMITATIONS:\n**Assessment of Limitations and Potential Negative Societal Impact**\n\n**Addressed Limitations**\n\n* 1\\. **Technical Limitations**:\n\n    - The authors briefly mention the limitations related to the computational complexity and scalability of their proposed methods. They acknowledge the challenges in training diffusion models with ComboTs and the sensitivity of these models to gradient updates.\n\n* 2\\. **Experimental Constraints**:\n\n    - The paper indicates that the experiments were conducted on specific datasets (CIFAR10 and CelebA) and acknowledges the need for further validation on more diverse datasets and real-world scenarios.\n\n* 3\\. **Defensive Measures**:\n\n    - There is a mention of the need for further research to develop robust defensive mechanisms against the identified vulnerabilities in federated learning, suggesting an awareness of the current gap in defensive strategies.\n\n**Potential Negative Societal Impact**\n\n* 1\\. **Privacy and Security Risks**:\n\n    - The paper highlights significant privacy risks associated with federated learning, especially when training diffusion models. The authors acknowledge the potential misuse of the proposed attack methodologies to steal private data, underscoring the importance of addressing these vulnerabilities to protect user privacy.\n\n**Constructive Suggestions for Improvement**\n\n* 1\\. **Detailed Discussion on Limitations**:\n\n    - The authors should provide a more comprehensive discussion on the limitations of their proposed methods. This could include:\n\n        - A deeper exploration of the computational costs and complexities associated with implementing ComboTs and AdaSCP.\n\n        - An assessment of the scalability and generalizability of the proposed methods across different datasets and federated learning frameworks.\n\n        - Potential challenges in deploying these attacks in real-world scenarios, considering practical constraints and resource limitations.\n\n* 2\\. **Proactive Defense Strategies**:\n\n    - Include suggestions or preliminary evaluations of potential defense mechanisms to counter the DataStealing attack. This would provide a more balanced view and contribute to advancing the field by not only identifying vulnerabilities but also proposing ways to mitigate them.\n\n* 3\\. **Ethical Considerations and Responsible Disclosure**:\n\n    - The authors should discuss the ethical considerations and responsible disclosure practices followed in conducting this research. This could include:\n        - Steps taken to ensure that the research does not cause harm.\n        - Collaboration with organizations or communities to address the identified vulnerabilities responsibly.\n        - Suggestions for policy or regulatory measures to protect against the misuse of such attack methodologies.\n\n* 4\\. **Broader Societal Impact**:\n\n    - Expand on the broader societal impact of the research by discussing:\n        - How the findings could influence the development of privacy-preserving technologies in federated learning.\n        - The implications for industries and applications relying on federated learning, such as healthcare, finance, and IoT.\n        - Potential benefits of raising awareness about these vulnerabilities to foster more robust and secure machine learning practices.\n\n**Conclusion**\n\nThe authors have made a good start in addressing the limitations and potential negative societal impacts of their work. By providing a more detailed discussion on these aspects and incorporating the constructive suggestions above, the authors can enhance the comprehensiveness and responsibility of their research. This approach would contribute positively to the field and ensure that the research is conducted and presented in an ethically sound and impactful manner.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the privacy risks of diffusion models in federated learning (FL) through the lens of data stealing attacks. The authors propose the ComboTs method to target a large amount of images. To further defeat advanced distance-based FL defenses, the authors propose the AdaSCP attack method which adaptively scales the critical parameters in the updating process. The evaluation results demonstrate the effectiveness of this attack.\n\nSTRENGTHS:\n- The first work to consider the privacy risks of diffusion models under the FL setting\n- Good performance with low MSE of the recovered images\n\nWEAKNESSES:\n- Problematic attack settings\n- Insufficient experiments and explanations\n\nQUESTIONS:\nThis paper explores the data leakage risks of diffusion models in federated learning. The authors conduct the data stealing attack via backdoor attacks. To target a large amount of images and bypass the advanced FL defenses, the authors propose the ComboTs and AdaSCP methods, respectively. The evaluations showcase the effectiveness of the proposed data stealing attack.  \n\nHowever, there are still some critical issues.\n\n- The attack settings are problematic. In this paper, the attacker is assumed to have access to the training dataset of the poisoned client, such that the attacker can insert the backdoor triggers into the target images, and finally the attacker will be able to recover the target images according to their backdoor trigger patterns. However, since the attacker already has access to the target images, why bother stealing them using the proposed complex methods? Just because of the so-called \"Under strict data privacy protection, attackers can not upload or download images from the infiltrated client\" in Section 3.1? This assumption does NOT make any sense. The attacker can even check every pixel value of the target image to locally reconstruct it without downloading it from the client. I would suggest the authors clearly re-explain the motivation and the settings of the attack. \n\n- The authors mentioned the FID metric to measure the performance of diffusion models. However, the authors did not clearly claim either higher or lower FID is better. Furthermore, when discussing the evaluation results, the authors only focused on the MSE score. I would suggest the authors elaborate more on this metric if they use it as an evaluation metric. \n\n- The authors only conduct the experiments on two image datasets with low resolutions. I would suggest the authors also evaluate the performance of their attack on another dataset with a much higher resolution, e.g., a subset of ImageNet.\n\nLIMITATIONS:\nSee my detailed questions", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-13"}